Blitzy and Autonomous Software Development for Enterprise

Codebases in enterprise organizations often span hundreds of thousands, or even millions, of lines of code. In many cases, changes must follow established patterns precisely, or risk introducing costly security or operational issues. A common challenge with AI coding tools is that they struggle at this scale: LLM context windows cover only a fraction of a large codebase, and output quality degrades the longer an agent runs. Blitzy is an autonomous software development platform for the enterprise. It starts by understanding your entire codebase, building a knowledge graph that acts as a shared source of truth. From a well-defined spec that engineers review and approve, it orchestrates specialized agents that work in parallel to build, review, and test code autonomously at scale. The result is a pull request that engineers can then review and merge Sid Pardeshi https://x.com/sidppar is the CTO and co-founder of Blitzy https://blitzy.com/, and he previously worked at NVIDIA. In this episode, he joins Kevin Ball to discuss why large codebases break today’s coding agents, how a knowledge graph extends what models can reason about, how a reviewable plan keeps specialized agents aligned, and where autonomous software development is headed. Full Disclosure: This episode is sponsored by Blitzy Sponsorship inquiries: [email protected] mailto:[email protected] The post Blitzy and Autonomous Software Development for Enterprise https://softwareengineeringdaily.com/podcasts/blitzy-and-autonomous-software-development-for-enterprise/ appeared first on Software Engineering Daily https://softwareengineeringdaily.com.

• 0:00Speaker 00

  Codebases and enterprise organizations often span hundreds of thousands or even millions of lines of code. In many cases, changes must follow established patterns precisely, or risk introducing costly security or operational issues. A common challenge with AI coding tools is that they struggle at this scale. LLM context windows cover only a fraction of a large codebase, and output quality degrades the longer an agent runs. Blitzy is an autonomous software development platform for the enterprise. It starts by understanding your entire codebase, building a knowledge graph that acts as a shared source of truth. From a well-defined spec that

• 0:43Speaker 00

  engineers review and approve, it orchestrates specialized agents that work in parallel to build, review, and test code autonomously at scale. The result is a pull request that engineers can then review and merge. Sid Pardeshi is the CTO and co-founder of Blitzy, and he previously worked at Nvidia. In this episode, he joins Kevin Ball to discuss why large codebases break today's coding agents, how a knowledge graph extends what models can reason about, how a reviewable plan keeps specialized agents aligned, and where autonomous software development is headed. Kevin Ball, or K-Ball, is the Vice President of Engineering at Mentos and an independent

• 1:26Speaker 00

  coach for engineers and engineering leaders. He co-founded and served a CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action Discussion Group through Late and Space. Check out the show notes to follow K-Ball on Twitter or LinkedIn, or visit his website, kball.llc.

• 1:56Speaker 01

  Sid, welcome to the show.

• 1:58Speaker 02

  Hey, K-Ball. Thank you for having me. Just to quickly introduce myself, I'm Sid Pardeshi, CTO, co-founder of Blitzy. I've been at Nvidia since 2016, you know, back when Nvidia's adjusted market price was a few cents, and it was not the trillion-dollar company that it is. It was nowhere near that. And when I told people I work at Nvidia, they said, "NV Media. You didn't get a job at Google, huh?" Nvidia operated so much like a startup, and it was a fun journey. You know, I lived through, I saw the AI wave happen, and I lived through it. And I saw how Jensen was a visionary. During that wave, I realized what an incredible opportunity this is for, you know,

• 2:42Speaker 02

  applying AI to the world of software development, particularly autonomous software development. And that's how Blitzy was born, and that's my peer.

• 2:50Speaker 01

  Well, let's give us a little bit of an overview of Blitzy. What do you all do? And what's the, don't give me all the secret sauce, but like, what's the, the unique take?

• 2:57Speaker 02

  Yeah, so you know, Blitzy is an autonomous software development platform. So people are used to platforms like Claude Code, Codex, which are, you know, more tools, or even tools like Devin or Factory. The way we're different is that we specialize on very large enterprise jobs. Unlike the other platforms, we are enterprise-only, and we have enterprises designing their entire SDLC around us. So what happens with the other tools is that they're really great at taking bite-sized chunks of work that humans can delegate to, right? You give them tasks. Many of them position themselves as junior engineers or interns, like Devin, for example, right? Claude

• 3:38Speaker 02

  Code, for example, is a terminal tool that you can run micro projects with. But the vast majority of the work at the enterprise, about 80% of it, is well-defined work, especially with legacy enterprises, with very large-scale enterprises. It's very well-defined work that was reviewed by a designer, that was reviewed by an architect, was documented, it's ready to pick up, you know, exactly what the acceptance criteria is, you know, exactly what the process is to get from development all the way into production, right? And it's just sitting there, waiting to be done. The humans that need to work on it, they're juggling across models, you know, it's so unpredictable. Like, Office 5.5 just released yesterday, and it saw some of the

• 4:22Speaker 02

  problems that Office 5 had, but Office 5, a lot of people struggled to work with it. It changed so many things about the model. GPT, on the other hand, has a different product, right? So what you have is engineers find themselves optimizing their prompts, building their skills, hopping between tools, trying out the different models that come out, finding new ways to work with them, finding fit between the features and the prompts and the tools that work for us, et cetera. So the other tools, they've changed what engineers are doing, but they haven't changed the fact that engineers still have to do things, and sit there and babysit the tools and the agents to get work done. Where we're fundamentally different is, you take that spec you have, you give it to Blitzy, you press Build, and you're done. You go and do other things. You

• 5:06Speaker 02

  go and use the other tools, you go and do other, more important things, and Blitzy does all of the work for you correctly, exactly as a person who's been on your team for like 30 years has been doing, right? It understands exactly the right way you implement things, because it does this batch learning across your entire codebase when you onboard, understanding and mapping out all of the patterns, relationships, you know, identities across the entire codebase, and then applying those at runtime using the right model for the right time. So we use Office, we use GPT, you know, we use a large variety of models and model sizes, and we always deploy the right model for the right job. It's all about, for us, we're not optimizing for cost, we're optimizing for

• 5:49Speaker 02

  quality, right? What we want to do is, when you get the PR back, it is correct. It is as close to correct as possible. And what our customers have seen is that majority of the times, we are at least, we've saved 80% of the time that you needed on the project to get to production, and the remaining 20% is often because we didn't have some access, you're not comfortable giving us that level of access, we didn't support some integration, or there's just other processes or policies that your team follows before you get something to production, right? So we map against production, we're enterprise-only, and we focus on doing all of the work one-shot. That's how we're different.

• 6:26Speaker 01

  Awesome. I want to dig into a few pieces of that, but first, let me say, I love the focus on quality. This is the thing I constantly challenge my team on in my day job of like, hey, a lot of folks are using these tools to pump out a lot of things faster, but at a lower level of quality. How do we do the same amount of work at a higher level of quality? What do we do to up the level with it? Let's maybe start a little bit with what the enterprise-only ends up meaning, what you get to do differently because of that, right? So, you know, if you look at an enterprise customer, what differentiates them from selling to a developer or a small company?

• 6:59Speaker 02

  Yeah, very, very good question. So there's, I would say there's two fundamental things, and we'll talk through more as we identify them. One is that the challenges and the kinds of codebases an enterprise is looking at are very fundamentally different than someone who's working on a hobby project or vibe coding, right? So one is the size, two is the expectations and the standards, right? Like, if I don't follow a style guide on a hobby project, that's fine, as long as it works, I'm not going to complain on a daily basis. And the enterprise, if I make a mistake in the style pattern, or if I make a reckless change that ignores a pattern that's well-established

• 7:42Speaker 02

  and required, or I make a compliance error, or I introduce a security vulnerability that, you know, doesn't take into account all of the constraints you have, or even a load, you know, a load-balancing defect, right, or a performance concern, that item, if I, if I introduce that with my, you know, shortcutting and not following standards, that means material cost in dollars as the code progresses through steps, right? Someone has to spend more time reviewing it, someone has to spend more time testing it, and then ultimately, if it reaches, if it gets to production and the customer is impacted, the business has lost dollars, right? So the stakes are really, really high. The complexity is high, right? It's, you're not looking at a 10,000-line

• 8:26Speaker 02

  codebase or 20,000-line codebase. You're looking at at least hundreds of thousands of lines for a small enterprise, and tens of millions of lines for the largest enterprises, just for a single product, right? They have multiple codebases at that scale, that they are actively maintaining with, you know, hundreds of commits and thousands of commits, right? So the scale, the complexity, the standards, they're all really high. The sizes of the projects and the features you're building are also very high, but it's a very high tax. Like, for example, if I have to make... A twenty line change in a codebase, right? I may think that I could get away with, let's say, a simple agent that maybe it's a

• 9:11Speaker 02

  sets thinking to medium and gets the job done, right? But the requirements of that enterprise is that we've already added the libraries or the logic that does this. And that logic or that pattern is buried deep into millions of lines of code, right? And we don't want you reinventing the wheel for that. So finding that just at the right moment and then doing it exactly the right way, that's the hard part, right? That's the big challenge that we're pushing ourselves to solve day and night and we've solved to the most extent.

• 9:43Speaker 01

  That problem is hard for humans, right? To map out millions of lines of code and understand where it is. So let's talk then about the start of the process, right? Because you mentioned a little bit, hey, we go and we ingest a lot of what's going on so that you know, oh, what is that, you know, one library off in the weeds that should be used for this? So what does that look like at Blitzy?

• 10:06Speaker 02

  Yeah, so, you know, if you look at how far models have come, right? Models are really intelligent these days, but whether they fall down, they fall down at the context level. They have one million tokens of context, which maps to about one hundred thousand lines of code. But here's the kicker, right? Every token in the context window needs to map to every other token by a dot product in the matrix. And there's no amount of compute or, you know, trick that you can apply that does this effectively on across the entire context window. So most of the model providers that you have there, Anthropic, OpenAI, there, they have the effective context up to less than three hundred thousand tokens, right?

• 10:51Speaker 02

  And they're applying techniques like sparse attention, you know, sliding window attention or something else across the rest of the window. So this is also why, for example, now this is not the official reason, but if OpenAI has a pricing differential at two hundred and seventy two thousand tokens, when you send more than that, it's a long context request, you get charged more, right? But because of that fundamental gap, as soon as you send more than thirty thousand tokens, more than three hundred thousand tokens in a request, which maps to thirty thousand lines, intelligence degrades. So that's why, as you run these single agent tools, like for example, the CLI tools or the single agent persona tools, quality degrades as they are more long running, right? The encouragement from all the providers is to do short running jobs.

• 11:37Speaker 02

  Now, the way we solve this is that we invented a graph database system that maps your entire codebase by identifying its relationships and understanding what every file in the codebase is doing and understanding the structure, like, okay, this codebase has twenty thousand files, but here are the modules that are structured, right? Here's what this file is doing in this module. Here are the features that live in different parts of the codebase, right? If I'm looking for authentication, these are the functions and these are the patterns that the enterprise is using across the entire codebase. And we do that free run in about twenty four to forty eight hours when you start working with us. So it's not like you turn on the terminal and you start working, right? Just like Claude code.

• 12:22Speaker 02

  We, you have, there is a cost and there is a free run. You don't like, you know, you have to go through that free run. But the benefit is we've now mapped out the entire codebase and represented it structurally, right? So this graph database lets us, so for example, let's say you want to find a pattern in a codebase that's thirty million lines, right? And there's tons of products, there's so much noise. So Claude, Code, Curse, any of the tool, they're going to try to use grep or some variant of grep and try and find it. But there's so much noise. So the agent is going to burn tokens and hopefully find the right one. But what we've seen in many of these codebases is sometimes there's deprecated stuff in there that's not, that's not even marked deprecated, right?

• 13:05Speaker 02

  So how do you know that you found the right item without actually profiling the code, without actually running the code, without having reverse engineered the entire codebase? So that's exactly what Blitzy does first. But reverse engineering it and saving those markers in the graph database is contextualized, right? Like even memory, for example, right? Memory for the other tools are files strewn across the codebase. You don't see it because it's in Claude's projects folder, but you can go and see it if you like. But then finding the right memory for the right module or file at the right moment, that's the hardest part, right? So by having the graph database represent that structurally, semantically and naturally, you have a massive advantage.

• 13:45Speaker 02

  So the magic of Blitzy is really just making it easier, drastically easier for these models to operate within their effective context window effectively.

• 13:55Speaker 01

  Yeah, no, I'd love to dig in one level deeper without, you know, asking for any secret sauce on this, but I think, you know, I've been experimenting with an open source graph database tool for my coding agents and it does make a huge impact. So thinking about that, you sort of alluded to, it's not just, you know, the structure, like you could use a deterministic tool, compile this to an AST, map the dependencies and have that surface that up. But there's more that you're talking about here because you were talking about, oh, we're putting some weights on this. We're saying, oh, this one's actually, it's not marked deprecated, but it's not actually used. So like, what are the different signals you're looking for and how do you expose those to the agent?

• 14:31Speaker 02

  So we're looking for, we're actually trying to build and run the code, right? And we're actually mapping out what is the piece of code that's used based on what the agent can observe. We're looking at not one file, but the entire codebase and right? And that like, at that scale, when you have like twenty thousand files identifying which file is relevant and which is not, it's hard, right? So there are, for example, graph database solutions. There are even open source versions of those implementations, but it's really, really hard to solve for the entire codebase. And that is what we've invented and solved, right? Inventing and solving and building a system that is reliable at scale and works for any

• 15:13Speaker 02

  language and is actually able to pull, you know, that context reliably and make it helpful for the agent and not actually distracted.

• 15:22Speaker 01

  Oh, absolutely. Like that, that's like what I'm wondering about. And, you know, as I said, without going into secret sauce, like what, like, what does that query approach look like? Are you exposing a full graph language? Is it a sort of higher semantic tool? Like what, what does the agent actually interact with?

• 15:39Speaker 02

  So the agent has tools that let it, you know, search the graph database. So for example, we also employ semantic summarization. So, you know, we save embeddings of summaries and let agents search that. The agent is writing cipher queries that is traversing the graph. It's identifying, you know, what is the best way to query this. So the agent has an understanding of the graph database's schema, right? And it knows the kind of information that it can get and the kind of information it can save in case it finds an inconsistency, right? If something changes, what happens? How do you update the graph? That kind of stuff. So giving the agent access to the schema as well as access to the kinds of queries

• 16:24Speaker 02

  it can make and tools it can call, that's what makes all the difference when, at inference time, right?

• 16:31Speaker 01

  Yeah, awesome. All right. So that's anything else interesting from the kind of ingestion process? How are you pre-processing this code to give your agent like full knowledge that we should talk about?

• 16:42Speaker 02

  So I think memory, as I said, is one of the most important items. What happens with memory is in today's world, memory has to be earned. So you have to have used the product to be able to earn that memory, right? And then save it. And so you have to get hurt to know how not to get hurt that way. But magic happens when you're not able to get hurt in the first place because you learned from someone else's memories, right? So a few ways we approach this. One, when we look at the commit history and we try to understand what journey has this code been through in getting here, right? And how can we imbibe that and follow that process so that when we write something,

• 17:25Speaker 02

  it lands on the same planet that the customer expects, right? The others is the customers often use agents dot MD, skills dot MD, Claude, et cetera, learn from those, apply those, but again, identify the context of when they are actually relevant, right? So saving that information, understanding the tech stack, summarizing that, compounding that over multiple levels, that starts paying dividends as you start going, right? You can make fewer mistakes, right, with the customer's input because, you know, to be fair, humans and or users are not the best as at giving it the most clear instructions, right? So to really deliver value, you need to be able to understand and interpolate instructions that were never provided.

• 18:11Speaker 01

  Yeah, that's super interesting. I've done some amount of like, oh, let's analyze my previous sessions. What do I get out of that, things like that. But you're looking across essentially their entire history of this project and saying, oh, where were the things that had to be reverted? What were the missteps? What was the journey? So during that process, can we talk a little bit about like, what does the, are there like certain sets of signals you're looking for? What does the compiled version of this look like? You know, is it coming down to a set of guidelines or is this another graph of searchable practices? Or like, what does this look like?

• 18:43Speaker 02

  We're embedding this into the actual graph. So we try to replicate the structure of the entire codebase. And the problem that you have is, as you correctly described, you have, you can create multiple graphs, right? You can create a graph of the commits, commit history, you can create a graph of the file structure, you can create graph of the folders, et cetera.

• 19:05Speaker 02

  But the problem is how do you tie all that information together? And then how do you provide that to agents in a digestible format where they can operate efficiently, quickly? What we're doing is we, but we use a structure that maps to the structure of the codebase. And then we actually process all of those commits, you know, as much as we identify are relevant. And we apply the learnings from those commits to the nodes of the graph. So you don't have multiple graphs that you're trying to, you know, deal with. You have all the information contextualized. And another unique aspect, something that we perfected really early on where the rest of the industry is about, is has yet to catch up.

• 19:45Speaker 02

  It's not even being discussed, is running agent swamps without a coordinator. So the existing model is you can run. You know swarms with there's one coordinator and tens of agents, right? But there's not, in fact, research, current research has proven that if you run multiple agents in parallel, it's really hard to ground them and to get them to not overlap and do things, you know, solve the shared goal because the other agents can have multiple opinions. We run all of this processing in parallel and use multiple coordinators at different levels to achieve the shared goal consistently and reliably, right?

• 20:27Speaker 02

  So there's a different agent, for example, that's focused on crawling the graph. There's another agent that's focused on integrating that just in the right place and handing off to the next agent, you know, at the right point. So that I think is one of the hardest problems that we had to solve.

• 20:41Speaker 01

  Yeah, so let's dig into that a little bit because as you highlight, right? If you, you can have a single centralized orchestrator, but that breaks down fairly quickly. How do you deal with collisions, ownership, handoffs? Like, what does that look like in your, your agent swarm?

• 20:57Speaker 02

  So we use the graph database as the source of truth because that changes everything. So what happens is if you can use, for example, disk, but disk is volatile, right? A- as soon as I change the folder, as soon as I kill the pod or destroy the location, it's gone. You can use Git, but you could destroy the beauty of the commit history and you can have a very cluttered workspace that makes it incredibly hard to find information. It's one thing to dump information, it's another completely different to retrieve it intelligently, right? So the graph database helps you do both by, you know, structuring it on the graph, by contextualizing it at runtime and then structuring it on the graph. What you can now do is use that structure to recruit agents.

• 21:41Speaker 02

  So for example, if your graph is based on folders, you can assign agents to folders and limit scope. Once you have a scope document, you can break down the scope and then distribute it based on a structure that's anchored on the graph. So what that does is it, it couples us to the graph, right? Whatever we do has to be on the graph, which is a trade off. But the benefit is that you can run as many agents as you like. There is no theoretical limit to how many agents you can run in parallel as long as you solve the compute behind.

• 22:15Speaker 01

  So let me dig into that a little bit more. So we already talked about when you do your analysis of history and things like that, you're attaching that to the graph. I'm assuming that's some sort of either metadata or a distinct type of node attached to the code node in some form. So now you're saying, okay, we're going to also attach tasks or desired transformations or something like that also to the graph. And now those are claimable and, and agents can just come in and say, oh, I, I can do this part. I can navigate this. Does that require you to deal, like, what is the synchronization primitive? Oh, I have a dependency here. I need this part of the graph to be complete before that can go. Like, does that mean I have to claim this whole piece? Or does that mean, you know, I do this and then someone else can claim that?

• 23:00Speaker 01

  Like, what does that end up looking like?

• 23:02Speaker 02

  Yep, that, that is another hard problem we had to solve. So we have dedicated steps for synchronization where, you know, if you've completed. So all of this is pre-planned. When you're running a job, first figure out what is the review process? So for example, with any other tool, you have to, Claude just finished working on the tool. Now let me do a code review. And then, okay, the code review is done. Let me use, let me do a security review. A security is done, let me run a test agent that tests everything. I have to ask it to do this every time, right? It's not going to do it on its own. For us, we do that out of the box by default. You don't have to ask Blitzy to test what it did. And again, whether or not it needs testing is completely dynamic. That is also agent determined, right? So by deciding what is the path to production, right?

• 23:45Speaker 02

  By being an enterprise only product, what you can now do is just focus on production, right? And then you decide what the checkpoints are, what the stages are for synchronization, when you should do it, what you should be checking at that point, and what are the expectations and acceptance criteria at each of those stages? And then you can just drive the agents through it until, you know, you have something that is shippable, at which point you can hand off to the human.

• 24:11Speaker 01

  Yep, so let's, let's walk through that process a little bit more, right? So if I think about, you know, and, and because I'm not deeply familiar with Blitzy, I'm like mapping this to my mental model of an orchestrator. So right, I take, let's use the example you use. I have a, an orchestrator handling my, you know, handing things off to Claude code. It says, hey, go and implement this. Okay, I got back, it's implemented, hand it to a reviewer. What does that review phase look like for you? Is it also somehow tied into the graph? Like, what is, do you have to have versions of your graph? Because now you're saying, oh, this is in this branch, it's still under review. Like, what does that end up looking like?

• 24:48Speaker 02

  Yeah, so we anchor on the SCM for progress. And so that is the progress in terms of the work done. And there's the agent work stream that is in flight, right? And there's a scope of the review of the, you know, based on the expectation of what should be complete at this point. So the review as a process is anchored on the graph, the results of which are committed onto the SCM. And so throughout the process, you know, when you, when you have a workflow that needs to be run by the agents, that is mapped and tracked onto the graph. And again, the results are captured into the SCM. So in case you pause the system at any point, you can reconcile both and know exactly where you are in the overall journey.

• 25:34Speaker 01

  And does that then continue on through, you mentioned a security review. What does security review look like in this context?

• 25:39Speaker 02

  So security review, the, the thing is you don't want to be writing any code that introduces security vulnerabilities, right? With the mythos scare that everyone had, especially because we were working with banks, you have to guarantee that the code that Blitzy writes is free from issues. So by default, you have, you know, another set of agents watch the works that's being done, catch security issues and schedule security checkpoints to make sure that, let's say those issues were temporary, maybe it's, it was a scratch that the agent wrote and it lasts to discard that later. So let's make sure that happens and that gap doesn't make it into production. Or two, when we plan this feature, you're introducing a new API, I got to make sure that it follows the existing authentication patterns,

• 26:23Speaker 02

  that it, it, you know, it follows load balancing, it follows all of the core items that you follow, you know, not just from security, but also from a housekeeping standpoint, right? So let's do that testing, let's do adversarial testing. Let's try to break the system and product before someone else tries to do it, right? And let's flag that, maybe there's a design gap that no one else saw, right? So it's, it's Blitzy's job to make sure that it exposes that. If that item is out of scope, it documents that. If it considers that this should be addressed as part of the same job in scope, it will go ahead and adjust that.

• 26:57Speaker 01

  Documenting is interesting. So I think one thing to map back to like a human process, as you're doing work, you often uncover future work, right? You're like, oh, this is out of scope for this project. I don't want to explode my scope, I don't want to explode my surface area, but like this, I, I found this, we should really fix this.

• 27:15Speaker 02

  Yep.

• 27:15Speaker 01

  Does that end up getting resurfaced to a human to decide? Does Blitzy schedule itself future work to do those cleanups? Like, what does that end up looking like?

• 27:25Speaker 02

  Yep, you're onto something here. So we're working on the ability for Blitzy to file tickets and schedule itself and to be given a goal and, you know, recursively self-improve itself to achieve that goal. But what we do at the moment is at the end of Blitzy's work stream, we have two things. One, a project guide that documents all the work that Blitzy did and all the work that it could not or decided not to, right? So that includes these findings and that also affects the completion percentage if it was an in-scope item. The second item is the PR review experience where it walks you through all the changes that it did, but just focuses on the key changes. So you don't have to review 500 files, you can review like 50 files and you understand the work that was done.

• 28:10Speaker 02

  Because if I have 100 files just with unit tests and you know they're all green and you know how they were written, you don't need to review my unit tests, right? You, you're focused on getting this to production. So using those two items, a user is able to feel more confident while approving the changes, reviewing it and move faster to the review process.

• 28:31Speaker 01

  So that that kind of raises another interesting question. I feel like one of the things that that many of us are struggling with right now with AI-generated code is how do we keep our mental models of our systems up to date? Right? It's one, it's great to have it working, but if I want to understand how I can evolve the system well, I need to understand the underlying structures. And I think, you know, in an enterprise setting, you have a lot of existing structures. The highlight Blitzy is very good to kind of respecting those. But sometimes things need to evolve. Sometimes things need to change, and especially if you have a long-lived code base, there may be pieces in there where like, hey, this is hairy. We don't want to build in, even though everything is depending on that.

• 29:12Speaker 01

  New code should not look like that. We want to shift our model. We want to do things. So kind of what? How do you handle the like, keeping the brains of the humans up to date with what Blitzy is doing for them?

• 29:24Speaker 02

  Yeah, there's multiple, you know, strategies. So for example, especially we run into that that use case a lot where, you know, I have this, but don't do that. That was, that was a dark past. I don't follow that. So again, even when we look at commits, right? It's about recency. You may have one pattern earlier on, maybe years ago, and you maybe have moved on to a new pattern or a new technology framework more recently. So recently, recency is weighed into the, you know, factorings of that agent that applies patterns. But other than that, we have knowledge base, which is one of the features, but as you onboard and give Blitzy more code, it documents all of those branches. And it's understanding of those branches we expect, right? So it's a living document of what that code contains.

• 30:10Speaker 02

  So you can look at it and as you update and resync code, that document gets updated. So if you need to understand what any branch in your entire code base does, you can look at that knowledge base and you can understand it. And then the more tactical items that, you know, our forward deployed engineers work with customers on is creating documentation and keeping that documentation up to date using Blitzy. Right? So what is the current state? What are the best practices before running a massive project? What makes the most sense is we follow this trial, walk, run approach where we suggest that you start with documentation. Let Blitzy confirm its understanding of your entire code base. Let us reverse engineer and let us show you what that looks like.

• 30:50Speaker 02

  Move on to adding testing, right? Because how do you validate the work that Plitzy did or anyone else did, right? So let's increase test coverage and let's, let's add that. And then what, what, what those two things do is now it's very, it's much easier to, to describe acceptance criteria. Right? It's much easier to explain to a human what that code does. And it's much easier even for Blitzy, once the human has actually reviewed the documentation to validate its understanding and make sure it's on the right track. So by doing that, and then again, if there, if there are any caveats by capturing that in the agent action plan, sometimes there are conflicts to what the human thinks is in the code. And what they asked us to do, or even in their own prompt, right?

• 31:33Speaker 02

  So flagging that early and making sure that's addressed. So like some of the tactical things that you can do to make sure that the cost of this item is reduced.

• 31:41Speaker 01

  Yeah, that makes sense. Well, and it's, it's interesting again with the enterprise context, because, you know, long-lived companies often have parts of their code base that are poorly understood at best, maybe behave in odd and unexpected ways, but then there's downstream dependencies. I think my favorite definition of legacy code is untested code, right? And so this idea of, hey, first, let's, let's validate our understanding, both with documentation, but like, you may not have anyone who can verify that documentation. So let's, let's write tests that show what the system actually does.

• 32:17Speaker 02

  Yeah, I think our favorite moments and the customers that love us most are the ones who have a lot of legacy code. And a few people who can understand it. So it feels amazing when, you know, you have architects who have been there for like 35 years. You have COBOL developers who no longer write that and having them validate the code and, and, you know, really express satisfaction that it's easy to understand and operate in a way that they would. I think that's the best part of building and working with Plitzy.

• 32:48Speaker 01

  Let's talk about another thing you mentioned here, which is how course corrections happen, right? No matter how good the system is, there will be mistakes, there will be stuff, and you have this human review process to, to check and verify. Let's say something goes wrong. What does the correction structure look like? How does that get parsed, represented, et cetera?

• 33:07Speaker 02

  Yeah. So I think you bring up a very important point and part of my frustration with, you know, working with other tools has been that you blew up all of the context writing the project. It's like 700K tokens and it's done the work. And now to do the next task, let's say you have to compact. So what happens is that it forgets many times the work that it, even though you have the plan, right? The plan is like a two pager. The work is maybe tens of thousands of lines, if not hundreds of thousands of lines. How do you reconcile that, right? It's, it's a huge problem. So again, because all of the work that Plitzy does follows the graph, right? The graph that it updated on the fly corresponding to the branch, we have a massive advantage.

• 33:49Speaker 02

  The way we operate is you saw the PR, you found some gaps. We have a workflow called refined PR. And what that does is that it resumes the process, including the graph, it preserves the history, preserves the history from both the graph and the SEM and addresses your feedback. And the best part is that we build on lines of code, right? So we don't charge you for mistakes that we make. Everything that you give us feedback on is a mistake. So what that means is if by addressing your items, lines of code in your resulting PR went down, you will only be charged for the final lines of code. So let's say Plitzy went off

• 34:32Speaker 02

  and wrote 500,000 lines of code and when you were only expecting 300,000 and then.

• 34:36Speaker 01

  Sounds like a Claude project.

• 34:38Speaker 02

  Yeah, recent, recent Claude project, although I hope they, they fixed it with the recent models. You found that 200,000 lines were just comments and fluff that you don't agree with, that you didn't ask for. Fine. You know, you can ask it to clean that up and then you only get billed for the correct answer. So that I think is very unique and special for Blitzy. And I think is the right model for the enterprise, because there's a lot of token anxiety these days. The tokenomics and the fluff of, you know, let it rip, burn all you can is all gone, right? Enterprises know that they need to be cost-conscious with these things, because if you let it run wild, it's going to, the wheels are going to skyrocket. And I, I believe we position our product and our pricing exactly with that idea in mind.

• 35:24Speaker 01

  Yeah, I like that a lot because it aligns all of your incentives towards like, let's write the right code, not all the possible code. A couple of things that that actually brings to mind. So one is sort of the convergence problem. And this is a problem that anthropic models have been more vulnerable to, but I've seen it on, on GPT models as well and, and other things, which is, you know, agents love to find one more thing to do. So you go, you build a thing, review it. Oh, you haven't thought about this thing. Okay, right. Okay. I'm going to go and build it. Okay. You haven't thought about this. And eventually it's like telling you, hey, once every Tuesday in July, like this is going to break in this way. So like, how do you within Blitzy say like, this is the line. This is what's enough. We're done.

• 36:10Speaker 02

  I think you, you highlighted like a very, very important problem. It's not just, you know, anthropics. It's, it's, it's a problem with a wide variety of models and people believe it can be solved. You know, prior art things that you can tweak it with, let's say, prompt tuning or with effort tuning, you know, hey, if you think it's doing too much, just drop the effort level. But it doesn't really solve the problem, right? Because the way these models work, if there's one thought in there that is going to let it to think about finding new problems or have it document like one thing it found, you can't go back and change that unless you really edit the conversation, which is not something a regular user does, right?

• 36:50Speaker 02

  So it's a really hard problem. But the way we solve it is at any point of time, we use different models. The beauty of using different models is that you get fresh perspectives and you can align on the stronger model to be the tiebreaker. Right? So we have internal evals and benchmarks. We know which model is relatively stronger for decision-making or which model is relatively stronger for front end. And then so on and so forth, right? So by applying that at runtime, so even if Claude thinks that two things were flagging or this item is load-bearing in typical Claude speak, you have GPT come in and check and say, nope, that's not in scope. That's not what the agent action plans is.

• 37:31Speaker 02

  And GPT is more effective because it's doing less, right? Claude has the burden here of implementing everything. It has blown up its context. It's operating probably in its, in the ineffective side of its context window, right? It's loaded up with more than 300K tokens. GPT in this case was only looking at the AAP. And because it had to deal with much less constraints, it did a more effective job of identifying the scope. So this is a very vague example, not what we actually do, but we do something very similar across many models. And we arrive at the correct answer to what the scope should be. So before the user sees it, that job has been seen by at least eight different models.

• 38:15Speaker 02

  And all of those models, you know, are specialized at the task that they do, and they've been proven to be the best at that particular task, right? So imagine switching across eight different products locally to do one task, right? That's, that's painful. It's not something you want to do and you're going to have to maintain your environment and prompts and instructions for all of them, right? You don't have to do that because you use Plitzy.

• 38:36Speaker 01

  So that is, that is interesting and a nice segue into another question I had. So you're multimodal, which is great. I think that's definitely an advantage being independent from the, right now, a lot of the harnesses are like tied to a particular model set and things like that. So being independent, letting you pick and choose is phenomenal. How do you decide which models to use for what? I think you mentioned internal evals, like what does that process look like and how does it evolve?

• 39:02Speaker 02

  Yeah, so we work very closely with labs, model providers, you know, influence providers. And whenever a model is launched, we run it internally, benchmark it, we run it in the platform, we test how it's doing, we score it. Right, and then we apply those learnings to decide which is the try breaker. But at runtime, the process is entirely dynamic, right? We have these ladders and rungs. We have dynamic selection of the models. We have switches and routing in place based on capacity, you know, based on intelligence, based on task specificity. And the agent chooses the L L M for the next agent based on the task. It chooses the system prompt, it chooses the tools, right? So this is literally agent designed by agents, right?

• 39:44Speaker 02

  And what happens is as the agents and the models get better, it pays dividends because they're more precise. They're more effective at selecting the right model for the right job, right? So to build that engine, it took us years, right? And now we've perfected it. So it just, it's always, it's really, really good at defining even the guardrails, right? Here's what's out of scope. And here's where you can check with me if you have questions, right? So rather than, the agents are trained to go back to the human to ask questions. But adding these, you know, adding this framework in there just helps us do autonomous work, right? Where you don't have to go back to the human.

• 40:21Speaker 01

  Yeah, so that's interesting. I'm curious about that, you know, having the agent choose the model, which I think is, is a really nice, you know, self improving approach. But what do you use to describe the models to give the agent sort of dimensions on which to choose, right? Like if I just say, hey, your, your options are Opus five, five or G P T five, six, like it doesn't know what those are. Maybe it does, but like, what are the kind of features that you're calling out about those models for the deciding agents?

• 40:50Speaker 02

  Yeah, so we ignore all the fluff that the, that the more, the labs claim about the model, you know, like one lab may think Model X or X Y Z is, is amazing at something, right? Or the benchmarks may say one thing. So that's why you don't allow a web search for something like this. But instead, all of the results of the benchmarking and the evals that we had internally that identifies the strong points, you distill that into tags, right? And you distill that, you rank the tags, like for example, front end, you have a ranked ladder of which models to choose, right? Based on complexity. So front end, very complex, choose this model. Front end, low complexity, choose this model, it's good enough, right? And then so on. So because you're, you're optimizing for quality and speed, right?

• 41:33Speaker 02

  You need to choose the fastest model that gives you the best quality, right? So for that, we've built our system to be intelligent enough to make the right decision at runtime.

• 41:42Speaker 01

  That's really nice. So you're kind of mapping out what is that Pareto frontier along a number of dimensions. I think, yeah, front end, back end, language, complexity, those make sense. Okay, awesome. Is there anything that we haven't talked about, about Blitzy today that we're missing that would be worth talking through?

• 42:01Speaker 02

  Yeah, let's talk about sandbox, right? Which is the new plan we have. So we wanted to make it more accessible to enterprises and which is why private sandbox is what we've introduced. And what that means is if a company has more than two fifty employees and you sign up with your work email, you automatically get access to Blitzy with a set quota and a significant amount of lines of code to go to reverse engineer and to write. It's a, it's, it's a decent code. I don't have the numbers handy and they keep changing, so I won't cite them. But it's decent enough to do a small to medium sized project, right? Depending on what you scope it. And you could use that to build a completely new product, which is very easy to get started, right? You don't have to code your environment in or try to get it to understand your code base. There's no waiting, just get started right away.

• 42:46Speaker 02

  Can easily evaluate how good Blitz is. Or bring in a piece of your code base. We don't train on any of it. We don't, you know, we have very strict data retention policies and all of that. So it's an enterprise gate platform, right? We don't, we don't do anything sketchy. Your code is owned by you, et cetera. We don't store any of it. So you can run that project through Blitz and experience the value, right? When you get the P R, when you go through the review experience. So it's in, it's a quick way to expedite the process where you don't need someone to handhold you. You can experience the value and then speak to our team if you'd like to move forward. I think it's a very generous offer. I don't think that's available with any other tool. So we really hope that a lot of other enterprises get to experience the value.

• 43:29Speaker 02

  And we're really confident about that, which is why we're doing it.

• 43:32Speaker 01

  That's awesome. So let's maybe now shift and look forward a little bit, right? So we've talked a lot about what Blitz does. I think there's a lot of interesting things that to pull out of that, whether you're in an enterprise where you can apply Blitz now or you're considering or trying to think about how you're managing your own tools. What's like the, the frontier? What are the things coming down the pipe?

• 43:55Speaker 02

  Well, I think the frontier is this two wild visions of the frontier, two broad buckets. I think there, there's some people who think that what's going to happen going away is code review as a process is going away. Software engineering is dying. We're not going to be writing any code. We're not going to be reviewing anything. We're just going to ship. I think, I, I find that personally outrageous, especially when you're talking about the enterprise. I don't think that's the speed at which the compliance frameworks evolve. Or, you know, that, that level of comfort is what the architects who, who actually use the models on a day to day basis have, right? I think they have a much stronger and clearer understanding of what the models are and are not.

• 44:34Speaker 02

  And to be fair, they're, they're on the spectrum of clarified autocomplete to A G I. They're, they're not on either end, they're somewhere in the middle. At the end of the day, they're sequence, sequence models that are predicting the next token, right? But they're really, really good at real world use cases, right? So where does that leave us? If you go back to the beginning where I said, most enterprise work is very highly defined and can be captured in a spec. And that's how enterprises have been building software for generations, right? We see the opportunity for the enterprise to hand off that work, submit the job, do something else.

• 45:17Speaker 02

  Do more complex discovery work, do more complex research work that you cannot hand off. The requirements are not clearly defined. And the models are amazing at that, right? So the tools that you have, Cloud Code, Codex, et cetera, they're amazing at helping you do that work, right? And the well defined work gets done by autonomous tools like Blitz, right? And these tools, the next frontier is like recursive self improvement, right? So you can set an autonomous system, perfect something. Like I want to ensure that performance for my e-commerce website, my page always loads in less than a second. I have a very large team, they're all shipping tons of commits every single day, but I cannot regress on this one metric.

• 46:00Speaker 02

  And it's very hard for me to enforce this with every single committer. So what I do is I set the system behind it and this system monitors and makes updates every single day, floating up for human review dynamically when it needs it. But it achieves that goal, right? And you can, and we see that growing into not just, you know, distinct goals within the enterprise, but to entire products, right? If you can capture the product objectives and the metrics, right? And the standards that you want to maintain and achieve, then what you can have is autonomous systems like Blitz. Run these products that are very well defined, that only need human supervision once in a while.

• 46:44Speaker 02

  And that can be easily managed, right? At the same time, shift the human attention to more complex items, to products that we need to be far more dynamic, where you cannot trust an L L M. Where you need research and you need to do a lot more auditing and then compliance checks, et cetera, and focus the humans on that. So it's an incredible opportunity for every player in the system because the market is big enough to accommodate everyone. The software, you know, software engineering is such a huge, huge market with such an elastic demand. That it's a really incredible time to be alive and, and be an engineer in the space.

• 47:20Speaker 01

  Yeah, no, absolutely. Actually, one thing you did, you mentioned there that I want to dig a little bit. So in this view of almost self managing or autonomously managed projects, where do things like runtime environments and logs end up? We didn't end up talking about that for Blitz, but like, are you feeding back operational data in? Like, how does that end up working?

• 47:40Speaker 02

  Yes, so Blitz is connected to the cloud, you know, your, your cloud, essentially A W S G C P, whatever that is. Blitz has its own environment. So typically when we work with customers, we have them capture their environment variables, secrets, configurations, instructions into Blitz's environment. And then Blitz replicates that environment across projects. You can have micro environments, you can share environments across projects, et cetera. And for, for, for the vision that I talked about, right? That is the first step of getting to work with the system, right? Get it access to your development environment, have it, it operates like an independent engineer. So have it recreate its environment, make sure it's, it's good at that. And our environment is completely dynamic. There's no static item, right?

• 48:24Speaker 02

  The agents set up the environment every single time they operate, they do it dynamically and they maintain that, right? So once you solve that environment problem, once you've given it access to observability logs, and once you've set goals, guardrails, boundaries, right? The agents are incredibly good at optimizing all of those simultaneously to making sure they do the right thing. And if they're unable to do that, to escalate to a human at the right step, to stop and get the humans in, right? So we do that internally and we've been successful at getting customers to trust us and to deliver value by doing that. And so therefore we believe like that is the place where the industry is going.

• 49:04Speaker 01

  Yeah, no, I really like that. And also thinking about things like purity, vulnerability updates, all these different things. L L M's have increased the pace of vulnerabilities being found. But I think autonomous systems like this can also then increase the, the pace of reacting to and fixing those issues.

• 49:21Speaker 02

  Yeah, I think the biggest problem that people don't realize with L L M's is that L L M's have increased the pace of vulnerabilities being found. But they've also drastically increased the pace of vulnerabilities being introduced. And that's because you have to prompt an L L M to write secure code. You don't have to prompt a human, right? A human learns it and remembers that L L M's don't learn things until they're trained, right? And I'm sure labs will continue to work on and get L L M's to write better code, more secure code, you know? But they'll always be, because L L M's get much better at writing code, it's not a lever that you can control. There's not like, write secure code optimized to point nine nine. So this is always right secure, you, you can't do that, right?

• 50:05Speaker 02

  It's, it's, training is very dynamic. You can only control some other variables that are not this. So because of that, it's on the harness and it's on the system to enforce security, right? And which is why we take that very, very seriously. And we have an entire team dedicated to this and we have entire workflows within Blitz. We have, we factored this into the design of how all lines of code written by Blitz are, are, you know, are built. Because once you solve that, once you help the customers get comfort about security, compliance, the guardrails, right? Code quality, that's when you can move to the next level and actually bring in this transformation that, you know, you're, you're thinking about.