
Latent Space: The AI Engineer Podcast
Written by a model from the transcript; not yet reviewed.
This episode covers the concept of software factories with Zach Lloyd from Warp. The guest and host explore how engineering teams are moving away from individual terminal-based coding agents and toward centralized, measurable, closed-loop agentic systems in the cloud. They discuss how a proper software factory is defined declaratively in code, integrated into existing knowledge work tools like Slack and Linear, and optimized through agent traces, online evals, and benchmarking. The conversation highlights the transition from engineering software directly to building the systems and factories that produce software, offering a practical look at modern developer infrastructure.
Hosts introduce the episode and give a brief overview of Warp and its focus on software factories.
Zach explains what a software factory is and how it differs from traditional automation approaches.
Details on how Warp defines factories in code, including repos, skills, MCPs, and workflows.
Explains how Warp evaluates agent performance and uses self‑improving agents to suggest factory changes.
Shows how companies can benchmark different models and harnesses to balance cost and correctness.
Covers the stack layers, sandbox providers, inference, and harness options for building a factory.
Talks about the size of teams that benefit most from Warp and the challenges for very small or very large companies.
Addresses pushback on complexity, subscription models, and the economics of building a factory.
Hosts recap key points, share Zach’s upcoming talks, and provide hiring and contact information.
Software Factories Definedinsight
The guest defines a software factory as an agentic system in the cloud with a certain amount of autonomy, designed to be measurable and closed-loop over time.
Factory vs Arbitrary Automationsinsight
Teams often string together arbitrary automations, but a true software factory is defined in code with concrete workflows, skills, and MCPs per product surface.
Factory Orchestration Platformstrategy
Warp's platform operates on specific product surfaces and repos, executing predefined workflows that handle triage, specking, and implementation.
Centralizing agent conversations and computer use screenshots across the team provides a complete record of what every coding agent is doing.
DORA Metrics and Cost Slicingdata
Teams can track traditional DORA metrics, PR throughput, and slice costs by person, automation, or where they spend money in the SDLC.
Self-Improving Factory Loopsstrategy
Self-improvement agents analyze failure cases and make pull requests against the factory definition itself to optimize the system.
Benchmarking Harnesses and Modelsstrategy
Teams can replay task suites against a matrix of different models and harnesses, producing cost versus correctness trade-off charts on company data.
Internal tuning and testing at Warp reduced the cost per PR from $80 down to around $15 while increasing throughput by 40 percent.
“Like, you should define your factory in code.”
“So for us, we basically have a YAML file, which says, 'Okay, these are all the repos that you can be using.'”
“We measure like the revert rate and it hasn't gone up.”
“And so you basically get a whole bunch of benefits that flow out of it.”
“is a result that looks very similar to like the published results for Sweetbench or Terminalbench.”
Watch the full show on YouTube: https://www.youtube.com/watch?v=LL0YmIyBKSU
6,567 words · 2 speakers
Page updated · transcript and summary are machine-generated