
⚡️ The 2026 Software Factory Standard — Zach Lloyd, Warp.dev
Latent Space: The AI Engineer Podcast
- 34m
- Transcript
Summary
Written by a model from the transcript; not yet reviewed.
This episode covers the concept of software factories with Zach Lloyd from Warp. The guest and host explore how engineering teams are moving away from individual terminal-based coding agents and toward centralized, measurable, closed-loop agentic systems in the cloud. They discuss how a proper software factory is defined declaratively in code, integrated into existing knowledge work tools like Slack and Linear, and optimized through agent traces, online evals, and benchmarking. The conversation highlights the transition from engineering software directly to building the systems and factories that produce software, offering a practical look at modern developer infrastructure.
- Define your internal software factories declaratively in code so they can be versioned, tested, and self-optimized by agents. 19:48
- Centralize agent conversations and artifacts to gather data on cost, throughput, and DORA metrics across your team. 8:24
- Use online evals and LLM-as-a-judge grading to catch specific failure modes like redundant tests and bloated codebases. 11:14
- Benchmark different models and harnesses against company-specific runs to find the optimal trade-off between cost and correctness. 16:16
Chapters
- 0:00Episode Intro and Warp Overview
Hosts introduce the episode and give a brief overview of Warp and its focus on software factories.
- 1:40Defining a Software Factory
Zach explains what a software factory is and how it differs from traditional automation approaches.
- 2:23Warp’s Factory Architecture
Details on how Warp defines factories in code, including repos, skills, MCPs, and workflows.
- 5:12Evaluation and Self‑Improvement Loops
Explains how Warp evaluates agent performance and uses self‑improving agents to suggest factory changes.
- 6:48Benchmarking and Cost Optimization
Shows how companies can benchmark different models and harnesses to balance cost and correctness.
- 22:34Infrastructure Stack and Plug‑In Flexibility
Covers the stack layers, sandbox providers, inference, and harness options for building a factory.
- 27:02Scaling Considerations and Target Audience
Talks about the size of teams that benefit most from Warp and the challenges for very small or very large companies.
- 30:24Common Criticisms and Economic Trade‑offs
Addresses pushback on complexity, subscription models, and the economics of building a factory.
- 34:06Wrap‑Up and Next Steps
Hosts recap key points, share Zach’s upcoming talks, and provide hiring and contact information.
Highlights
- 2:34
Software Factories Definedinsight
The guest defines a software factory as an agentic system in the cloud with a certain amount of autonomy, designed to be measurable and closed-loop over time.
- 3:55
Factory vs Arbitrary Automationsinsight
Teams often string together arbitrary automations, but a true software factory is defined in code with concrete workflows, skills, and MCPs per product surface.
- 4:30
Factory Orchestration Platformstrategy
Warp's platform operates on specific product surfaces and repos, executing predefined workflows that handle triage, specking, and implementation.
- 6:57
Centralizing agent conversations and computer use screenshots across the team provides a complete record of what every coding agent is doing.
- 8:47
DORA Metrics and Cost Slicingdata
Teams can track traditional DORA metrics, PR throughput, and slice costs by person, automation, or where they spend money in the SDLC.
- 14:57
Self-Improving Factory Loopsstrategy
Self-improvement agents analyze failure cases and make pull requests against the factory definition itself to optimize the system.
- 16:16
Benchmarking Harnesses and Modelsstrategy
Teams can replay task suites against a matrix of different models and harnesses, producing cost versus correctness trade-off charts on company data.
- 24:08
Internal tuning and testing at Warp reduced the cost per PR from $80 down to around $15 while increasing throughput by 40 percent.
Quotes
“Like, you should define your factory in code.”
Unnamed speaker · 3:49 “So for us, we basically have a YAML file, which says, 'Okay, these are all the repos that you can be using.'”
Unnamed speaker · 5:20 “We measure like the revert rate and it hasn't gone up.”
Unnamed speaker · 24:44 “And so you basically get a whole bunch of benefits that flow out of it.”
Unnamed speaker · 6:45 “is a result that looks very similar to like the published results for Sweetbench or Terminalbench.”
Unnamed speaker · 17:07
Sponsored segments
- warp.dev 34:17–34:26
About this episode
Watch the full show on YouTube: https://www.youtube.com/watch?v=LL0YmIyBKSU
Transcript
6,567 words · 2 speakers
Topics
- Warp
- Slack
- Claude Code
- AIE
- Warp Factories
- Linear
- Codex
- Cursor
- Pi
- AIE Code
Page updated · transcript and summary are machine-generated