Skip to content

⚡️ The 2026 Software Factory Standard — Zach Lloyd, Warp.dev

Latent Space: The AI Engineer Podcast

34m
Transcript

Original podcast

Summary

Written by a model from the transcript; not yet reviewed.

This episode covers the concept of software factories with Zach Lloyd from Warp. The guest and host explore how engineering teams are moving away from individual terminal-based coding agents and toward centralized, measurable, closed-loop agentic systems in the cloud. They discuss how a proper software factory is defined declaratively in code, integrated into existing knowledge work tools like Slack and Linear, and optimized through agent traces, online evals, and benchmarking. The conversation highlights the transition from engineering software directly to building the systems and factories that produce software, offering a practical look at modern developer infrastructure.

  • Define your internal software factories declaratively in code so they can be versioned, tested, and self-optimized by agents. 19:48
  • Centralize agent conversations and artifacts to gather data on cost, throughput, and DORA metrics across your team. 8:24
  • Use online evals and LLM-as-a-judge grading to catch specific failure modes like redundant tests and bloated codebases. 11:14
  • Benchmark different models and harnesses against company-specific runs to find the optimal trade-off between cost and correctness. 16:16

Chapters

  1. 0:00
    Episode Intro and Warp Overview

    Hosts introduce the episode and give a brief overview of Warp and its focus on software factories.

  2. 1:40
    Defining a Software Factory

    Zach explains what a software factory is and how it differs from traditional automation approaches.

  3. 2:23
    Warp’s Factory Architecture

    Details on how Warp defines factories in code, including repos, skills, MCPs, and workflows.

  4. 5:12
    Evaluation and Self‑Improvement Loops

    Explains how Warp evaluates agent performance and uses self‑improving agents to suggest factory changes.

  5. 6:48
    Benchmarking and Cost Optimization

    Shows how companies can benchmark different models and harnesses to balance cost and correctness.

  6. 22:34
    Infrastructure Stack and Plug‑In Flexibility

    Covers the stack layers, sandbox providers, inference, and harness options for building a factory.

  7. 27:02
    Scaling Considerations and Target Audience

    Talks about the size of teams that benefit most from Warp and the challenges for very small or very large companies.

  8. 30:24
    Common Criticisms and Economic Trade‑offs

    Addresses pushback on complexity, subscription models, and the economics of building a factory.

  9. 34:06
    Wrap‑Up and Next Steps

    Hosts recap key points, share Zach’s upcoming talks, and provide hiring and contact information.

Highlights

  • 2:34

    Software Factories Definedinsight

    The guest defines a software factory as an agentic system in the cloud with a certain amount of autonomy, designed to be measurable and closed-loop over time.

  • 3:55

    Factory vs Arbitrary Automationsinsight

    Teams often string together arbitrary automations, but a true software factory is defined in code with concrete workflows, skills, and MCPs per product surface.

  • 4:30

    Factory Orchestration Platformstrategy

    Warp's platform operates on specific product surfaces and repos, executing predefined workflows that handle triage, specking, and implementation.

  • 6:57

    Full Agent Traceabilitydata

    Centralizing agent conversations and computer use screenshots across the team provides a complete record of what every coding agent is doing.

  • 8:47

    DORA Metrics and Cost Slicingdata

    Teams can track traditional DORA metrics, PR throughput, and slice costs by person, automation, or where they spend money in the SDLC.

  • 14:57

    Self-Improving Factory Loopsstrategy

    Self-improvement agents analyze failure cases and make pull requests against the factory definition itself to optimize the system.

  • 16:16

    Benchmarking Harnesses and Modelsstrategy

    Teams can replay task suites against a matrix of different models and harnesses, producing cost versus correctness trade-off charts on company data.

  • 24:08

    Measuring Cost Reductionsdata

    Internal tuning and testing at Warp reduced the cost per PR from $80 down to around $15 while increasing throughput by 40 percent.

Quotes

  • “Like, you should define your factory in code.”
    Unnamed speaker · 3:49
  • “So for us, we basically have a YAML file, which says, 'Okay, these are all the repos that you can be using.'”
    Unnamed speaker · 5:20
  • “We measure like the revert rate and it hasn't gone up.”
    Unnamed speaker · 24:44
  • “And so you basically get a whole bunch of benefits that flow out of it.”
    Unnamed speaker · 6:45
  • “is a result that looks very similar to like the published results for Sweetbench or Terminalbench.”
    Unnamed speaker · 17:07
  • warp.dev 34:17–34:26

About this episode

Watch the full show on YouTube: https://www.youtube.com/watch?v=LL0YmIyBKSU

Transcript

0:00
34:48

6,567 words · 2 speakers

Topics

  • Warp
  • Slack
  • Claude Code
  • AIE
  • Warp Factories
  • Linear
  • Codex
  • Cursor
  • Pi
  • AIE Code

Page updated · transcript and summary are machine-generated