Skip to content

20VC: Horizontal vs Vertical Agents: Who Wins | Model Routing: Commodity or Increasing in Value | The Future of the Inference Market | Why Founders Should Take Discounts for Tier 1 Firms with Harrison Chase, LangChain

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

Hosted by Harry Stebbings

1h 1m
Transcript

Original podcast

Where to listen

Summary

Written by a model from the transcript; not yet reviewed.

This episode features a conversation with Harrison, the founder of LangChain, exploring the complexities of building AI agents, the evolving infrastructure stack, and the economics of inference and coding tools. The guest explains why non-deterministic models cause cascading errors in agent systems, how businesses must regularly erase parts of their product as foundational models improve, and why the guest believes model routing belongs primarily at the harness layer rather than the gateway layer for vertical applications. They also discuss the true cost of engineering tools, the future of agent memory, and what makes top-tier venture capital firms like Sequoia and Benchmark invaluable partners for early-stage founders.

Throughout the discussion, the guest shares specific metrics from running LangChain, noting that engineering teams spend roughly 30% of their salary on coding tools and detailing how implementing visibility, basic cost controls, and custom cloud harnesses helped bring those expenses under control. They also analyze the state of open-source versus proprietary models, the limitations of current agent memory solutions, and why vertical agents will ultimately interact through standardized natural language interfaces. The conversation offers a candid look at the realities of technical execution and startup scaling in a rapidly moving market.

  • When building vertical agents, start at the harness and context layer first before attempting to post-train models. 12:38
  • Track agent spending and implement visibility measures to catch runaway loops and optimize engineering tool costs. 38:41
  • Model routing for domain-specific tasks belongs inside the harness rather than the gateway to leverage accumulated state and prompt caching. 18:49
  • Founders should leverage top-tier venture capital partners for recruitment networks, operational support, and strategic positioning. 0:00

Chapters

  1. 0:00
    Podcast Intro and Host Greeting

    The episode opens with the host introducing the show and setting the stage for the discussion.

  2. 4:55
    Why Building Agents Is Hard

    Harrison explains the non‑deterministic nature of LLMs and the cascading errors that make reliable agents difficult to build.

  3. 6:29
    Evolving Product with Better Models

    The conversation turns to how LangChain’s product must adapt as models improve, often erasing previously needed features.

  4. 8:48
    Neutral Harnesses and Portability

    Harrison discusses the importance of neutral harnesses that keep context and memory independent of specific model providers.

  5. 11:42
    Vertical vs Horizontal Agent Strategy

    The hosts debate whether vertical, domain‑specific agents or horizontal, general agents provide more value and how to choose.

  6. 16:32
    Model Routing: Gateway vs Harness

    The discussion focuses on where model routing should live—at the gateway level for simplicity or within the harness for context awareness—and the trade‑offs.

  7. 30:09
    Memory Layer and Continual Learning

    Harrison explains the challenges of building a memory system for agents and how continual learning can improve performance over time.

  8. 36:00
    Cost, Tokens, and Infrastructure Scaling

    The hosts examine how token costs are rising, the need for cost controls, and the infrastructure strain caused by agent‑to‑agent interactions.

  9. 37:38
    Future Outlook for Agents and Markets

    The episode concludes with predictions about the future of agent technology, market consolidation, and the evolving role of providers and enterprises.

  10. 40:12
    Agent Cost Overruns and Mitigation

    Discusses a costly agent task that spent $40k and the importance of monitoring usage to prevent waste.

  11. 40:59
    Future of Coding Agent Spend

    Explores how agent spending could equal or exceed human salaries as organizations shrink and become more agent-driven.

  12. 42:30
    Anthropic’s Value and Agent Adoption

    Examines how the percentage of spend on agents underpins Anthropic’s enterprise value and the potential for broader adoption.

Highlights

  • 4:58

    Why Building Agents Is Hardinsight

    The guest explains that agents are non-deterministic black boxes operating in infinite input and output spaces, making it difficult to prevent cascading system errors.

  • 6:29

    Erasing Products As Models Improvestrategy

    As foundation models improve every few months, developer tool companies must be willing to delete large parts of their product that are no longer necessary.

  • 16:32

    Harness Versus Gateway Model Routingdebate

    The guest argues that smart routing belongs at the harness layer because it has awareness of prompt caching and operational state, unlike API gateways.

  • 24:11

    Automating Harness Updates With Tracesstrategy

    LangChain uses an internal agent called Engine to run over agent traces and automatically update the code of the harness itself.

  • 30:15

    Killing Commercial Memory Solutionsstory

    The guest reveals that LangChain launched and later killed a commercial memory product because it was too general and failed to find product-market fit.

  • 38:28

    Engineering Salary Spent On Toolsdata

    The guest shares that LangChain spends roughly 30% of its engineering salary on coding tools, though total token usage and costs fluctuate.

Quotes

  • “We spend probably roughly 30% of our engineering salary on coding tools.”
    Unnamed speaker · 0:02
  • “The hard part about building agents is they don't behave how you want them to behave.”
    Unnamed speaker · 4:58
  • “I think the main way our business changes as the model gets better is we basically have to erase big parts of our product.”
    Unnamed speaker · 6:38
  • “If value is derived through verification, I think the people who win are the domain experts, the people who are able to kind of like quantify what that value means, which is very hard.”
    Unnamed speaker · 12:02
  • “The harness can be very tied to the model. It can also be neutral.”
    Unnamed speaker · 10:01
  • “I think in the next 12 months, we will absolutely have comparable US open-source models.”
    Unnamed speaker · 44:48
  • “I think the demand for inference, and two is the demand for like coding agents.”
    Unnamed speaker · 46:42
  • “I think the only place we've seen agents really have effect is in software engineering so far, though, and there are a ton of other places within an enterprise that I would argue have yet to kind of be affected by agents.”
    Unnamed speaker · 42:58
  • “I think the tricky part is that the only place we've seen agents really have effect is in software engineering so far, though, and there are a ton of other places within an enterprise that I would argue have yet to kind of be affected by agents.”
    Unnamed speaker · 42:58
  • Fireworks 1:05–2:13
  • Larrident 2:08–3:07
  • Superhuman 3:37–4:11
  • Fireworks 58:45–59:52
  • Larrident 59:48–1:00:46
  • Superhuman 1:00:42–1:01:47

About this episode

Harrison Chase is the co-founder and CEO of LangChain, which helps developers build, test and deploy AI agents. Backed by Sequoia Capital, Benchmark and IVP, LangChain has announced approximately $160 million in funding with the last round at a $1.25BN valuation. The company works with 35% of the Fortune 500, has surpassed one billion open-source downloads and processes over one billion events daily through LangSmith. AGENDA: 04:10 Why Is It Still So Hard to Build AI Agents? 08:50 Will the Model Labs Eat the Agent Stack? 16:25 Is Model Routing a Real Business or a Commodity? 25:45 Why Has No One Solved AI Memory? 35:45 Why Are AI Bills Exploding as Tokens Get Cheaper? 43:25 Should Companies Be Afraid of Chinese AI Models? 46:20 Where Would Harrison Not Be Investing Today? 53:05 Should Founders Take a Discount for Benchmark and Sequoia? 57:35 OpenAI at $1.5T or Anthropic at $2T: Which Would You Buy?

Transcript

0:00
1:01:49

13,554 words · 2 speakers

Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.

Topics

  • LangChain
  • OpenAI
  • Anthropic
  • Modal
  • Temporal
  • Crusoe
  • PyTorch
  • Superbase
  • Convex
  • Sasta

Page updated · transcript and summary are machine-generated