Skip to content

20VC: Horizontal vs Vertical Agents: Who Wins | Model Routing: Commodity or Increasing in Value | The Future of the Inference Market | Why Founders Should Take Discounts for Tier 1 Firms with Harrison Chase, LangChain

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

Hosted by Harry Stebbings

1h 1m
Transcript

Original podcast

Where to listen

Summary

Written by a model from the transcript; not yet reviewed.

This episode features a conversation with Harrison, the founder of LangChain, exploring the complexities of building AI agents, the evolving infrastructure stack, and the economics of inference and coding tools. The guest explains why non-deterministic models cause cascading errors in agent systems, how businesses must regularly erase parts of their product as foundational models improve, and why the guest believes model routing belongs primarily at the harness layer rather than the gateway layer for vertical applications. They also discuss the true cost of engineering tools, the future of agent memory, and what makes top-tier venture capital firms like Sequoia and Benchmark invaluable partners for early-stage founders.

Throughout the discussion, the guest shares specific metrics from running LangChain, noting that engineering teams spend roughly 30% of their salary on coding tools and detailing how implementing visibility, basic cost controls, and custom cloud harnesses helped bring those expenses under control. They also analyze the state of open-source versus proprietary models, the limitations of current agent memory solutions, and why vertical agents will ultimately interact through standardized natural language interfaces. The conversation offers a candid look at the realities of technical execution and startup scaling in a rapidly moving market.

  • When building vertical agents, start at the harness and context layer first before attempting to post-train models. 12:38
  • Track agent spending and implement visibility measures to catch runaway loops and optimize engineering tool costs. 38:41
  • Model routing for domain-specific tasks belongs inside the harness rather than the gateway to leverage accumulated state and prompt caching. 18:49
  • Founders should leverage top-tier venture capital partners for recruitment networks, operational support, and strategic positioning. 0:00

Chapters

  1. 0:00
    Intro and Host Greeting

    Harry Stebbings opens the episode, introduces the topic and guest, and sets the stage for a discussion on agents and inference markets.

  2. 0:58
    Fireworks and Intelligent Model Routing

    The hosts explain how Fireworks and its Nexus platform enable intelligent routing of requests to the best model, improving quality, latency, and cost.

  3. 2:00
    Why Building Agents Is Hard

    Harrison discusses the non‑deterministic nature of LLMs, cascading errors, and the difficulty of ensuring reliable agent behavior.

  4. 6:10
    Evolving Product with Better Models

    The conversation turns to how LangChain adapts its product as models improve, emphasizing the need to erase outdated features and focus on harnessing new capabilities.

  5. 8:48
    Neutral Harnesses and Portability

    Harrison explains the importance of neutral harnesses that separate model logic from state, enabling portability across providers and protecting against lock‑in.

  6. 11:42
    Vertical vs Horizontal Agent Value

    The hosts debate the value of vertical agents—domain‑specific solutions—and horizontal agents, noting that most applications will benefit from a vertical approach.

  7. 27:35
    Memory Layer and Learning Agents

    The discussion shifts to the critical yet unsolved memory layer, how agents learn from interactions, and the future of domain‑specific learning agents.

  8. 34:40
    Token Costs and Infrastructure Scaling

    The conversation covers rising token costs, the need for cost controls, and how agent workloads strain infrastructure compared to human usage.

  9. 37:38
    Enterprise Priorities: Accuracy, Cost, Security

    Harry shares insights from enterprise surveys, highlighting accuracy as the top priority, followed by latency, cost, and security concerns.

  10. 38:21
    Coding Tool Spend and Cost Management

    The episode concludes with a practical case study of how LangChain manages coding tool spend, using visibility, cost controls, and harness improvements to keep budgets in check.

  11. 40:12
    Agent Cost Escalation and Misuse

    Discussion of a costly agent task that incurred $40K due to excessive looping and the importance of monitoring misuse.

  12. 40:59
    Estimating Coding Agent Spend

    Exploration of how rapidly changing organizations affect future coding agent costs and the potential for small teams to drive high spend.

Highlights

  • 4:58

    Why Building Agents Is Hardinsight

    The guest explains that agents are non-deterministic black boxes operating in infinite input and output spaces, making it difficult to prevent cascading system errors.

  • 6:29

    Erasing Products As Models Improvestrategy

    As foundation models improve every few months, developer tool companies must be willing to delete large parts of their product that are no longer necessary.

  • 16:32

    Harness Versus Gateway Model Routingdebate

    The guest argues that smart routing belongs at the harness layer because it has awareness of prompt caching and operational state, unlike API gateways.

  • 24:11

    Automating Harness Updates With Tracesstrategy

    LangChain uses an internal agent called Engine to run over agent traces and automatically update the code of the harness itself.

  • 30:15

    Killing Commercial Memory Solutionsstory

    The guest reveals that LangChain launched and later killed a commercial memory product because it was too general and failed to find product-market fit.

  • 38:28

    Engineering Salary Spent On Toolsdata

    The guest shares that LangChain spends roughly 30% of its engineering salary on coding tools, though total token usage and costs fluctuate.

Quotes

  • “The hard part about building agents is they don't behave how you want them to behave.”
    Unnamed speaker · 4:58
  • “I think the reason we need neutral harnesses is that there is a lot of state that you build up either before you build the agent or as part of the agent running.”
    Unnamed speaker · 8:58
  • “If value is derived through verification, I think the people who win are the domain experts, the people who are able to kind of like quantify what that value means, which is very hard.”
    Unnamed speaker · 12:02
  • “We spend probably roughly 30% of our engineering salary on coding tools.”
    Unnamed speaker · 0:02
  • “When agents are interacting, it makes the number of requests go way up.”
    Unnamed speaker · 34:13
  • “I think we spent 40K in a week on that one thing.”
    Unnamed speaker · 40:31
  • “I don't like fundraising at all.”
    Unnamed speaker · 55:35
  • “I think the only place we've seen agents really have effect is in software engineering so far, though, and there are a ton of other places within an enterprise that I would argue have yet to kind of be affected by agents.”
    Unnamed speaker · 42:58
  • Fireworks 1:05–2:13
  • Larrident 2:08–3:07
  • Superhuman 3:02–4:11
  • Fireworks 58:45–59:52
  • Larrident 59:48–1:00:46
  • Superhuman 1:01:17–1:01:47

About this episode

Harrison Chase is the co-founder and CEO of LangChain, which helps developers build, test and deploy AI agents. Backed by Sequoia Capital, Benchmark and IVP, LangChain has announced approximately $160 million in funding with the last round at a $1.25BN valuation. The company works with 35% of the Fortune 500, has surpassed one billion open-source downloads and processes over one billion events daily through LangSmith. AGENDA: 04:10 Why Is It Still So Hard to Build AI Agents? 08:50 Will the Model Labs Eat the Agent Stack? 16:25 Is Model Routing a Real Business or a Commodity? 25:45 Why Has No One Solved AI Memory? 35:45 Why Are AI Bills Exploding as Tokens Get Cheaper? 43:25 Should Companies Be Afraid of Chinese AI Models? 46:20 Where Would Harrison Not Be Investing Today? 53:05 Should Founders Take a Discount for Benchmark and Sequoia? 57:35 OpenAI at $1.5T or Anthropic at $2T: Which Would You Buy?

Transcript

0:00
1:01:49

13,554 words · 2 speakers

Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.

Topics

  • Benchmark
  • Sequoia
  • LangChain
  • OpenAI
  • Claude Code
  • Open Router
  • Anthropic
  • Langsmith
  • Reflection
  • Modal

Page updated · transcript and summary are machine-generated