20VC: Horizontal vs Vertical Agents: Who Wins | Model Routing: Commodity or Increasing in Value | The Future of the Inference Market | Why Founders Should Take Discounts for Tier 1 Firms with Harrison Chase, LangChain
The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
Hosted by Harry Stebbings
- 1h 1m
- Transcript
Summary
Written by a model from the transcript; not yet reviewed.
This episode features a conversation with Harrison, the founder of LangChain, exploring the complexities of building AI agents, the evolving infrastructure stack, and the economics of inference and coding tools. The guest explains why non-deterministic models cause cascading errors in agent systems, how businesses must regularly erase parts of their product as foundational models improve, and why the guest believes model routing belongs primarily at the harness layer rather than the gateway layer for vertical applications. They also discuss the true cost of engineering tools, the future of agent memory, and what makes top-tier venture capital firms like Sequoia and Benchmark invaluable partners for early-stage founders.
Throughout the discussion, the guest shares specific metrics from running LangChain, noting that engineering teams spend roughly 30% of their salary on coding tools and detailing how implementing visibility, basic cost controls, and custom cloud harnesses helped bring those expenses under control. They also analyze the state of open-source versus proprietary models, the limitations of current agent memory solutions, and why vertical agents will ultimately interact through standardized natural language interfaces. The conversation offers a candid look at the realities of technical execution and startup scaling in a rapidly moving market.
- When building vertical agents, start at the harness and context layer first before attempting to post-train models. 12:38
- Track agent spending and implement visibility measures to catch runaway loops and optimize engineering tool costs. 38:41
- Model routing for domain-specific tasks belongs inside the harness rather than the gateway to leverage accumulated state and prompt caching. 18:49
- Founders should leverage top-tier venture capital partners for recruitment networks, operational support, and strategic positioning. 0:00
Chapters
- 0:21Intro and Host Setup
Harry Stebbings introduces the episode, the guest Harrison Chase, and the topic of agents.
- 1:05Fireworks and Nexus Overview
The hosts explain Fireworks’ role and how Nexus routes requests to the best model for each task.
- 2:08Larrident and AI Workforce Metrics
Discussion of Larrident’s dashboard that measures human and AI workforce impact and ROI.
- 3:07Superhuman Go for Productivity
Harry showcases Superhuman Go’s in‑document AI assistance and its effect on meeting efficiency.
- 3:37Superhuman Go: Enhancing Productivity with AI
Demonstration of how Superhuman Go integrates AI into email and document workflows to reduce friction and improve efficiency.
- 4:23LangChain: Building Agents and Observability
Harrison explains LangChain’s framework, Langsmith observability, and the challenges of agent reliability.
- 4:58Agent Complexity and Non‑Determinism
Harrison details why agents are hard to build due to non‑deterministic LLMs and cascading errors.
- 6:29Evolving Product with Better Models
He describes how LangChain must erase or adapt product features as models improve and become more capable.
- 8:48Neutral Harnesses and Portability
The conversation covers why neutral harnesses are needed to keep agents model‑agnostic and portable across providers.
- 12:02Vertical vs Horizontal Agent Strategy
Harrison argues that vertical agents win in specialized domains, while horizontal agents serve general use cases.
- 31:30Future of Memory and Learning in Agents
They discuss the current state of agent memory, the need for domain‑specific learning, and the promise of continual learning.
- 35:52Cost, Token Usage, and Infrastructure Scaling
The hosts examine how token costs rise with agent usage, the importance of cost control, and infrastructure challenges.
Highlights
- 4:58
Why Building Agents Is Hardinsight
The guest explains that agents are non-deterministic black boxes operating in infinite input and output spaces, making it difficult to prevent cascading system errors.
- 6:29
Erasing Products As Models Improvestrategy
As foundation models improve every few months, developer tool companies must be willing to delete large parts of their product that are no longer necessary.
- 16:32
Harness Versus Gateway Model Routingdebate
The guest argues that smart routing belongs at the harness layer because it has awareness of prompt caching and operational state, unlike API gateways.
- 24:11
Automating Harness Updates With Tracesstrategy
LangChain uses an internal agent called Engine to run over agent traces and automatically update the code of the harness itself.
- 30:15
Killing Commercial Memory Solutionsstory
The guest reveals that LangChain launched and later killed a commercial memory product because it was too general and failed to find product-market fit.
- 38:28
Engineering Salary Spent On Toolsdata
The guest shares that LangChain spends roughly 30% of its engineering salary on coding tools, though total token usage and costs fluctuate.
Quotes
“I think I really underestimated the demand for inference.”
Unnamed speaker · 0:11 “The hard part about building agents is they don't behave how you want them to behave.”
Unnamed speaker · 4:58 “If value is derived through verification, I think the people who win are the domain experts, the people who are able to kind of like quantify what that value means, which is very hard.”
Unnamed speaker · 12:02 “We spend probably roughly 30% of our engineering salary on coding tools.”
Unnamed speaker · 38:28 “We have a few senior engineers who spend a lot, and I 100% trust that what they're doing is well worth it.”
Unnamed speaker · 39:54 “I think in the next 12 months, we will absolutely have comparable US open-source models.”
Unnamed speaker · 44:48 “I think the demand for inference, and two is the demand for like coding agents.”
Unnamed speaker · 46:42 “I think the only place we've seen agents really have effect is in software engineering so far, though, and there are a ton of other places within an enterprise that I would argue have yet to kind of be affected by agents.”
Unnamed speaker · 42:58
Sponsored segments
- Fireworks 1:05–2:13
- Larrident 2:08–3:07
- Superhuman 3:02–4:08
- Fireworks 58:45–59:52
- Larrident 59:48–1:00:46
- Superhuman 1:00:42–1:01:47
About this episode
Harrison Chase is the co-founder and CEO of LangChain, which helps developers build, test and deploy AI agents. Backed by Sequoia Capital, Benchmark and IVP, LangChain has announced approximately $160 million in funding with the last round at a $1.25BN valuation. The company works with 35% of the Fortune 500, has surpassed one billion open-source downloads and processes over one billion events daily through LangSmith. AGENDA: 04:10 Why Is It Still So Hard to Build AI Agents? 08:50 Will the Model Labs Eat the Agent Stack? 16:25 Is Model Routing a Real Business or a Commodity? 25:45 Why Has No One Solved AI Memory? 35:45 Why Are AI Bills Exploding as Tokens Get Cheaper? 43:25 Should Companies Be Afraid of Chinese AI Models? 46:20 Where Would Harrison Not Be Investing Today? 53:05 Should Founders Take a Discount for Benchmark and Sequoia? 57:35 OpenAI at $1.5T or Anthropic at $2T: Which Would You Buy?
Transcript
13,554 words · 2 speakers
Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.
Topics
- agent
- harness
- context
- evals
- Benchmark
- traces
- LangChain
- Sequoia
- API
- OpenAI
Page updated · transcript and summary are machine-generated