Making the Case for the Terminal as AI's Workbench: Warp’s Zach Lloyd

Zach Lloyd built Warp to modernize the terminal for professional developers, but the rise of coding agents transformed his company's trajectory. He discusses the convergence of IDEs and terminals into new workbenches built for prompting and agent orchestration, and why he thinks "coding will be solved" within a few years, making human expression of intent the ultimate bottleneck. Zach explains how Warp competes against subsidized tools from Anthropic and OpenAI, and why the terminal's time-based, text-oriented format makes it perfect for managing swarms of cloud agents. Hosted by Sonya Huang, Sequoia Capital

• 0:00Speaker 00

  Just the general form factor of the terminal is perfect for agentic work because everything is like time-based. It's all about input of text and output of text. You get a log of what you're doing. You can multitask agents in the terminal really easily. And so I think it's been like actually a great stroke of luck for us in a lot of ways that the terminal has become the center of agentic development. It's a huge opportunity for us.

• 0:41Speaker 01

  In this episode, Zach Lloyd, founder of Warp, reveals why the terminal is becoming the center of AI-powered development. Zach shares how coding interfaces are converging into a new workbench built for prompting and agent orchestration, and why the next frontier isn't developers typing prompts, but ambient agents running in the background that autonomously respond to system events like server crashes or security incidents. We discuss the brutal competitive dynamics of the coding market and why model providers are racing into the application layer. And finally, Zach shares his thesis that coding is nearly solved and that the ultimate bottleneck for AI will be humans' ability to clearly express intent. Enjoy the show.

• 1:20Speaker 01

  Zach, thanks so much for taking the time to join today.

• 1:23Speaker 00

  Thanks for having me on. Good to be here.

• 1:25Speaker 01

  Before we get started, can you tell our audience a little bit about yourself? And what is Warp? And what company did you set out to build and why?

• 1:32Speaker 00

  Yep. So I am Zach. I'm the CEO and founder of Warp. Warp is a developer-focused startup. Our goal has always been just like help pro developers ship better software more quickly. The product that we've built, it has an interesting history. We're like five years old. We started off building a modern reimagination of the terminal. And today, the product has evolved into it's sort of a terminal with agents built in is one way of thinking about it, pretty simplest. It's a workbench for building software with agents is kind of the more general way of framing it.

• 2:12Speaker 01

  Awesome. Let's dive right in. What made you decide that the terminal was the right place to build?

• 2:19Speaker 00

  So I've been a developer for a really long time. I've always used the terminal. In prior life, I was a principal engineer at Google. I used to run engineering on Google Docs. I'm not a good terminal user. Like, I always worked with people who were good at using it, and I saw that, you know, you get just a ton of stuff done as a developer because of where it sits in the stack. So it's like a super duper powerful thing if you know how to use it right, but the sort of stock version or classic version of the terminal, I think, is like a horrible product. It's hard to learn. It's easy to make mistakes in. The mouse doesn't work. And so, you know, I was interested in how do you build something that's impactful for

• 3:02Speaker 00

  developers? How do you build something that helps more good software exist in the world? And trying to reimagine the terminal felt like a cool thing to take on.

• 3:11Speaker 01

  And how much of the thesis is around making the terminal great for a single player versus multiplayer?

• 3:16Speaker 00

  It's a good question. So the multiplayer part was gonna be with the business model. What was gonna be? So it's like, you know, I came from the Google Docs world. I built collaborative software. I think the closest analogy would be something like Postman where, you know, they have like a collaborative API platform. We were gonna do that around the terminal where you could share commands, you could share sort of like run books, share incident response manuals. And Warp actually has all that stuff, and it's super, it's super useful, not just for people, but for agents at this point to have all that knowledge baked into the product. So that was like gonna be the the business model, but where we actually started was just like

• 3:57Speaker 00

  the hands-on keys interaction with the terminal itself. Could we reimagine the developer experience of that? And so we spent, you know, the first like year, year and a half of just like, how would we like this thing to work? How do we want the input into the terminal to work? How do we want the output to work? And like, how can we make it just like easier without diminishing the power of the tool?

• 4:22Speaker 01

  Yeah, awesome. And you made the decision to focus on rebuilding, reimagining the terminal pre generative AI, pre coding models taking off. Do coding models and agents, do they change your answer to the question of like, how important is the terminal as the workbench?

• 4:38Speaker 00

  Terminal is like ironically more important now. The terminal has become I think the preferred form factor for working with agents, I mean, basically you can work with them in the IDE or you can work with them in the terminal. Or you like, you can create some other workbench, which Warp, you can see Warp that way. Actually Warp started as a terminal, is a broader workbench now for agents. But just the general form factor of the terminal is perfect for agentic work because everything is like time-based. It's all about input of text and output of text. You get a log of what you're doing. You can multitask agents in the terminal really easily. And so I think it's been like actually a great stroke of luck for us in a lot of ways

• 5:22Speaker 00

  that the terminal has become the center of agentic development. It's a huge opportunity for us.

• 5:29Speaker 01

  I'm curious if you thought that we were headed towards a world where people just weren't going to spend time in the IDE? And do you think that's been accelerated now?

• 5:36Speaker 00

  I think that the kind of tools are morphing.

• 5:40Speaker 00

  And so, you know, pre-agent world, you had pretty clear distinction between terminals and IDEs. Today, you have tools like Warp, which are, you know, we've like grown from the terminal and added a bunch of IDE features like code editor and code review features and the file tree. And like we get yelled at on Twitter for having a file tree in Warp because it's not like a pure terminal thing. But then if you look at like the latest iteration of Cursor, which, you know, started as an IDE, it looks a lot more like Warp, like the primary interface is now more of a chat interface and talking to your computer. But you still have all the file editing things.

• 6:23Speaker 00

  So I don't know if I would be like, terminal is going to die or the IDE is going to die. What I do feel strongly about is that the, there's going to be innovation and there is innovation happening where the form factor is changing to match what the agentic workflow should be. So the form factor is like, it should be geared towards prompting. It should be geared towards adding context. It should be geared towards reviewing agent-generated code diffs. I think actually now like team is even more important, like especially as you have more and more agents that are not just like launched locally by people, but are coming to Be launched by system events. And so I think the workbench is changing and I actually

• 7:07Speaker 00

  think it will end up looking more like a terminal than an IDE, but probably won't strictly speaking be a traditional version of either.

• 7:15Speaker 01

  And is the rough framing of, you know, the reason to use each that, you know, terminal is roughly equivalent to the chatbot and like you can, you can chat with a coding agent and hand off tasks. And then IDE is roughly equivalent to like a GUI for actually editing and writing code. Is that, is that the right mental model?

• 7:33Speaker 00

  Yeah, that's definitely where things have like started. Uh, like, yeah, right. The IDE is like Microsoft Word for your code and the, um, the terminal is like, is like chatting with your computer. Um, and the, if you're doing professional agentic development, which I would distinguish from vibe coding, you kind of want both of those. Like, I don't think for the pro use case, we're at a spot yet where you can be so disconnected from the code that you don't need some way of like falling into hand editing it. I would think of it as like, the hand editing is like, almost become like a fallback interface

• 8:14Speaker 00

  or a secondary interface. And the primary interface now is the prompting interface. And so, um, yeah, I basically, I think that that is the right, like distinction, but I think it's all kind of merging product wise.

• 8:27Speaker 01

  Merging, yeah. Interesting. Um, you've mentioned pro coders a few times. What was the decision to focus on pro coders about? And, um, how do you think that plays out over the next decade? Like, will there be pro developers left? Will everybody be a pro developer?

• 8:42Speaker 00

  So, okay, it's a great question. So I think what I really care about is, um, helping build software that I use every day. Like, there's probably-

• 8:55Speaker 00

  10 apps or whatever in my Mac dock and pinned as Chrome tabs, which are apps like Google Docs or Spotify or Notion or Figma or Warp, which are hard to build apps that I think

• 9:10Speaker 00

  it's those hard to build apps that the world spends most of their time using. And those are built, I think, more by pros and they're definitely built more by enterprises. And I just want to be a part of like creating that kind of software. Whereas I do feel like the the non-pro segment is cool. And I do think it's actually really, it's empowering that kind of anyone can make an app at this point. But I just think like the the sort of economic value of the apps that you build with a vibe coding tool, you know, like lovable or Replit or whatever, is lower than the economic value of the apps built with the tool that's geared towards pros.

• 9:51Speaker 00

  And it's also, it's just like, I can't, I don't think I've ever spent my day using an app that's been built in like a no-code, low-code, vibe-code tool, whereas I spend all of my days like literally living in software that is built by pros. It's really like built for like these immersive, hard, important, economically valuable use cases.

• 10:12Speaker 01

  Totally. The world is a museum of fashion projects. And I think that includes the software we choose to to use every day.

• 10:19Speaker 00

  Yeah.

• 10:19Speaker 01

  Let's talk about competition in the coding market. This is like

• 10:23Speaker 00

  Sure.

• 10:23Speaker 01

  this is the most brutally competitive

• 10:25Speaker 00

  Yeah.

• 10:26Speaker 01

  of software I have ever, ever seen. And you're playing in interesting waters, right? You're competing with a lot of folks, you're collaborating with a lot of folks. Maybe just help orient our audience. Where do you see yourselves in the broader competitive landscape in the coding market?

• 10:43Speaker 00

  Yeah, it's like it is competitive out there because it's such a big, important market that a lot of people want to play in it.

• 10:54Speaker 00

  And, you know, where do we sit? So

• 10:59Speaker 00

  So we are a sort of general purpose agentic development workbench, which means you can use us like Cursor, you can use us like Claude Code. I think we have a unique product approach to doing agentic development where we are truly the only platform out there that has grown out of the terminal. So there's a lot that have grown out of like IDEs specifically forking VS Code, and they're all very similar products. And then there's a lot that are just apps that run within the terminal. And so those are like text-based apps, and those are also basically all the same.

• 11:41Speaker 00

  And so Warp is a very differentiated product approach.

• 11:48Speaker 00

  I think one area where our product approach really shines is for people who are doing like traditionally terminal-heavy workflows. And so that would be things like stuff beyond coding. So stuff like the software development lifecycle, it could be setting up projects. It could be deployment. It could be working with like Docker and Kubernetes. It could be incident response. So like backend, DevOps, SRE, people who do production work. I think Warp is an amazing tool for them because it integrates so well with all of these non-coding terminal workflows. But the truth is, I don't know, Warp is like, we're at any given moment, we're one of the top five agents on Sweet Bench. We're typically number one or two on Terminal Bench.

• 12:32Speaker 00

  And so it's a great general-purpose coding agent. And so we're in the market, but it is really competitive.

• 12:42Speaker 01

  Yeah.

• 12:43Speaker 00

  We're trying to compete on the quality of the product. It's like we are in competitive, there's competitive pressure around cost, which is like a really challenging thing for us.

• 12:54Speaker 01

  Let's talk about that directly and just to hit it head on. Like, how do you compete when Anthropic, you know, can subsidize their tool with model profits?

• 13:03Speaker 00

  It's Anthropic, it's OpenAI, and it's Google. So we have to compete based on the quality of the product, for one thing. And so I think like we can be a little bit in the more premium part of the market here, like coding agents and like Developer experience, it's not, it's not bananas. It's not like a commodity. These aren't like totally fungible things. Like the product experience does actually matter. And so we can get people who, who, who care about that. I think that matters in that, um, basically I, I also think you want to stay away from certain user segments who are most

• 13:47Speaker 00

  cost conscious and cost shopping. And so that would be like five coders, people who are running like agents, like 24 hours a day making, you know, making prototypes. And that's just not actually the usage pattern of a pro developer. And so for, you know, a pro developer, um, I think you can make a pretty strong argument that like the, the actual holistic experience of using the tool might be worth like 20 bucks more, 40 bucks more, 80 bucks. Like, these are tiny sums compared to the amount of like productivity that people are gaining and the amount of software that's being produced. I think there's also a, a way that we are trying to sit above the model providers and, you know,

• 14:30Speaker 00

  I think there's been a positive development here, which is that for about, I don't know, three months, maybe three, six months. I think Anthropic was kind of like the main show in town when it came to frontier coding. And now I think Gemini three and, and even like the latest Codex are basically on par with the, like the latest Claude model. And so there is advantage to being able to let people choose between those or model route amongst them. To model route with like cheaper open source models. So it's not easy. And if you view it, it's just like, we're in like a cost race and all these coding tools are the same. And I think that's like, we have to differentiate way more on the product and also just like

• 15:12Speaker 00

  the orchestration of these agents, but I don't think it's quite, that's quite the situation.

• 15:17Speaker 01

  Got it. Okay. So you're winning people because they, they love the overall warp product and that includes?

• 15:23Speaker 00

  I think so.

• 15:24Speaker 01

  But also product like, like the actual, you know, terminal, the better terminal you set out to build.

• 15:29Speaker 00

  Yeah, it is kind of a funnel. It's like, we have a lot of, you know, we have like 700,000 developers in warp. Actively and like, there's like a bit of a funnel from the terminal into the coding use cases and the, at least into the terminal use cases.

• 15:45Speaker 01

  Yeah. You mentioned being one or two on Terminal Bench. What goes into that? Like, are you training your own Terminal models? Or is this harness is on top of the existing foundation models?

• 15:57Speaker 00

  So for us, it's a harness on a mix of models. So and then like the actual capabilities of the app kind of matter, which is interesting. So and what I mean by that is like Terminal Bench, it's not just coding tasks. It's like all sorts of things that you might do in the in the terminal. And so we have some intrinsic advantage there by actually being the terminal and not like an app running within the terminal. And so like, for example, one of the Terminal Bench tasks was like playing Zork or something, which is like an interactive terminal game. And so we can, we can like use the terminal. We can do computer use in terminal is probably the easiest way to think of it.

• 16:38Speaker 00

  So just like there's, there's companies that they're doing browser use. We can do terminal use at the layer of the terminal as opposed to the layer of like a webpage, which is at the what the equivalent would be for the browser analogy. And so that helps us like do certain tasks on that particular eval that is hard for other harnesses to do.

• 17:01Speaker 01

  Hmm, got it.

• 17:04Speaker 01

  You recently redid your pricing. What'd you learn about developers and how they want to pay for AI?

• 17:09Speaker 00

  Oh, my God. So we're still like not fully out of this. Yeah, I mean, I could just explain like the whole, the whole thing here. So we, our initial pricing was basically you do a subscription and you get a fixed amount of AI credits every month. And we priced it so that this is when we were at smaller scale. If you like fully utilize your plan, it would cost us money. But the hope was that the on the sort of on the average utilization that we would make money, right? So it's like you have a plan that gives people 50,000 credits and most people only use 20,000, you can kind of price it around that.

• 17:54Speaker 00

  What happened was like the people just used more and more. And so we got to a point where we were losing more and more money. And so from a company's strategy standpoint, we had a choice. We talked to Andrew a bunch about this, like we could either kind of play the like,

• 18:12Speaker 00

  and like we're growing really fast, like the revenue was growing. You know, we're adding like a million in revenue every, it's since slowed down a little. It was like every five days or something. And it's like, we could play the game, go raise more money, but the margins were really bad. And so we decided that wasn't just like, wasn't the smart, long-term strategic thing to do. And like, also not like a race we can win to the earlier conversation here. Like we, we, we just can't beat people if the thing is cost. And so we wanted to know, like, are people paying for value? Will they pay if, if we are margin positive? And so the way that we have like changed the pricing is so that it's much more consumption based.

• 18:56Speaker 00

  So you, you now pay for like a base plan of 20 bucks a month. And then you buy credits on top of that. And we, we ensure that that like, you know, it's like in the old world, we didn't want people fully utilizing their AI because it would cost us money. Now it's like much better if people use more AI.

• 19:16Speaker 00

  It is more expensive for sure. And we've had a lot of user complaints around that, which sucks. If any more customers are listening, like it's a bummer. It really does suck, but it's like, we just could not afford to keep subsidizing the way that we were. And like all in all, I would say it's gone pretty well. Like we're still, we're still growing pretty well. And now it's like a growth that is sustainable and not like a, you know, unsustainable subsidized revenue growth. So tricky thing to do.

• 19:49Speaker 01

  Would you ever train your own models?

• 19:52Speaker 00

  I think we would definitely do like what some of our competitors are doing where we would fine tune models and do RL and that type of stuff. I it's hard for me to imagine us competing with training like a full frontier level model, just the amount of capital that costs. We do have a ton of interesting data, like I think it's like actually really interesting strategic asset for us in terms of like the workflows people are doing in the terminal, how to improve them, how people are interacting with our agent. So I think it's likely that we will, we will do some sort of RL.

• 20:31Speaker 00

  And I think it's also very likely that we are going to lean more into a mixture of models and more model routing to try to like give users the best experience when it comes to sort of latency, cost and quality, which are the three, the three sort of like vectors here.

• 20:49Speaker 01

  And do you see your role as kind of like optimizing that on behalf of users? Do you want to see yourselves as giving all options to users for them to pick?

• 20:58Speaker 00

  Yeah, so our, our philosophy has been like make a great default, but then because these are developers, they want control. So we have like an, we actually have a couple of variants of default. So we have like a default that's geared towards like efficiency and one that's geared towards performance. And then after that, we give people the raw choice. The raw choice is like a little weird because like increasingly we really want to like use different models for different things internally. And it doesn't map that cleanly under using, say like, you know, GPT five two for everything. So it's, it's a little complicated, but I think it's actually something that developers like is the control. And so I don't see us moving away from that right now.

• 21:42Speaker 01

  Yeah. One of the most interesting parts of where you sit is that you can actually see which models different developers are using. And so I'm curious, in your user base, which models are most popular? Has it evened out a lot? And are like, are there different flavors or personalities of what the different models are good at?

• 22:00Speaker 00

  Like 70 to 80% of our user base will use whatever we set our auto to and not touch it.

• 22:08Speaker 00

  And what we set the auto to currently is like, it's a different one for efficient, a different one for performance. It's a mix of the Codex, sorry, GPT-5 too, but it could be, it's related to Codex. And then Sonnet 4.5. When people are opting into choosing a model, lately Gemini 3 Pro has been very popular. It's a really good model. You know, what we will do is like, we will test different variants in our auto model and see how people respond, how they engage with them. And I think we'll probably test Gemini 3. I've been impressed with it.

• 22:51Speaker 00

  So yeah, I would say, if I had to like stack rank, I would probably say like, the anthropic models are probably still most popular. And then between Gemini and OpenAI, there's, there's a decent amount of people opting into each of those.

• 23:04Speaker 01

  What about Grok?

• 23:06Speaker 00

  Grok is not in warp. It could be in warp. They've reached out a bunch of times to put it in warp. We, we, I'm not, I'm not at all opposed to putting it in warp. It's just like every time we put a model in warp, we have to, I like with like some concrete benefit to users and because it's a bunch of work to, to tune our harness to work well with the model.

• 23:29Speaker 01

  Hmm, I see. Interesting. Can you say a word more about that harness? Like, what do you do to make your harness good?

• 23:37Speaker 00

  So harness is like how you, how you prompt, what tools you make available, how it, how you manage context.

• 23:46Speaker 00

  And so like the big things that like are determining quality of harness are like it's literally like the language of the prompting, it's the toolset definition. It's things like handling the context window. So specifically, when do you use something like a sub agent where you where you go out and have something has a separate context window? When do you summarize? When do you truncate? Like, we have things that have like, you know, you might run a terminal command that has like gigantic output and you don't want that all on your context window, but you might want some part of it in your context window. So how do you sort of pick out the right stuff? How do you do RAG? How do you integrate with MCP?

• 24:27Speaker 00

  And so it's, um, there's just like some engineering and like alpha that goes into that. The way that you make that good is by measuring. Uh, like you can start just by like in a pretty like naive way and just give it a bunch of prompts. But the way that you make it really good is by measuring. And by measuring, you can do it with, um, sort of like a fixed set of evals where we know, uh, what the results should be and like, not all of them should work. Uh, and so we have internally valves. You can do it on public benchmarks. And that's actually been really good for getting our harness to be awesome. It's just like going through the exercise of making our agent perform well on the public benchmarks. And

• 25:06Speaker 00

  then you can do it by looking at like, uh, you know, user data. And so we use, um, brain trust. So, so there's various platforms you can use. So like, sort of look for patterns and failure modes in the agent interaction and then try to tune the harness and sort of replay them as evals. So, you know, that was a big mindset shift for us, like to, to get to doing that. But that was a hundred percent necessary to do it all data driven to get something that was good.

• 25:34Speaker 01

  Yeah, got it. And do you have your own tab autocomplete models? Is that just not even relevant for your, your product?

• 25:41Speaker 00

  We don't have that. Um, it's not super duper relevant at the moment. Like, I think there would be incremental benefit if we were doing tab completion in the terminal for people. And I, we do have, and I think for like, the hand-editing parts of Warp, by far, like, the more typical use case is for code review of an agent's code than it is like, typing code. But it is, it would be nice to have, it's just not a, not an area of high priority for us.

• 26:10Speaker 01

  Yep, got it. Awesome. I'd love to talk a little bit about how you see the future of coding interfaces and see the future workbench evolving. So it seems like very much like, you believe there's a convergence happening between kind of the traditional, call it Microsoft Word, IDE, GUI, approach to writing code, and then the chat with your computer, agentic, kind of, style of, you know, terminal-first approach. And those, those two are starting to merge. What other kind of UI innovations do you think are happening in terms of how people work with coding agents?

• 26:48Speaker 00

  I think the biggest change that we're going to see in the next year is more and more, like, kind of like, cloud agents, or we call them ambient agents, where, and this is already happening, like, we're investing in this at Warp, where rather than a developer sitting at a keyboard giving a prompt, there's some system event that triggers an agent to do something. And that system event could be like, you have a server that's crashing, you have a cluster of user reports, someone has reported like, filed like, a security incident against you. And all those things are basically going to serve as context into an agent that gets launched and runs, not on some individual's machine, but like, somewhere in the cloud.

• 27:32Speaker 00

  And so what I think that implies is that you're going to want

• 27:38Speaker 00

  The the sort of like workbench to become more of an orchestration platform, more of a uh like kind of cockpit for managing not just your own agents, but your team's agents. I really think it implies you're gonna need like a strong team concept. Um, because you know, the these things aren't, it's not gonna be the normal workflow of like, I'm sitting at my desk like, I I'm writing a coding change and I push a PR. It's like agents are gonna push PRs and then agents are probably gonna leave an initial round of reviews on PRs and they're gonna file tasking or task tracking system. And so all this stuff needs like, it needs tracking.

• 28:18Speaker 00

  Um, and it needs coordination and you need different ways of integrating it into existing systems. And so whoever like, this is like what warps probably like our our biggest product focus for next year is on this type of of uh evolution off of just like interactive agents into the cloud agents, because I think it's gonna, it's gonna be pretty transformative.

• 28:43Speaker 01

  And I imagine that's like a massive infrastructure push to be able to kind of, you know, run up and spin up.

• 28:49Speaker 00

  It is. It's it's it's turning, um, yeah. So like for us, it's turning us much more from a like a product to a platform. And so the way that we think about building this out is building it on different layers of the stack where you have like an agent SDK, you have um uh agent hosting if you want it. So like, you know, if if you're a smaller company, you don't want to like set up spots in the cloud for your agents to do their their their work. Warp will host that for you. There's a whole category of startups that are going into this business of agent hosting, uh, which I think is really interesting. It speaks to like, this is like a real thing that's happening.

• 29:30Speaker 00

  Uh, there's an API layer for once you have the agent running, how do you get its status? And how do you, um, how do you maybe take it over or see its progress? Where's it right, its logs? Uh, and then there's like a management layer of like, what are all these things doing? What states are they in? What's the log like, you know, who started them? When did they produce PRs? And so I think it's cool because it's going to be the most impactful way to use these agents a lot. It's just not to have a person like driving them. I don't think that this means that the person driving the agent is going to go away either. I think that there's like, it's a, the, the sort of types of tasks that are going to start with these ambient cloud agents are more like toil tasks or things that are one-shot-able.

• 30:14Speaker 00

  And I think harder, more interesting software engineering is still going to be done by a developer like at their workbench. But, um, yeah, this is how I, I see things evolving in the next year.

• 30:23Speaker 01

  That's awesome. And I think one of the more important scaling law charts is like the, the meter, like how long can the, the agent run for?

• 30:31Speaker 00

  Yeah.

• 30:32Speaker 01

  What are you seeing in terms of how kind of long horizon these agents can be?

• 30:37Speaker 00

  I don't know. At its max, I would say like doing real coding tasks for us now is like 20, 30 minutes, something like that. Maybe like you can have it run longer, just to be clear.

• 30:51Speaker 01

  Yeah, you can have it run in circles.

• 30:53Speaker 00

  The problem is it will start going in circles still. There's still context limitations and like, it's a, it's a costly proposition to have agents running without people checking in and guiding them. And just by far, you get the best results when the agent is really steered. So when you do like an upfront plan with the agent, when you check in on the agent's work. And so I think, yeah, I think this will just keep on going up, but the. I don't, I don't know. It's like hours and hours of work. It needs a really clearly defined task in order for that to even make sense to me.

• 31:35Speaker 00

  Needs to be doing like some like big code migration or some, some big, big task.

• 31:39Speaker 01

  Got it. What do you think the product might look like when it's good at kind of being this cockpit to manage, you know, swarms of these agents?

• 31:48Speaker 00

  Yeah, we're, so we're, we're building this out right now and like we're having all sorts of internal debates on whether it's, it's, it should be one product or two products.

• 32:01Speaker 00

  The way that we're doing it right now and how I think other people will probably approach this is like a sort of like area of our app, which is about agent orchestration. The reason I wonder if it should be a whole separate product for us is because like,

• 32:18Speaker 00

  you know, it's, it very much, it feels more web centric to me, which we, we can, we can make warp work on the web, but it's not the primary interface. It feels like potentially it has like a different user some of the time.

• 32:34Speaker 00

  But the advantage of having it bundled into warp is that it makes the handoff from one of these cloud tasks to a developer extremely seamless. And so very, very common workflow. Like we have this thing, like we have it running in our Slack and our linear. And so what will often happen is like you'll tag something in Slack and you'll be like, you know, can you make this fix for me? Change this button position or whatever. And right now you need a developer to like tie the loop on that. So it'll do the work in the cloud and then you'll bring it on to your local machine. And it's very nice to have that be in one environment where you can just keep working on it seamlessly. So short answer is I don't know, but it'll be a little bit more like a task management UI.

• 33:18Speaker 00

  I don't think it's going to quite like, I know there's, there's like thoughts of like, is task management the primary primitive for developers to be working with? I don't buy that either. Like, I don't think like every developer is going to be doing all their work out of linear Jira. But I do think there's some aspect of seeing what the agents are doing across various systems that developers are going to want.

• 33:38Speaker 01

  Yeah. Awesome. I'd love to close by maybe talking a bit about the state of agentic development and, and how the software engineering market will play out. Sound good?

• 33:48Speaker 00

  Sure.

• 33:49Speaker 01

  I guess maybe for starters, where do you think we are in terms of like the frontier model? You know, capability frontier like where are the models good today? Where are they not?

• 34:02Speaker 01

  You know,

• 34:02Speaker 00

  Yes.

• 34:03Speaker 01

  still producing all their errors? Like where are we?

• 34:07Speaker 00

  So I'm constantly using this stuff. I'm somewhat biased because I use it on Warp's codebase, which is like a very custom big Rust codebase, but I think that's still an interesting perspective. The agents can do what I would think of as like medium complexity tasks pretty well. If you give them a bunch of guidance, they can't do like whole big projects. At least we haven't had success doing that. They can't, I don't trust them to make like very fundamental architecture decisions for us.

• 34:47Speaker 00

  So it's like you want like pretty constrained tasks, but they're well beyond doing trivial tasks like change the button color, take the text like that's that like they can make apps. They're very good at zero to one. They can solve like kind of hard bugs. We have a medium sized feature like, I don't know. What a good example would be like, I was adding a new slash command to Warp the other day and it's like, I just tagged the agent to do that, you know, in Slack and it made a 300 line PR and it was basically right. And so I think there's a bunch of headroom at the upper end.

• 35:24Speaker 00

  I if if I had to put it on like a scale of zero to 10, I think we're at like a six maybe. So I think it's like, it's real. It's game changing for how people work, but it's not at like the level of doing what a full time engineer on a hard product needs to do.

• 35:42Speaker 01

  And where do you think the bottlenecks are? Like, is it just, you know, the models don't have enough context? We need to get better at giving them instructions? Is it just we need to keep scaling these things up? Like, what are the biggest bottlenecks?

• 35:53Speaker 00

  So I definitely think context window is still a big issue. And even with the bigger context windows, it having like attention over the whole context window in like a reasonable way is is hard.

• 36:06Speaker 00

  I think like there's like an issue of it always having to like relearn everything. Like, memory is not, it just seems like a slow, inefficient, repopulate the whole thing with a bunch of files. Take like, there's no like continuous learning with it. So it's, it's like this big stateless thing where you're kind of always starting from scratch and have to fill it up before you can set it loose. That's, that sort of stinks. I would, I would like to see that solved.

• 36:40Speaker 00

  There's still like, how do you use it effectively as a developer? We're very early, like this stuff didn't exist a year ago. And so how should you be doing context engineering? How should you be setting up your project so that agents can work well with them?

• 36:59Speaker 00

  That's like a problem. We, if you were to look across how people on our team use Warp to build, it's like high variance and, you know, that's not great because it's like, we have very, very great, we have very, very like rigorous standards around writing code and like almost no standards. I mean, we've tried around like how to use the agents. No one has been taught how to use the agents. There aren't even agreed best practices on how to use the agents. And so I think that's pretty nascent.

• 37:28Speaker 01

  Yeah, got it. My experience whenever I try to vibe code a little bit is that the coding models still produce a lot of errors.

• 37:35Speaker 00

  Yes.

• 37:36Speaker 01

  Is that going better over time? And that seems like to me in the category of stuff of like, if you can verify it, like, did it work or did it not? You should be able to RL it. And it's like, where are we today in terms of the state of, you know, how frequently are errors coming out? And like, can we actually RL that? Or am I misunderstanding something?

• 37:58Speaker 00

  No, I think that they're still definitely producing errors.

• 38:03Speaker 00

  It's, it's interesting. So it's pretty infrequent that the agent at this point will produce something that doesn't compile for me, which I think is an interesting milestone. So like,

• 38:15Speaker 00

  uh, I don't know, not that long ago, four or five months ago, that was a problem, like getting to a compiling version of the thing. It, it compiles for me about a hundred percent of the time right now, which is amazing. Um, it produces stuff with bugs and errors relatively frequently. I don't think it has a good way of closing the loop uh, in terms of does the thing work? And so I think some version of browser use or computer use where the agent can not only make the change, but verify the change from the user's perspective, not the code perspective is pretty important.

• 38:55Speaker 01

  Are people doing that yet?

• 38:57Speaker 00

  Yeah, I like, we're working on stuff like that. Like the, the, the computer use, all of the model providers are, have like beta versions of like computer use APIs and,

• 39:09Speaker 00

  um, you know, browser use for sure. Computer use, we're looking at like, I, I would be surprised if this wasn't a thing. And I, I think it becomes even more important of a thing pretty soon as more work is done remotely because the real pain in the ass with the remote work is verifying that it works from a user perspective. And so I think that's like a big part of it. And then I think if you have that loop, it's probably easier to do RL and get to things that are behaviorally correct, not just like static compile correct.

• 39:37Speaker 01

  Yeah, yeah, absolutely. Okay. Well, looking forward to that. Um, and then I guess, do you think that we're going to reach a super intelligence moment here? Like where, where the models are better at coding than the best human coders?

• 39:50Speaker 00

  I have no idea.

• 39:51Speaker 01

  No idea.

• 39:53Speaker 00

  What I do think is gonna happen is I, I think, um, I don't know if this is super intelligence. I do think like coding will be solved by models. Uh, and what I'm, what I mean by that is like, I think that the limiting factor that we're going to come up against is just like expression of intent from, from humans in terms of like, what do you want built? How do you? How do you build it? Like, how do you express that clearly? Like, English is ambiguous.

• 40:20Speaker 01

  Isn't coding the the truth expression of intent, though?

• 40:23Speaker 00

  Yeah, but that the problem is we're moving from a world where people speak in code to one where they just speak in English to try to build apps. And so you we're like reintroducing ambiguity because developers, people building apps are no longer actually directly expressing what they want. They're going through this translation layer of telling it to saying to a model what they want. And then the model produces the code. So it's an interesting, it's like an interesting step backwards there in a sense, but it's also way, way, way more efficient to do it this way. But yeah, I think like we'll get to a point where you actually don't need to be on the frontier to have something that produces code that is

• 41:06Speaker 00

  as well matched to a person's intent as possible. So I and I think that actually is an interesting thing from a competitive perspective. I wouldn't want to be in the API business for coding tokens, because I do think like at some point, there's gonna you won't you just want you to be on the frontier and not be able to charge a huge margin on top of it. Which is why I think actually you see Anthropic and OpenAI and Google going so hard at the application layer because there's huge risk at the API layer is just for this vertical in particular that I think things are basically solved within a few years.

• 41:45Speaker 01

  Yeah.

• 41:45Speaker 00

  I don't know that. That's just I'm prognosticating.

• 41:49Speaker 01

  Well, that's awesome. Do you think that people will ever are are people already thinking about the amount they spend on coding tools being, you know, the replacement of what they would be spending on, you know, hiring few software engineers? Or are they thinking about in their heads as buying a tool still?

• 42:08Speaker 00

  So when we talk to enterprises, it is it is still viewed as like by and large is like a productivity boost.

• 42:17Speaker 01

  Yeah.

• 42:17Speaker 00

  And that's like the way that it's being evaluated. In fact, it's really hard to measure even what like the effectiveness of this stuff. And so it tends to fall back to subjective measurements from engineers. Like, do you feel like you're getting a bunch of value out of this or not? Or maybe you look at like DORA metrics or like, it's really hard to like to know. So I don't think that they're viewing it yet, by and large, at least as as labor spend. And I think today, if you pitch like, here's a $200,000 agent to replace your $200,000 engineer or whatever, they would be like, what? Like, no, like, not even, not even close. Like, um, so, but I would expect that this starts to change.

• 42:58Speaker 01

  Why do you think we'll change that?

• 43:00Speaker 00

  It's a great question. I think it's like in increasing the automation use cases. Or maybe another way of thinking is like, if if companies start to launch products without engineers, I think that that will be like a major proof point. And to be clear, I don't want this to happen. I'm like an engineer at heart and and I don't want people losing their jobs. But there will be projects, products that are launched where there's like very, very minimal engineering involved. And you're going to look at the spend for that and be like, okay, this was the cost of delivering the product. And you're going to be like, okay, with and without engineers, what's that like? So I think I think you need more of that to happen. I don't think that's happening very much yet.

• 43:45Speaker 01

  Hmm, got it. And then maybe last question. I'd love to chat about how you see coding as an art form and therefore, you know, your role in the world evolving. You wrote this blog post that I loved back in 2023, I think. Everyone should go go give it a read. It's called, I think it's about the future of productivity interfaces being ask and adjust. Maybe say a word on that and and how you think, you know, three years in, how you think that's evolved?

• 44:15Speaker 00

  Yeah, so I wrote this like pretty shortly after ChatGPT came out and we started like trying to deeply integrate it into Warp and the The idea was, it's gonna sound really obvious right now, but the way that like productivity interfaces have always worked in the past was that that they were geared towards hand editing, right? And by hand editing, it could be like you go into Figma and you're like drawing vectors, or you go into Google Sheets and you're entering cells, or you go into VS Code and you're typing code. And my thesis in that article was like, that's going to change to a point where the primary interface is one is a, I didn't have the word agentic at the time, but I, it was like AI

• 45:00Speaker 00

  based, where you would ask, ask the app to do, to make, do the thing for you. And then you as a human author would be responsible for adjusting. And adjusting might mean like re prompting, or it might mean, if re prompting failed, it might mean like going in like treating the prior hand editing interface as like, like using that to like complete your change. And I kind of think that's where we're at right now, for a lot of like, especially for coding, it's really transitioning to, you start by asking for something and then you adjust it. And another thing I said in that

• 45:46Speaker 00

  article, which I don't know if it's right or not, was that I was thinking about, are you going to be able to get rid of the adjustment piece? And my thesis was that the, the area where you're going to need the adjustment piece the least is in areas where there's like a lot of like acceptable solutions. So that would be like Creative Domains. You know, if you ask for an image of something, there's probably a thousand outcome, a thousand images that might work for you. And so you can just re prompt, re prompt, re prompt until you get what you want. Whereas for something like code,

• 46:21Speaker 00

  or a spreadsheet where there's one thing that needs to be right, that you would have to keep that ability to like, get it perfect with a hand editing interface. So that was the thesis. I think it's, it wasn't bad. I think it's held up okay.

• 46:34Speaker 01

  Not bad. Yeah, I guess you didn't coin Agentic everything back then.

• 46:39Speaker 00

  No.

• 46:40Speaker 01

  Yeah, the pieces was flat on.

• 46:42Speaker 00

  We did. You want to know something we coined at Warp?

• 46:45Speaker 01

  What'd you coin?

• 46:46Speaker 00

  Which we should have like trademarked is Agent Mode. So we were the first product to launch a branded thing called Agent Mode. And if you like look this up on ChatGPT and just like ask like where this come from, it came from Warp. And now that's like a very common like way of describing the feature, which I wish we were getting, you know, some kickbacks for that or something.

• 47:11Speaker 01

  Totally. I love it. Well, thanks so much for coming on to share what you're doing. And, you know, your observations on the coding market as a whole. It's such a white hot competitive market and the way that you think, you know, the terminal will be the workbench of the future and how it's going to evolve. It was awesome to have this chat today. Thanks, Zach.

• 47:30Speaker 00

  Thanks, Sarah. It's awesome to be here.