Scaling Agent Workloads at Vercel

Most AI agent setups today are built around a single session, where one user interacts with one agent at a time. However, that model breaks down when an agent has to serve a business, where thousands of requests can arrive at once and each session needs to be isolated, durable, and recoverable. Getting agents to run reliably at that scale has meant a lot of hand-rolled infrastructure beneath the agent itself. eve https://eve.dev/ is an open source, cloud-native agent framework from Vercel https://vercel.com/ that removes much of the agent scaling burden. In the eve framework, an agent is defined declaratively through configuration files and these files compile into infrastructure as code so the platform provisions only what the agent actually uses. Andrew Barba https://x.com/andrew_barba is a Member of Technical Staff at Vercel, and Shar Dara https://x.com/shardarais the Product Lead for eve at Vercel. In this episode, they join Kevin Ball to discuss what it means for an agent framework to be cloud-native, why they chose to express agents in plain English, and their view that company building is becoming agent building. Sponsorship inquiries: [email protected] mailto:[email protected] The post Scaling Agent Workloads at Vercel https://softwareengineeringdaily.com/podcasts/scaling-agent-workloads-at-vercel/ appeared first on Software Engineering Daily https://softwareengineeringdaily.com.

• 0:00Speaker 00

  Most AI agent setups today are built around a single session where one user interacts with one agent at a time. However, that model breaks down when an agent has to serve a business where thousands of requests can arrive at once and each session needs to be isolated, durable, and recoverable. Getting agents to run reliably at that scale has meant a lot of hand-rolled infrastructure beneath the agent itself. Eve is an open source, cloud-native agent framework from Versale that removes much of the agent scaling burden. In the Eve framework, an agent is defined declaratively through configuration files, and these files compile into infrastructure

• 0:40Speaker 00

  as code, so the platform provisions only what the agent actually uses. Andrew Barba is a member of technical staff at Versale, and Shar Dara is the product lead for Eve at Versale. In this episode, they join Kevin Ball to discuss what it means for an agent framework to be cloud-native, why they chose to express agents in plain English, and their view that company building is becoming agent building. Kevin Ball, or Kate Ball, is the Vice President of Engineering at Mentoo and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript Meetup, and organizes the AI in Action Discussion Group

• 1:22Speaker 00

  through Leighton Space. Check out the show notes to follow Kate Ball on Twitter or LinkedIn, or visit his website, kball.llc.

• 1:42Speaker 01

  Andrew, Shar, welcome to the show.

• 1:44Speaker 02

  Thanks much for having us. Happy to be here.

• 1:47Speaker 03

  Yeah, thanks for having us.

• 1:48Speaker 01

  I'm excited to get to know you guys and to learn a little bit more about Eve and this new Agentic framework, but let's start with you. So we'll go to Andrew first. Can you tell us a little bit more about your background and how you ended up working on this project?

• 2:00Speaker 02

  Yeah, so I've been at Versale for some three and a half years now. I was actually Versale customer, maybe one of Versale's very first enterprise customers back in 2020. So a bit of an unconventional path to Versale, but yeah, I spent my first year and a half or so doing all CDN works, a lot of like deep systems, distributed systems, edge networks, firewalls, security stuff, things like that. And then basically went over to the other end of the business and worked on billing for a good year, which is where I met Shar. So Shar and I were leading, we're leading the billing team at Versale for most of 2025. And then, yeah, February came around this year and we had a really interesting opportunity to kind of look at what does it take to deploy agents to Versale? And we came up with Eve.

• 2:47Speaker 01

  Awesome. And Shar, how about you? Your background and how you ended up here?

• 2:51Speaker 03

  Yeah, I've been at Versale for a year and a half. I was in the billing space for a decade before, so that's where I joined Versale and the billing team where I worked with Andrew. And earlier this year, Andrew's like, you want to join, you know, build Eve? And I, of course, I loved working with him, so great opportunity.

• 3:10Speaker 01

  Awesome. Well, let's start then with a little bit about Eve. You know, it's brand new. A lot of folks probably haven't heard of it. What's the kind of high level of what this thing is? And then we can dive into the details in a little bit.

• 3:22Speaker 02

  Yeah, I mean, so Eve, it is a cloud-native agent framework. And the reason I say cloud-native is I think most people, when they're using agents, they're probably running a process on their computer. And so they're familiar with working with these things locally, very familiar. So, you know, your codexes, Claude codes, open codes, things like that. And these are obviously very, very powerful harnesses that people are interacting with on their machines. But we really wanted to look at, how do you use something like this in a multiplayer cloud environment? And what does putting these things in the cloud actually need you to do? And turns out, there's a lot of different products that go into making that happen. And so Eve is a framework that basically stitches together with very good defaults,

• 4:03Speaker 02

  all of those types of products to give you a very robust, you know, out-of-the-box harness that lives in the cloud.

• 4:10Speaker 01

  That makes sense. So one thing that immediately strikes me to follow up on there is like, okay, what makes for it to be different if it's cloud-native, right? And I'm thinking about all these folks who got excited about OpenClaw, and they got their own Mac Mini, and they're running it. And it's not exactly cloud, since it's running on their desk somewhere, but it is, you know, isolated off in this place. But what does it mean to be cloud-native in this context?

• 4:33Speaker 02

  Yeah, yeah, great, great question. The example I love to give is, you know, let's take an OpenClaw and let's take an Eve agent, and let's let a thousand people send a prompt at the exact same time. And what, you know, what's going to happen? And so the difference between the two is Eve on Versale or another platform that you host on will scale out horizontally to meet that demand, right? We can spin up these harnesses very, very quickly in parallel. We can isolate these sessions. They're durable. A whole bunch of mechanics behind it for like recovery and retries and error handling and things like that. And so it's a very different use case than just you communicating back and forth as one person with one agent, right? This is really meant to drive businesses, whether that's internal agents for a 10,000

• 5:17Speaker 02

  person enterprise, or if you are, you know, some B to C company and a agent is your primary product, Eve will support that type of multiplayer type of model. And it can do this very, very scaleably.

• 5:30Speaker 01

  Nice. Okay. So let's maybe then go one level of detail further. And let's start with, as you mentioned, there's a lot of folks doing agent frameworks out there, many of them focused at, you know, running on a laptop or a single machine or things like that. But what kind of prior art were you able to bring into this? And what were the things you said, hey, this is actually something brand new? We got to solve this.

• 5:51Speaker 02

  It started from our own data agent that we built inside Versale. This is called D0. And this was really the first agent inside of Versale that the entire company was interacting with. And it was a lot of work, a lot of hand-rolled systems to go put D0 onto Versale. We had to wrap a lot of our products, like our service functions and sandboxes and our workflows and various other Versale products. And it was a ton of work by that team, basically just getting it to run correctly before even thinking about what D0 does, right? So it felt like there was this very large layer sitting below D0 that basically every single agent that we would then deploy to Versale was going to need. So that was the first piece.

• 6:36Speaker 02

  And then when we actually looked at D0 and what it was, so much of D0's functionality actually came from Markdown files, YAML files, other types of spec files, because D0 being a data agent, it's basically trying to provide this semantic layer, right? When you say how many monthly active users are there, maybe that's, you know, too obvious, but the semantic layer is telling the agent what these types of words mean. And so when certain people ask questions, they're using different acronyms and things like that. So there's this entire layer that's basically educating the agent. And that idea is really where we started. So we kind of took two approaches here. We said, well, it has to become way easier to deploy an agent to Versale in the same durable fashion that D0 was doing.

• 7:19Speaker 02

  And we thought that a declarative file system was really, really interesting. We think a lot of other agents can almost be entirely expressed in English. And so we took this very dramatic stance in the beginning where we basically said there was no code. Eve was going to be purely English Markdown text files. And so, yeah, with those two extremes, that was kind of like the very, very early idea behind what Eve is now today.

• 7:44Speaker 01

  Yeah, that's super interesting. Well, and I go back to, I remember a conversation I had, I think with Tom Okino at some point about Versale's vision of automatically provisioning infrastructure and being able to infer that from the application. And that requires having applications with certain sets of constraints and structure and all of those different pieces. So what does this end up looking like for someone who wants to develop an agent? Like, what is the Eve? Is it an SDK? Is it something where it's like, plug in these Markdown files in this place? Like, what does this actually look like?

• 8:16Speaker 02

  Yeah, it is literally a directory. And the most simple Eve agent could actually just be an instructions.md file, and that's it. And so, so yeah, it's of course a framework because the grammar behind Eve and our slots, those are all compiled down into this, into this manifest. And that manifest is basically the infrastructure as code that you kind of just, just reference where, you know, we're provisioning things based on what you have. So if you add skills, for example, then you get a sandbox. If you define schedules, you get our event infrastructure for schedules, right? So, but at its core, it is a, effectively, we call it, you know, ChatGPT in a box. If you drop in an instructions.md, you are getting a frontier model as trained, and you're getting a cloud-hosted version of it.

• 9:01Speaker 01

  Interesting. So this reminds me back of like the original page router days of NextJS or something like that. So what are the conventions? You said instructions.md, you've got a directory, you put things, you referenced here schedules or skills. Like, what are the different primitives you have to play with here?

• 9:19Speaker 02

  Yeah, great question. So yeah, and instructions is the identity, of course. And so I think most people are familiar with this. Skills is the things that you want to progressively disclose. So I think skills have been around for maybe a year, a year and a half now. And I think people are generally pro adding skills, but I don't think necessarily understand how it works behind the scenes. And so the key thing to understand with Eve or any of these harnesses is that is, that is your progressive learning mechanism. So skills are basically giving a hint as to when they should be used. And the models are really good at understanding when they need to learn more about something. And so they can take that hint and then basically pull in additional context. And so D0 is like 90% skills. And then tools, of course. So

• 10:03Speaker 02

  you most likely want to run deterministic code, pull in, you know, deterministic set of data from some outside source, maybe, you know, like a Postgres database or some sort of, you know, open API connection. And so tools kind of facilitate that. And then we have a specialized version of tools called connections. And connections are for things like MCP. And so Eve has a first-class way to just drop in an MCP URL, and we automatically expose those tools for you. So you don't have to think about it. Those are like the core primitives. I think that's kind of like the brain and the operations of Eve. Then the next layer that is fairly critical is how do you invoke it? Right? You need to start the agent somehow. And this is what we call channels. And so

• 10:45Speaker 02

  channels are basically all your entry points into your harness. And so we have a default channel, which is just some HTTP route. And this is like what our TUI uses. And if you were to like hook it up to a NextJS app, you're basically using that default Eve channel. But the more interesting ones are things like Slack and Teams and GitHub. And we basically support, you know, kind of all the first party events that you'd expect from those platforms. And we come up with some opinionated defaults for how we think you probably want to use your agent inside of those platforms. And we of course, you know, customize it beyond that. So that's really like the core primitive. And then there's, it expands quite a bit out from here and much more advanced use cases. So we have this concept of subagents, which in its most simplest form is effectively

• 11:29Speaker 02

  context management. You're kind of taking some larger set of tasks that you don't want to blow in your parent agent. And so the subagent can go take that and work with it on its own and then kind of report back. But subagents also enable completely different ways of kind of building teams around agents. So like internally in Vercel, we use subagents where like literally the entire teams own a subagent. And then we have, you know, kind of another agent that interacts with those. So D0, for example, is actually a subagent of another one. And so the data team owns that. Now, we have a help, you know, Eve agent, for example, that knows all about how to build Eve. And so the rest of the company can, you know, can invoke that. And that's effectively a subagent of our, of our larger agent as well. So yeah, subagents enable kind of like really a different way

• 12:11Speaker 02

  of kind of organizing who owns what. So those are pretty interesting. And then something else that I'm very excited about that is first class in Eve is evals. And so we think it's very, very important when you build a non deterministic system like an agent, that you can actually test it and monitor it over time and see how it's performing, right? And that could be based on changes that you're making. It could be testing a new model or different levels of reasoning and things like that. So evals is like a very first class primitive inside of Eve.

• 12:40Speaker 01

  I want to double click on that because evals is something I think everyone is talking about and trying to do. But it, at least in most of the folks I talk to, and certainly what I see just diving in with people in places, it still feels a lot harder than it needs to be. So how do you set up an eval in Eve? Is it a sort of static test case? Are you doing evals on live queries? Like, what does that whole lifecycle look like?

• 13:04Speaker 02

  Yeah, so we have two different types of evals. So the first type is more deterministic. It's probably more what you're familiar with, where you're looking for conditions to be met. And these are fairly deterministic things. And Eve's example of that is like, you might want to send a query and then make sure that a tool was called. So not perfectly deterministic because the models might not call it. But in general, you can say, you know, okay, I have this tool defined. And, you know, let's say you're building a weather agent. When you ask, what is the weather? You would like to make sure that the get weather, you know, tool is called. So those are like a little bit more deterministic, very straightforward to write. And then we take this one step further where we let you do, I believe the term is LLM as judge. And so you can actually, you know, reason about responses and kind

• 13:49Speaker 02

  of grade and score these types of responses. And so this is, you know, for D0, this is where things can get really interesting and much more complex, where we can actually grade the SQL query that might be generated and make sure that it's looking for the right, you know, tables and things like that. And so, yeah, in eval in Eve is literally a, it is a defined eval function. And you basically get this test function that you can write what you want. It's effectively a full interface to your agent. So you can, you can start turns via prompts, you can trigger things like human in the loop. And so basically all the functionality of Eve, you can, you can programmatically write and set up an eval. And you can do as much or little with it as you want. What I think is particularly interesting about the way we have the Eve repo setup is our test suite is in fact a

• 14:33Speaker 02

  full set of evals. So we actually use our eval framework to test Eve. And that is basically what gives us the confidence to, you know, ship new versions like many times a day.

• 14:44Speaker 01

  Yeah, no, that's super cool. Okay, I'm going to push on this. You may not have anything. I haven't seen anybody really do this, this yet. But it's a thing I'm kind of minorly obsessed with. So what we just described is essentially evals as guardrails, right? Like, we have something we believe to be true. We're using evals to make sure it remains true. It's kind of a AI or L, non deterministic variation on unit and functional tests. A kind of direction that I'm interested in is this concept of essentially turning prompting into an optimization function, right? So your evals lay out what should be true as an outcome. And then you use maybe an LLM or something else in the loop to actually iteratively evolve your agent to get it to behave the way you expect. Is that something

• 15:26Speaker 01

  that you've played with at all in Eve? I'm totally fine if not. I don't know anyone has. But it, when you started talking about this, I was like, oh, does this make it easier to create that optimization loop?

• 15:37Speaker 02

  Well, yeah, this is really interesting. So I had never thought about it in this way. But if you use a coding agent to build your Eve agent, I think you'll find that it will actually write and run evals while it's iterating. And part of this is because of the way we expose our docs inside of the package. And so we hint that you test Eve with evals. And so the coding agents actually understand this. And when we work locally on Eve, we have a bunch of fixture style apps, the coding agents are running those evals. So I think it wouldn't be asking too much to actually prompt your coding agent to get what you want out of here. And I think, I think it would, it would do like a pretty reasonable job at using those evals to iterate. I've never quite seen it like write an eval

• 16:20Speaker 02

  first and then go build the implementation. But, but I think, I think you could probably get something pretty close to that.

• 16:27Speaker 01

  Nice. That's cool. And I really like that idea of thinking, I think more and more frameworks are starting to do this where they think of our documentation needs to be designed, not just for humans, but for coding agents, right? To expose the right things, put the right guardrails in place.

• 16:42Speaker 02

  Yeah, we had a lot of learnings from like, from NextJS in that regard, where the models are trained on much older versions than Next. So it's like, how do you get it to learn the latest thing? And, you know, obviously Eve doesn't exist as far as these models are concerned, right? So yeah, so we spend a lot of time making sure that the package had to come with LLM friendly docs. And I think we've, we've seen really, really good payoff there.

• 17:04Speaker 01

  This episode of Software Engineering Daily is brought to you by WarpBuild. AI is writing more code than ever, which means GitHub Actions is running more than ever. Your GitHub Actions bill is now a function of how much AI code you generate. And every engineer knows the feeling. You push a commit and then you wait. WarpBuild makes GitHub Actions twice as fast at half the cost with a one line change to your workflow. Linux, macOS, and Windows runners in WarpBuild's cloud or your own, enterprise ready, SOC 2, Type 2 attested and trusted by teams like Sky from Comcast, Bitcoin, and Brain Trust AI. Get started with $50 in free credits at warpbuild.com/SED.

• 17:42Speaker 00

  Every visitor on your platform looks the same at first. Can you trust them? Or are they a threat? Fingerprint lets you answer this question right away. With one API call, no model to maintain, and implementation in minutes. Fingerprint recognizes bots, AI agents, VPNs, tampering, and over 100 more unique signals to give you durable risk scoring and threat detection in real time, even as human and agentic activity blends together. You can identify suspicious traffic faster and build safer and more secure sites and apps. Over 6,000 companies rely on it every day. Join them now at fingerprint.com. Think about your mobile app source code. Once it hits the App Store, it's out in the

• 18:27Speaker 00

  wild. And without the right protection, decompiling is easy for malicious actors looking to steal your IP or tamper with your software. That's where GuardSquare comes in. GuardSquare provides the highest level of mobile app security for Android and iOS applications and SDKs. Their advanced tools integrate seamlessly into your CI/CD pipeline. We're talking polymorphic, multi-layered code hardening techniques and automated runtime application self-protection, paired with mobile application security testing and real time threat monitoring to deliver the highest level of mobile app security without compromise. Don't leave your hard work exposed. Secure your mobile applications today. Go to guardsquare.com to learn more.

• 19:15Speaker 01

  Curious, actually, any lessons learned there? And one of the things I'm thinking about here is similar to skills. I feel like you probably want some sort of progressive disclosure or something like that, because otherwise the model may get overwhelmed. Like, you don't want to stuff the model with all the context about your whole framework all the time. So like, how did you design those to make that work well?

• 19:35Speaker 02

  Yeah, two things. So our scaffolding includes an agents that MD file, which of course points at the docs. And we also include our change log in the package. And this is really, really useful because the agent basically knows that it only has to look at certain things, like when you start like upgrading, you know, the E versions and things like that. So it doesn't have to start from a clean slate. It's really good at knowing, oh, I'm on version, you know, 0.2 and I see the change log is on 0.5, let me just, you know, look at the diff between that. And then in terms of like the docs themselves, the coding agent, like Codex and Cloud, wherever, they're so good at knowing that there's tons of files in a directory and not to scan them all. So like, we didn't really do much to optimize there, other than just picking a folder structure for the docs that we think would make sense.

• 20:20Speaker 02

  But the change log is really interesting, and especially for a framework that's pre 1.0, it's like, it's critical because we're breaking stuff, you know, every week. And so that's, that's kind of the main way that we can make sure people are upgrading without, you know, throwing a fit.

• 20:33Speaker 01

  Yeah, no, that's super useful. Actually, that's a technique that I think I've found for a lot of code bases. When you have multiple people in a code base at all, like having that change log really helps the agents keep up with what's going on.

• 20:45Speaker 02

  Yeah, yeah, exactly.

• 20:47Speaker 01

  Different direction here. So still in this like mental model of, okay, we're moving from my pet agent living on a box somewhere to these cloud-based agents. One of the big challenges I feel like a lot of us are grappling with is how do you manage things like permissions, authorization, like what is this agent allowed to see depending on who's interacting with it and that types of things? So is that something that Eve has primitives for? Like, how do I think about data access control and stuff like that in the Eve framework?

• 21:19Speaker 02

  The funny thing is, when we started Eve, like, we were pretty set on enterprise right off the bat. Like, Eve was, we knew it was going to be code first, and we knew that, or, you know, thinking about use cases like D0, like, we expected companies to go build their own Eve agents to run their businesses. And so now we're actually starting to double back on more of the personal use cases. But the reason I said it's like, a lot of the permissioning and whatnot, like, that, that was baked very early on. And yeah, there's, there's quite a bit built in, in terms of where we look at permissioning. And because there's quite a few layers, right? Like, there's, at the channel layer, we basically are saying, you know, who are you? And you define what that is. So for Slack, for example, the who are you is going to be your workspace ID plus your

• 22:03Speaker 02

  Slack user ID, right? And Eve was designed to be multiplayer from the beginning. And so we give you things like, and so basically everywhere you write code in Eve, you have a context object, and that context object is going to give you things like, who is the current user in this turn? Who is the user that started this turn, right? They can be different people, right? Different people can, you know, follow up at different times. And that kind of gives you the ability to do basic type of authentication, right? Or permissioning, where, or what I would say is like, the more traditional permissioning, where if you're writing any typical backend REST API, right? You're looking at some auth header coming in, you're authenticating it, and then you're saying, okay, you can do XYZ. And so Eve facilitates that basically by resolving that identity at the channel layer.

• 22:47Speaker 02

  Where it gets more interesting is, can the agent perform some action? And so we have this concept of tool approvals, which basically surface as these human in the loop, you know, style messages, and it's completely dependent on the channel that invoked it. So all you have to say is, you know, always require approval on some tool. And if the invocation comes in from Slack, we can render, that channel knows how to render human in the loop, right? If it comes in over SMS, it could take on a completely different form. And so those approvals give you a lot of control. You can look at, again, who asked the question and who is asking for approval? Who is then giving the approval? Maybe it's not the same person, right? So you may decide that that's okay, you may decide it's not. And so this is where Eve really shines as a code-first framework.

• 23:32Speaker 02

  So all these cases, we can't necessarily predict, right? Like, how people want to run these types of things. And, and we basically just, you know, say, hey, you know, go write the code to make the experience that you want. And so, yeah, so approvals are baked in. And then the last piece, which I think is maybe most interesting, and something that, like, we haven't even necessarily solved ourselves yet at Vercel, but you hinted at it, is how can you see what you're supposed to see? And this creates a whole bunch of really interesting problems. So for example, let's take D0. D0 effectively has access to all of Vercel's data lake, right? And so if Guillermo is asking a question, he may have access to things that, when I ask a question, that I shouldn't, right? And so how do you resolve my credentials versus his?

• 24:16Speaker 02

  And so we basically have a pretty advanced API, which we called Defined Dynamic. And this lets you resolve different sets of tools, different sets of connections based on the current caller. And so you could do things like literally not expose a tool when I'm the one invoking a turn versus, you know, say Guillermo. And so that's like, again, a very advanced use case. It is extremely well documented. So the agents do a really good job figuring this out. And you start prompting for this type of behavior, it will learn that this is basically, you know, the, the way to do it inside of Eve.

• 24:47Speaker 01

  That's super interesting. Before we go deeper on the tech side, something you mentioned, so you talked about going enterprise and then maybe dialing back to personal use and things like that. I'm going to direct a question over to Shar. Like, who are you seeing as the key user audiences for this? And what is the, the sort of rollout and usage look like?

• 25:05Speaker 03

  Yeah, as Barbara said, we did initially target enterprise, given that that was where most of the requests came from. And we have the infrastructure, so it was kind of a no-brainer. And also, we were scratching our own itch to an extent. So we actually use Eve internally for all of our internal agents. And now we have hundreds of them running on Eve. So that was the initial target audience. But very soon after we launched, we saw thousands of our users actually adopting it. And those are not necessarily just within the enterprise segment. You can see them around our hobby users or pro users building. So the range of use cases we've seen go all the way from business agents, what we call

• 25:48Speaker 03

  what enterprise is launching, where you automate a business function, down to a hobbyist building a iMessage multiplayer agent on a weekend. And we want to enable basically all of those use cases and let you extend the framework to your needs.

• 26:06Speaker 01

  And so I love that you, you started scratching your own itch, right? I think especially one of the advantages, working in a developer tooling company, is like, you can really do a lot of that. I'm curious, you know, if you were to look at the distribution of internal agents that you're building, how much of those are engineering-related versus other functions within the business?

• 26:25Speaker 03

  It definitely started within engineering, and engineering is the function that actually spun up a lot of these agents initially. But as we saw the adoption over time, it became go-to-market, move everything they had on Eve. Then marketing is working on automating content creation, socials, and so on, on Eve. And so on the, the rest of the team. So like, you see billing, that help Eve was an example. So you see examples of product teams that give the product documentation to the agents to answer questions, both on the docs and internally, and sometimes share those with customers directly on Slack. And so the distribution's still heavy on the engineering side, where those are the

• 27:09Speaker 03

  main users and builders of the agents. But I think you're seeing that distribution kind of get more neutralized as it gets easier to spawn up agents and build them as we get to like self-evolution and so on.

• 27:22Speaker 01

  Yeah, no, that makes sense. The advantage of the approach you guys are talking about in terms of like, hey, we make this really easy for coding agents to work with, is like, increasingly, coding agents are available to everyone, right? Like, it's not engineering-specific. Interesting. So I guess a question, are there any patterns that have been emerging in terms of how folks end up adopting this? And particularly, once again, thinking outside of the engineering context, like, I immediately am like, oh, yeah, I'm, I'm used to working with agents, I'm doing things and all of that. But as you start to see go-to-market adoption, as you start to see marketing adoption, are they approaching this differently? Or do their agents look very similar to the engineering-developed agents?

• 28:05Speaker 03

  Yeah, there are three types I've seen, I would say. The most common one is the conversational agents, where the agent is channeled through Slack, mostly. We are a Slack company, so most of the communication happens in Slack. That was the initial use cases. So like, D0, for example, which was the initial use case, did primarily work on Slack. The second one was kind of background agents. And these are mostly, the agent's job is to like, run some sort of a report and also send it to Slack or send it through email to internal folks. And the last one, which GTM, for example, is a big user of this, is actually applications.

• 28:46Speaker 03

  So actually build an agent, but your front-facing is an web app that you're working with. And that web app is actually powered by an agent, and you also have a chat interface with the agent on the web app. So those are kind of the three different types we've seen come out of it, but, yeah.

• 29:02Speaker 01

  So that makes me want to, I'm going to send this back maybe to Andrew then. So in the case where I'm building a web app of some sort, but it's got an agent-powered backend, how does that interact with Eve? Because you mentioned this is directory structure focused. Do I embed these directory structures inside my web app directory structure? Or are these independent repos running in different ways? How does this integrate with the rest of the development lifecycle?

• 29:28Speaker 02

  Yeah, so we wanted a pretty opinionated story with NextJS, right? So Next has the pages directory and then more recently the app directory. And so we knew the agent directory had to just work. So there's first party support for like that type of setup where the agent literally lives next to your, your NextJS app. But it does not have to be that way. And I think in many cases, like there's still lots of teams that are deploying a separate front end from a separate backend. And Eve can work absolutely just like that, where you deploy your Eve agent as its own project. And then you just connect to it over HTTP. And that's what that default, we give you the Eve channel, which is basically our HTTP protocol, our stream protocol. So it has things like sending messages and canceling and things like that.

• 30:13Speaker 02

  And then, and then ultimately we, we give you back this durable stream, which you can basically use to power any UI that you can think of. And so, yeah, it really is just an API. And so the way we think about internally is Eve is a backend framework. It is not a frontend framework. And so, yeah, it works with your front ends the way any of your other backends, you know, would.

• 30:33Speaker 01

  Okay.

• 30:34Speaker 03

  On the application side, we actually seen some really cool examples of our users, like shipping web apps with Eve. Like two of the examples we saw was, one of them was an open source CRM. So what happened was the user goes in, adds a lead, adds the name and email. And the background agent goes in and researches the lead and completes the whole thing behind the scene. And the second one was, I believe, an AI music maker, where you kind of go in and add the tempo. But then the agent kind of collaborates with you on the site and completes the music. So you kind of are seeing these agentic native applications that are being built. And the Eve agent really is like the backend brain behind the scene.

• 31:15Speaker 01

  Yeah, no, I think it's really interesting. And this is one of the spaces I feel like is underexplored yet. Like a lot of generative UI interfaces are like, let's throw a chatbot in this thing. Or let's asynchronously generate a report. But I think there's a tremendous untapped opportunity for almost collaborative interfaces. Like what does collaborative document editing look like when you have an agent involved? Right now, you can sort of do that with Claude and their artifacts. You could try to do it in Google Docs with Gemini, but I wouldn't recommend it. So I think there's a really rich space here for easy to interact with agentic pieces.

• 31:58Speaker 01

  So I'm curious, so Andrew, you said, basically you're exposing this durable stream and now you can do whatever with it. Is there any sort of primitive that is missing there? Like, I feel like that's pretty low level, I guess, in terms of, which is like, fine for first generation. But I wonder if there are also opportunities there in terms of what are the abstractions that make sense in interacting with an agent inside an app context?

• 32:23Speaker 02

  Yeah, it's interesting. And I think like the, so the, the Vercel Labs team is doing like really interesting stuff here. I think with, with tools like JSON render and, you know, taking kind of these lower level protocols and like producing UIs out of them. And Eve, the stream protocol that we have is very low level. You were literally getting like delta chunks of reasoning and things like that, right? So it's, it is a lot to work with. The TypeScript client makes this stuff much easier. So we have really nice APIs for iterating over the stream and strongly types, you know, phases and events and things like that. And so you can definitely work a little bit higher up, but you're right, not much. It's still a fairly low level API. So, yeah, I mean, I'd love to see a lot more done here. I do think Eve is a lot more than just interacting with a chatbot.

• 33:07Speaker 02

  Like you said, we have cool features where when you send messages, you can actually specify an output schema, for example. So, you know, this is really interesting because now if you know your UI takes some shape of data, you can agendically generate that thing. So this lets you do really, you know, completely different experiences than just, you know, asking a question and getting some, you know, English response back, right?

• 33:27Speaker 01

  Oh, interesting. So question on that immediately. Do you include built in like validation of that schema and serialization of it? Or like, is this just?

• 33:35Speaker 02

  Yes.

• 33:36Speaker 01

  Okay. So what is, what does that look like? Tell me more.

• 33:39Speaker 02

  Yeah, so we take the output schema and we basically plumb that all the way through down to the, Eve is built on, on a tool loop agent from AI SDK. So basically what we end up doing is we actually, we produce a tool called, you know, submit result. And that validates your schema and is retrievable, of course. So it's a lot of times these agents will, will call a tool like that and they produce the wrong thing. And then we can hint back to the agent basically saying, oh, you missed this required property and it'll do it again. So it's not guaranteed to get your schema, but especially with these frontier models and the way they are today, it's quite good.

• 34:14Speaker 01

  Yeah, they're good at that. Now, the challenge with tool calls is that you lose your streaming once you go to tool calls, right? So now you've suddenly got to wait for that whole block to do it.

• 34:23Speaker 02

  No, not exactly. So the, the stream will produce all the reasoning in between, right? Because it's just the final step. Instead of getting a message, right? It's, you know, some English message out, you're just getting that final tool call, but you get everything in between. So, you know, if you're using some high reasoning effort, and if you want to, you can inspect all of that, even though you're waiting for some sort of output. You could still see kind of everything, everything in between.

• 34:47Speaker 01

  Totally. No, no, no, I guess what I was saying is, yeah, you're streaming up until that tool call, but the tool call introduces asynchronous block, essentially. We're not getting pieces of the tool, like you're not getting a partial schema results along the way.

• 34:59Speaker 02

  Exactly.

• 34:59Speaker 01

  Which influences the UI you want to build, right? Now you have to have like a waiter or something like that, rather than streaming out the pieces of it as you go. But super interesting. Let's talk a little bit more infrastructure, getting back to this like cloud scale thing. So you mentioned a little bit of like, hey, if I got a thousand requests, suddenly I can just like spin up a thousand different processes to answer this. Very interesting. That now introduces questions around durability, resumability, all those different pieces. Like, how does Eve handle that?

• 35:31Speaker 02

  Yeah, so, I mean, it really started where we knew we wanted to build Eve on top of workflow, which is basically our version of temporal. I think a lot of people are familiar with. And so the way we modeled this very early on was a single workflow is a single Eve session. And today it's actually two workflows, but you could think of it as one. And so the, the partitioning is basically every session is, is completely separate from every other. And there's ways to share data between them, or I should say, we, you know, we provide guides on how you might want to share data between them if you bring your own storage. But that is the partitioning. And so a thousand sessions versus a single session, you shouldn't have to think about that. The cloud should do that work for you.

• 36:14Speaker 02

  But yes, as soon as that turn starts, you are, you are initiating a workflow. And then we do some clever things. So we, we run on, on service functions, which, you know, historically have 15 minute timeouts. These agents obviously can run for a lot longer than 15 minutes. And so we can do some really clever things here where, you know, every time we invoke a tool, we can move over to another function. And it just continues the workflow. And so.

• 36:38Speaker 01

  Because it's another synchronous break. Yeah, no, this makes sense. Yeah, yeah.

• 36:42Speaker 02

  Exactly. And in the future, we might decide to, you know, if we have a lot of time allotment left, maybe we run multiple tool calls on the same function. But today we actually, we moved to a new function for, for every single step. And so as long as, you know, workflow gets faster and continues to be performant, you don't notice this. Then the agents can effectively run, you know, indefinitely.

• 37:03Speaker 01

  That's cool. What about things like durable memory and stuff like that? So like, one of the things that's interesting, working with something like an OpenClaw or a Hermes or one of these things is that they have as a part of their built in loops, some amount of, oh, I'm going to remember this. I will write my own file that I can reference, or I will iterate on this skill that you maybe gave me a first version of. But I can, I can actually modify my own prompts and sources based on your feedback. Are there mechanisms for that type of learning agent within Eve?

• 37:36Speaker 02

  Yeah, this is probably the biggest thing I'm working on right now. So all of memory is open in PRs, which you can go inspect if you want to right now. But yeah, so I'd say all of this is, and this actually goes back to what I was saying before, where we started enterprise. And funny enough, like these types of things like aren't necessarily your enterprise use case, but they are very personal agent use case. And so now we're starting to double back and, and, you know, kind of build up these features for the personal case. And self-evolution is a really, really important part of that, for all the reasons that, that you just said, right? The idea is you just kind of deploy the out of the box agent and it just kind of starts getting better as it works with you. And there's different forms of self-evolution.

• 38:16Speaker 02

  And I think it's also different, you know, Eve is going to do it a little bit differently than maybe something like Hermes, which can literally just update its code in place. And then, you know, restart itself effectively. So on Eve, we're thinking about self-evolution in a few different ways. Memory is obviously one of them. And we are doing a first class folder called memory. And you will be able to define different memory slots. And so you can actually, you would be able to have, let's say you wanted like workspace memory for your Slack channel. So this is like literally global memory that's shared across all your users. But then you could define an additional slot that is just memory for yourself, right? And so now the, the agent actually has multiple memory banks that it can pull and save to, depending on what scope you define.

• 38:56Speaker 02

  And so it's kind of like a mix of, we'll give you like a really easy kind of personal memory provider. But we think it has the primitives for like much more advanced, kind of like enterprise memory use cases. So hopefully kind of like threads the needle on, on both sides there. But yeah, so memory is, is one form. And then the other one is obviously modifying yourself. So can you add a new connection to some, you know, MCP server or write new tools, produce new skills, things like that? Skills maybe is a special case, or it might actually be closer to memory. But for other things, you literally have to write code, right? To go make it happen in Eve. And for now, we're going to lean pretty heavily into Git, right? Like our customers are using Git. And so what we want to do here is open PRs.

• 39:39Speaker 02

  And so we will do a coding sub-agent that you can define. And that is basically your, your self-evolution. And so we'll kind of do all the hard work of, you know, when you deploy this thing, it'll have a sandbox for you with your code base. We'll have the right prompts, so it knows when it has to invoke the coding agent to change something. But the goal is to effectively just open PRs against itself. And so that's kind of our model of the VPS, you know, world where they just get to change their files.

• 40:05Speaker 01

  Yeah, no, that's honestly a really nice, clean model because it allows you to define whatever level of oversight you want on those.

• 40:13Speaker 02

  Exactly.

• 40:14Speaker 01

  You want to let itself merge? All right, you can build those automations. You want it to have heavy human review on everything? You can configure that the same way you would anything else. So yeah, that's really nice.

• 40:25Speaker 02

  Yeah.

• 40:26Speaker 01

  So you used a term slots, and this is, I think, the second or third time you used it in this conversation. May not be familiar to everyone. Can we define what you mean when you say like, oh, we're going to define a few slots for this?

• 40:37Speaker 02

  Oh, yeah, I use it specifically in memory. So yeah, when I generally say slots, I'm talking about basically our top level directories, right? So skills, tools, sandbox channels. So memory is going to be, if I was talking to my coding agent, it would call it an authored slot. I would refer to it. So it will be a top level directory. And then inside of memory, we're basically giving you, it's effectively kind of like memory, like drawers almost or shelves that have different drawers inside of them. And so one drawer might be global, one drawer might be user based, one drawer might be channel plus user, like maybe you want memory just in specific Slack channels. So it'll give you the control over how you want to scope those memories. And this lets Eve do interesting things where we can control.

• 41:20Speaker 02

  So the classic example is you almost never want a tool call that like takes in a user ID, for example, right? Like you don't want the agent to guess or think it knows the user ID. And so what these memory slots do is they're lowering tools that are bound to those things. So it cannot, it cannot guess the wrong slot, right?

• 41:38Speaker 01

  Yeah, yeah. You want all of those permission and other related things to be deterministic under the layer of what the agent does. It just, it's just ask for a thing and you say, oh, that is scoped to this user. We're going to include that in the actual functional call. Yeah.

• 41:51Speaker 02

  Exactly. Yeah. And so we went back and forth on whether we wanted memory to be first class or not. And the more, the more you look at cases like that and you realize that you could start grabbing memories from the wrong people and this and that, like when we see problems like that, it kind of hints to us that yes, this does belong as like a first class thing because we can provide the right guardrails and the nice APIs around them and things like that. So.

• 42:13Speaker 01

  Quick detail question on memory. How do you expose that to the agent? Is it through a tool call? Is it embedded in the context all the time? Like, what does that look like?

• 42:23Speaker 02

  Yeah, so we took a stance here where we have this concept called provider and the provider can give whatever tools it wants. And so Eve actually makes, we have no opinion on the tools that should exist when it comes to memory. And so, so the model is actually, we deterministically invoke three functions, recall, save, and this tools function. And the provider, it's entirely up to it on what it wants to do there. And so the provider that will ship in the framework is a file provider. And this one is very, very simple. Recall, it returns the file in full as context. Save, it completely ignores. So there's no implementation of save.

• 43:06Speaker 02

  And the tools it provides is basically add memory and forget memory. And so the agent can add a single memory, in which case we will re-save the file to blob storage and then forget memory, same thing. We just, you know, we remove the memory and re-save it. And so that's the file memory implementation.

• 43:22Speaker 01

  And so just to make sure I understand then, this is a deterministic thing. It's going to be called and the result would be whatever gets returned by recall is embedded in the context of the agent from the start and it has access to these tools. So in this version, the contents of the file are going to be in every agentic prompt and it has access to save or forget particular memories within it.

• 43:45Speaker 02

  Exactly. That's like the most basic version. This is very similar, I think, to like Hermes has like a user dot M D file, I believe. And so this is kind of our version of that. But the APIs we're providing let you go way beyond this. And so we really wanted to support something like super memory where all of the recall and saving is done on their service, right? So Eve has no opinions on those things. And so the super memory provider actually looks quite different where they might provide no tools, right? They don't want the model calling save memory or forget memory, whatever. They don't want to do that because what they want to do is when, you know, we'll call save at deterministic times, they actually take that message history and they will go run some agentic process and figure out what to save. And so it doesn't want to expose tools, right?

• 44:30Speaker 02

  Because they're doing that in their own service. And then their recall can do something much more interesting than like the file case. It can actually look at the inbound message and say, let me recall agendically based on this inbound message. And so it's a completely different recall.

• 44:43Speaker 01

  Interesting. So recall is called in every turn of the loop. So you could actually change what your sort of embedded prompt looks like based on, nice. Okay.

• 44:52Speaker 02

  Exactly. Yeah. And so we call recall, there's a few phases that we call it in. Obviously, turn start is the main one, but we also call it after compaction, for example, because we can't guarantee that compaction keeps, you know, your memories around. So we'll call recall again after compaction and someone like super memory, they might actually look at the compacted prompt and say, oh, no, we're good. It has what we need or no, we have to reinject. And so they can do like really sophisticated things and Eve doesn't, you know, Eve doesn't care.

• 45:21Speaker 01

  In that world, sorry, I'm going way down in the guts here, but are these each turn recall, is that being injected as a new message at the end of the stream? Are you actually editing the system prompt?

• 45:30Speaker 02

  So we give you the choice. The default is append. So we don't, yeah, we don't want to bust the prompt cache if we don't have to. So the, you know, we use like the role user for memories, but you can override this and you could do a role system. And so for like the file memory provider, maybe role system actually does make sense. But the downside of that is every add or forget does bust the system prompt. So.

• 45:53Speaker 01

  Yeah, that makes sense. Awesome. That's super fun. Thank you for geeking out with me on that.

• 45:58Speaker 02

  Yeah, it's all open in PR. So hopefully it launches soon. We'll see.

• 46:03Speaker 01

  Yeah, no, I actually, I was going and looking, you've got a lot of PRs open. So actually, that's, that's kind of an interesting question. So this is a Vercel project. You all are open sourcing it, it looks like. What's the contribution model? What's the sort of ongoing management ownership model look like?

• 46:20Speaker 02

  Yeah, so we have a bit of a funny problem right now where we want to accept outside contributions, but one of the problems we actually have is because our test suite is hitting models and is actually deploying projects to Vercel. And so it's using a bunch of secrets, basically. So we actually have this problem where we can't easily test outside contribution PRs. So we actually have to carefully pull them into a sandbox, make sure that they're not doing anything malicious. And then we have to basically recommit them under our accounts with a co-author and then we can actually run the suite. So yeah, this is definitely a bummer and we're trying to figure out a better process here. It hasn't been the easiest thing. But yes, we want the outside contributions and, you know, we've been trying to take in

• 47:05Speaker 02

  as many as we can. We obviously have Eve agents internally that are actually like analyzing these PRs daily, you know, to look for the ones that we should be pulling in. So, but yeah, the more contributions, the better. And the Eve team was really, it was really two people full time from March until about maybe like mid, mid July, I want to say. And we just brought on four more people that have been doing a great job. So yeah, it's about six people full time on it now, engineering wise.

• 47:32Speaker 01

  Well, we're getting closer to the end of our time. I have one more question I'm going to put to each of you. And then also anything that we haven't talked about that you think would be important, we can, we can bring that up. But the question for each of you in turn, and maybe we can start with you, Andrew, and then Shaw, you can close this out, is like, what's the vision for where this is going? What do you see coming down the road, the next three, six? I don't know if we can project out nine months in today's AI age, but what's coming down? What are you excited about?

• 48:00Speaker 02

  Yeah, for me, I mean, one of the very early principles behind Eve was we're going to bet on the models getting smarter. And the way we designed the framework was really meant to take advantage of this. I think a lot of other agent frameworks that we had seen, they scaled vertically where they just appeared to get more and more code. And eventually it kind of doesn't look like an agent anymore. You have a lot of determinism, like it's almost like these workflow style APIs. And what we really wanted with Eve was to go horizontal and basically just say, we are going to bet on the models making the right decisions. This is also generally like where the folder structure and things like that came from, where we just want to provide more things for the model and let it make the right decision. And so Eve is going to stick to that principle.

• 48:45Speaker 02

  And yeah, I mean, it's clear that the model intelligence is not slowing down anytime soon. And we think Eve is just going to take advantage of that in a great way. So.

• 48:55Speaker 01

  Awesome. What about you, Shaw?

• 48:56Speaker 03

  Yeah, for me, I think the way we're thinking about it as a company building is agent building. So the idea is that Eve is more fundamental than your certificate of incorporation. So your agent predates your websites, your domain name, and even incorporating the company. So you start with the agent and the agent incrementally builds the software factory for you, including your complete software development lifecycle. And your job is then to actually fine tune that factory. And we believe that Eve is positioned to basically become that brain of your company moving forward, that that's kind of where we're heading.

• 49:36Speaker 01

  Well, that makes me think immediately, what's the hosting story outside of Vercel? I mean, obviously it's great that you guys can manage all of this and all this sort of thing, but if I'm building my company on top of this, I want to know that I can move it where I need to move it whenever that might happen. So how does one host an Eve based agent framework or a set of agents?

• 49:55Speaker 02

  Yeah, so I mean, this is actually very important to us. And we do have customers internally that are running Eve on Kubernetes, on their own hardware even. And part of this is taking advantage of what is called a world inside of workflow. And this is part of our end to end test suite where we test different worlds, like a Postgres world, the local world and things like that. And so, you know, as long as you can define a world for your infrastructure and many of them exist, like there's, there's worlds for tons of other providers out there, that is really all you need to host Eve yourself. And so, yeah, there's, there's a lot more work that we're going to be doing here to make that experience much better. And the key with things like memory and self-evolution and these other types of things is like there has to be adapter contracts, right?

• 50:40Speaker 02

  Like we can only do so much in the framework without bringing in external things, right? Dynamic scheduling is, you know, for example, like that will be first class on Vercel, but you will need to bring in something. But the key is just making sure that those APIs and those adapters are thought of, you know, early on in the process and not some afterthought, you know, after it already works on Vercel.

• 51:00Speaker 01

  I love it. That's a pretty good cut, unless you guys have something else you want to talk about.

• 51:05Speaker 02

  No, nothing for me. That was great.

• 51:06Speaker 03

  Yeah, that was fun.