Build an AI code review bot in 30 minutes with Vercel Eve

AI writes most of my code now, and that created a new problem: a PR queue I couldn’t keep up with. In this episode, I walk through how I built Merge Mommy, a Vercel Eve agent that reads every PR after checks pass, scores it across six risk dimensions, auto-approves the low-risk ones, and pings me in Slack for anything that needs a human. I built the whole thing in one Codex session, it’s SOC 2 compatible, and it’s already cleared my backlog. What you’ll learn: Why AI-generated PRs create a review bottleneck and why the answer isn’t reviewing all of themHow Intercom 5x’d PR approval speed and reduced revert rates by putting AI in the review loopWhy Vercel Eve is the simplest framework I’ve found for deploying AI agents in Slack and GitHubHow I built a full PR review agent in Codex with one prompt and a few steering turnsThe six components I use to score PR risk (blast radius, reversibility, data security, ops impact, verification gap, and change surface)How I used Chrome browser use to handle Slack bot and GitHub app configuration so I never had to click through setup screens manuallyWhy auto-approved PRs can be SOC 2 compliant as long as the process is auditable, queryable, and in your risk policyHow to set up Slack escalation so low-risk PRs become a two-click merge with no manual review— Brought to you by: WorkOS https://workos.com/?utm_source=lennys_howiai&utm_medium=podcast&utm_campaign=q22025—Make your app Enterprise Ready today — In this episode, we cover: (00:00) The PR review backlog problem nobody’s talking about (02:35) Why you don’t have to review every AI-generated PR (05:14) How Intercom built AI-approved PRs (and proved they’re safer) (06:10) How the Eve framework works (directory, skills, channels, connectors) (09:16) The Codex prompt I used to build the entire bot (11:36) What the agent actually does: read, score, approve, or escalate (13:07) Setting up your Eve agent (15:47) The six-component risk scoring model (17:23) Merge Mommy in action: three live PR examples (21:10) Recap and how to build your own version — Tools referenced: • Vercel Eve: https://vercel.com/eve • Vercel AI SDK: https://sdk.vercel.ai/ • Vercel Chat SDK: https://chat-sdk.dev/ • Codex (OpenAI): https://openai.com/codex — Other references: • AI is approving our pull requests: Here’s how we made it safe:https://www.intercom.com/blog/ai-is-approving-our-pull-requests-heres-how-we-made-it-safe/ — Where to find Claire Vo: ChatPRD: https://www.chatprd.ai/ Website: https://clairevo.com/ LinkedIn: https://www.linkedin.com/in/clairevo/ X: https://x.com/clairevo — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

• 0:00Speaker 00

  Everybody is drowning in AI-created PRs. When anyone can code, anyone can cook, and anyone can vibe anything to GitHub, that means that a lot of us are sitting on a big queue of pull requests that then we have to review. If you are an engineering leader or a software engineer in a company that has embraced AI in any way, shape, or form, you feel this pain, and I get this question all the time. We've figured out how to code with AI. Now, what do we do with all these PRs? Well, people smarter than us have figured this out. And you know what the answer is? You don't have to review all your PRs. Yep, you heard it here first.

• 0:42Speaker 00

  You do not actually have to review one-on-one by a human all those PRs. So in today's episode, I'm going to show you how to build your own PR review, risk scoring, and automatic approval bot. So you can take all the low-risk PRs off your plate, get those to production, and focus on the ones that really require your human expertise. This episode is inspired by a couple folks, especially our past How I AI guest, Brian from Intercom, who has led the way in auto-scoring and auto-approval PRs. But I'm going to show you exactly how it gets done, and we're going to get it done very quickly in this 30-minute How I AI mini-episode. Let's get to it.

• 1:25Speaker 00

  This episode is brought to you by WorkOS. AI has already changed how we work. Tools are helping teams write better code, analyze customer data, and even handle support tickets automatically. But there's a catch. These tools only work well when they have deep access to company systems. Your copilot needs to see your entire code base. Your chatbot needs to search across internal docs. And for enterprise buyers, that raises serious security concerns. That's why these apps face intense IT scrutiny from day one. To pass, they need secure authentication, access controls, audit logs, the whole suite of enterprise features. Building all that from scratch? It's a massive lift. That's where WorkOS comes in.

• 2:10Speaker 00

  WorkOS gives you drop-in APIs for enterprise features so your app can become enterprise-ready and scale up market faster. Think of it like Stripe for enterprise features. OpenAI, Perplexity, and Cursor are already using WorkOS to move faster and meet enterprise demands. Join them and hundreds of other industry leaders at workos.com. Start building today. Okay, I know what you all are thinking. You're thinking, Claire, you are crazy. Of course, I have to review every PR that comes out of AI. That's just how it's done. We review all of our human PRs, so I have to review all of our AI PRs. And I am telling you, in 2026,

• 2:54Speaker 00

  the answer is you can safely not review PRs by putting AI in the loop. And I want to call attention to two blog posts that have really inspired me in this process. One is from Intercom about how they made AI-approved PRs safe. If you don't know, Intercom has 2X, 3X their PR throughput. They are shipping a lot of code. That means they had a lot of code to review. And so they built a PR review agent that scores their PRs and then auto-approved them. And what I love about what Intercom has done here is they don't just worry about speed and velocity and volume. Yes, they've multiplied the PRs across their R and D organization.

• 3:39Speaker 00

  Yes, they want to get those PR cycle time down and those reviews out quickly. But what they really think and what they've proven with their adoption of AI is that AI-approved PRs and AI-written code in general can be even safer and even higher quality than what you're doing with just a human in the loop. And so they have used AI as an advantage when it comes to safety and quality, as opposed to a disadvantage for safety and quality. And to set the vision of what you can expect here, I love that they shared some metrics. One is that AI-approved PRs are approved faster, actually, five times faster than their human PRs.

• 4:23Speaker 00

  So one, you are getting some speed. Two, they are seeing higher quality. So the revert rate, the amount of code that has to be fixed after it hits production, is much lower when they write that code with AI. And then they figured out how to label and trace and audit all those changes and approvals so that they are in compliance with all of their required compliance frameworks, whether those are SOC 2, HIPAA, et cetera. So I often get this objection from folks. They say, well, I'm in a SOC 2 environment. I couldn't possibly auto-approve PRs. But in fact, there are frameworks, as long as it's in your risk policies, as long as it's in your code review policies, as long as it's auditable, queryable, and defensible,

• 5:08Speaker 00

  you can do this within your compliance framework, although work with your security and compliance teams to figure out how this can work for you. So I really love just this whole concept. I also took some inspiration, who took inspiration from Intercom, on this Rewind bot, which is a little bit more of the technical way that they review PRs. And so this blog post about Diff Vader, which I love, shows a couple components of how they've scored risk, whether it's blast rating is, how correct it is, if all the actions have run, et cetera. And so between these two blog posts, I had a really good starting point for building an agent. I knew that I wanted to do this for chat PRD. I knew we had a lot of low-risk PRs

• 5:52Speaker 00

  that were just sitting in the queue because my colleague and I were not getting to review them. It was kind of boring, especially a lot of like dev and authored PRs. And so this was going to be a really high-impact thing that I knew I could build and also let me work with a new framework that I'm excited to demo for you all here on How I AI. So we're going to go to this Codex Vibe coded deck, and we're going to walk through how I built a code-review risk-scoring bot with Vercel's Eve. So if you have not heard about Eve, I'm a little bit obsessed with Eve. Why do I like Eve? Well, Eve has become the simplest way

• 6:34Speaker 00

  for me to deploy AI agents in the enterprise, in Slack and GitHub, without the torture and pain of something like Hermes Agent or my beloved OpenClaw. So when I'm working and helping other companies sort of like think about how to deploy agents, if they're using Vercel, I tell them to use Eve. Why do I tell them to use Eve? Well, what's awesome about Eve is a lot of it is just a directory. An Eve agent is very similar to OpenClaw, just a directory of instructions, skills, code. And it can talk to you in channels that are out of the box in Vercel.

• 7:16Speaker 00

  So again, it's like very easy to set up the agent. It's very easy to give it skills, which we all use. It's very easy to give it tools. It can even have a sandbox in which it can execute things, including your code if you're doing a code review. And then my favorite part is managing talking to the agent in specific channels and connecting it to enterprise data is super easy. So because Vercel has shipped Vercel connectors, which are like managed connections inside your Vercel account, you can just literally go through like a little wizard to hook up Slack to your Eve agent, a little wizard to hook up GitHub to go through your agent, and it's all set up, the refresh tokens, all that stuff.

• 8:00Speaker 00

  Everything that's painful about creating an agent is just taken care of. And I sound totally like a Vercel shill. I've just tried a bunch of stuff, and this has been the easiest framework to get up and running. It uses the chat SDK behind the scenes. And if you are creating any sort of chatbot in Slack, Teams, WhatsApp, whatever, I mean, bless, I was asking for this chat SDK for a long time. It got built. This is an open-source framework that I think is so useful, whether or not you use Eve or not to stand up a chatbot. It just like... Takes away all the complexities of managing like multi-channel agents.

• 8:45Speaker 00

  It handles all like the Slack configuration. It gives you a manifest for Slack. Like, I love it. The chat P R D Slack bot in Teams Bot actually runs on the chat S D K. But it's even better when it's bundled into this e framework. And so, and I highly recommend you try e framework. Again, like, they didn't pay me to say this. I just really like it and I found it super easy to use. And now every agent that I'm creating that I want to interact with in Slack for an internal use case, if I don't already have an open claw, I'm porting over to Eve. So how did I actually build this? What does the agent do? Well, I built it in Codex. And so I'm going to show you the amazing prompt where I started this P R review bot,

• 9:28Speaker 00

  which just says, I want to make an internal GitHub bot slash app that reviews P R's after all C checks, that's checks, are green and grades low, medium, high risk and approves the low risk. Oh, my God, you guys, I can't type. All bulletproof. I've raised a low risk P R's automatically. That's all I did. And then I interrupted it. I steered it and said, we could design it as a Versel Eve agent if you want. And it said, sure, that seems like a good, good idea. And it was off to the races. And so instead of going through kind of like the chat on Codex on how I built this, this took a couple of turns, but really not that many. I would say a couple of things that were like magical as I ran Codex is not that it

• 10:12Speaker 00

  did the code for the Eve agent and the repo. That was fine. That was interesting. And I'll tell you a little bit about how the code actually works in a sec. But what was actually awesome is if you've ever set up a Slack bot or a GitHub app, you have to like click through a bunch of configuration screens. You have to pick permissions. You have to do all this stuff. And so what I did is my favorite hack is I had it use Chrome browser use and I let it navigate Slack bot setup. I let it navigate GitHub setup. And I just like clicked buttons into F A D and just double checked that it was right. And so I said, you know, is it configured yet? It said, no, it needs to configure all this stuff and the permissions on the GitHub bot.

• 10:55Speaker 00

  I was like, great, you're logged into Chrome. And then I was like, I can press save. It was having a hard time press save. And so if you're just sort of like meta tip, if you ever have to do like complex SaaS configuration, set up an app, you're like, ugh, I'm fine writing code, but I don't want to do the configuration in somebody's third-party service. Using Chrome browser use is such a useful hack to get that stuff done. So again, I just built this in Codex. I actually didn't give it any instructions on scoring. I didn't give it any instructions on configuration. I didn't give it any instructions on risk. I just said kind of like one-ish shot it. Few shot this into production.

• 11:37Speaker 00

  And so I want to talk a little bit about what the agent does and how it works. So it's pretty simple. If you were building a P R review agent, this is what I would suggest you do, which is it reads the P R. It looks at the exact diff and it scores the risk and it publishes the evidence to the risk. If it's low risk, if it's assessed that it's a low-risk P R, it submits an approval on the P R. If it needs a human, it escalates to human. And if there's a blocking requirement, then it stops and it will require a request change. So it's like very similar to how a human would do code review.

• 12:22Speaker 00

  How does it work from a technical flow perspective? So the GitHub, I have a GitHub app. It keys off an event after all of the P R changes are done. Versel's GitHub integration picks that up in the GitHub Versel channel and sends it a little bit of information. Versel spins up its sandbox. It checks out the repo. It runs it and looks at the diff. It runs a couple of skills and tools to assess the risk and quality of the P R. And then on the end, it outputs like a thumbs up check, like it needs changes and a comment. And then one fun thing that it does is after it does that review, it pings my colleague

• 13:03Speaker 00

  and I in Slack to say this P R is ready for review or it needs some help. So if you were to set up your Eve agent to be similar to mine, what would it have? Well, it would have a couple of things. It needs to have the GitHub channel. It needs to have instructions, which I can show you a little bit of the instructions. It needs a skill to review the P R. It needs two tools, read like all the files and information around the P R and then how to actually put a risk decision in. And then it has, you know, a couple of things like a Slack notifier. So this is not that many files. It's pretty simple. I will show you our instructions just to make this like very clear how easy it is to set this up.

• 13:48Speaker 00

  It's a chat P R D engineering agent. It reviews P R's. It calls the risk context. It scores it. And then it has a couple of instructions. This is it. There's no scrolling. It is like four, four or five paragraphs, a couple of bullet points, and it's ready to go. You do not need to over-engineer this. And it works really, really well. So this is again, why I like this Eve agent framework, because you're just working in natural text or in TypeScript and it's super easy to implement. Setting up the GitHub app, you can ignore this. I mean, I guess like point your agents to this instructions. I did not do any of this. Again, Chrome browser use did this on our behalf. And so it just has to have access to the pull requests, the contents of your files,

• 14:33Speaker 00

  C I checks and action checks, and then some metadata. And I just had Vercel set all that up. It has a couple of rules on when it's triggered. So you don't want it triggered on every P R before checks are done. So I kind of had to like reduce some noise in the P R rules. And then there's just two basically steps to this. It reads the P R. It's like a skill to read the P R. I will show it to you. It's not that thrilling, but again, just showing you how simple it is. It reads and reviews the P R. It has a couple repo-specific risk kind of categories. So like docs are low risk.

• 15:16Speaker 00

  Feature logic is medium risk. And then like authentication billing is high risk. Diff size does not set risk. And then it gives some like instructions on writing. Again, this is it. It's like not that much. It's maybe one page of text. I did not write any of this. I refined it. And so what I'm trying to reinforce to you is like writing these agents is not that hard. It is literally just writing instructions and skills. That is what it takes to create an agent like this. And that's why I like this e framework, because you can do a lot of it in Markdown. And then once it reads the code, it scores the risk. And so it looks at six things. How big is the change surface and blast radius? Is it easily reversible, right? Like a huge data migrations may be harder to reverse.

• 16:01Speaker 00

  Does it touch data security? And does it cover data security? Does it change anything about our operations? And verification gap is like, are our tests complete? Did C I complete? Can we actually test a couple of ways that this is true? And then it has sort of like a script that it runs to calculate a score. And then anything under 24 points, I, again, I did not like choose these thresholds. Anything below 24 points is low risk. Medium risk is 25 to 64 and then 65 and above is high risk. Medium and high risk P R's require human approval. So it's pretty simple. There's some logic in here to only review the latest changes, which I like.

• 16:43Speaker 00

  And then the final piece is it escalates us to us in Slack for final action. So we have repo rules, probably very similar to all of you, that requires a review in our GitHub to like match all of our R SOC 2 compliance requirements. It just makes auditing and management of our compliance really easy. I went through a couple of ways where the bot could like, quote, quote, act like a human to do the review. It just didn't seem worth it. So what it does is escalate to Slack and then it says, this one's been automatically reviewed. You can punch the approve button and merge and you're good, good to go. And so we sort of like built in an operational process here as well.

• 17:24Speaker 00

  But let's see what this actually looks like in action. So I have three P R's for you, just to show you kind of how this works. Now, the P R bot's called Merge Mommy. She is Merge Mommy, the P R review bot. You know, when you make internal bots, we don't have diff vader. We have Merge Mommy because we're fun at chat P R D. And this was a docs-only update automatically done by Devin. And typically I just like smash the approval button on these, merge them. But I found that like we were just getting queued up of a lot of docs changes that didn't really need a bunch of review. And so what happens on, once the P R is ready to go, Merge Mommy runs.

• 18:09Speaker 00

  It gives a risk score, six out of 10. It's very low risk because it only changes docs, but it's not auto-approved because it has merge conflicts. So that's one of the scoring issues it has to check. And then there are some detailed notes here about why it's blocked in the approval blocker is the pull request has merge conflicts. A cleaner example here is this doc, which I already merged, which again, went through and said it's seven out of 10, low risk. It's auto-approved. And you can see up here, Merge Mommy gave a little check that said it's approved. Now, what we do with these approved P R's, as I said, is we send them to a Slack channel.

• 18:53Speaker 00

  And so the Slack channel says, hey, either of you can take this, approve it. The risk is low. The checks are green. You just need to smash that approve button and merge it. So it's just like a really nice automated flow. Now, why do we have to have a human do it? As you can see, this check mark is gray. A bot cannot actually meet our repository rules for required approval. And so we can either skip those rules, hack around it. We just decided this check would be a signal that our humans can approve it without really looking at it and then merge it as we want. And then let's show one where Merge Mommy did not approve it. This is a deprecation P R.

• 19:35Speaker 00

  So we had chat V1 and chat P R D and we're moving to chat V2. We kind of like re-factored a bunch of code, but we left the old stuff behind a feature flag. And it was like ready to delete. So there's like 35 changes, big red diff that we wanted to delete. And you can see here. Merge Mommy, I'm sorry, I can't take myself seriously when I say it, but I'm gonna say it anyways. Merge Mommy gave it a 45 out of 100, a medium risk, and it was not approved by a human because both it had some code issues as well as the policy said it changed server API behavior. It was a large change service. And so it's medium not low and it can't approve the PR.

• 20:20Speaker 00

  Now, we can run this against every PR that goes through our process. And again, we're going to just keep getting these notifications in Slack that just let us know, hey, this can be approved or this requires human review. Again, I like to say this thing where we can put AI to work for us or we can have AI put us to work. And this is an example where you do both. I've put AI to work, this lovely Eve agent, I put AI to work to review and score the PR and tell me if there's any risk and like be detailed eyes on it. And then I had AI put me to work by escalating into Slack for the final action

• 21:04Speaker 00

  and making it like a two-click process to get this thing to production. Now, I just have to say, this is something that I was really intimidated to build. I thought this was going to take me like days and days and days. I was like, I don't want to set up the GitHub app. I actually had tried this before Codex browser use was really good and before the Eve agents had come out. And it was just like not easy to do. But now that we have these open source frameworks, now that we have these agent building frameworks, now that kind of like everybody's agreed that we need Slack bots that do AI, a lot of these problems have been solved. And then Codex browser use was just like the cherry on top

• 21:48Speaker 00

  that made building this agent super, super, super simple. So to just recap for you all in this lightning fast mini episode about how to build a PR risk, score and review bot, use Vercel's Eve agents, give it instructions and a skill to read a PR and calculate a score based on a couple components, hook up that Vercel agent to GitHub and to Slack, let it read from GitHub and give you a little gray check for approval and then have it Slack you in Slack for that final human review and merge. And your cycle time will be super, super fast

• 22:33Speaker 00

  and your PR velocity will go through the roof. Now, I want to hear from you all. Is this crazy? Can you pull this off at your company? And what would you put into your risk scoring that I haven't thought of? The final bonus piece of this, which I didn't show, but a lot of great folks do, especially the folks at Intercom do as part of this process, is they run evals on this internal agent. So every time this review is run, it gets logged into, I think it's like an internal eval platform. And an engineer looks at it and says, did the agent get this right? Did the agent get this wrong? Are we happy with the scoring mechanism?

• 23:14Speaker 00

  So very similar to how you'd use evals to improve your customer-facing AI products, you're going to want to use evals to improve your internally facing AI bots, especially ones that touch really critical things like code. That has been my mini episode on how to build a PR approval and review bot. Here at How AI, I cannot wait to hear if you think this is crazy or this is the way of the future. Thank you so much for joining and we'll see you soon.

• 23:45Speaker 00

  Thanks so much for watching. If you enjoyed this show, please like and subscribe here on YouTube or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.