I spent $20,000 on Devin in a month. Here’s what I learned | Ryan Carson (solo founder)

Ryan Carson is a five-time founder and the current solo founder of Untangle, a B2B SaaS platform for family law firms. Before Untangle, he co-founded Treehouse, an online coding education platform, and has spent the better part of two decades building and leading tech companies. He’s active on X, where he shares his solo founder journey in real time, including what he actually spends on AI tools each month. What you’ll learn: Why Ryan manages 15 concurrent Devin agents with a folder system and a piece of paper, not a dashboardThe Watchdog playbook: what he built to replace a customer success team across every law firm accountHow his LAN PR skill closes the loop on 40 daily PRs without a QA team reviewing a single oneWhy he moved off local agents almost entirely, and the one situation where he still reaches for CodexThe design workflow we’re both using: Claude Design into a Markdown spec, then Codex to build the real thingWhat he found when he got away from his computer and met a real customer, and why it changed his entire product directionHow he’s hiring his first engineer without a single phone screen or interviewWhy we both think more AI output is actually the wrong goal, and what to optimize for instead— Brought to you by: WorkOS https://workos.com/?utm_source=lennys_howiai&utm_medium=podcast&utm_campaign=q22025—Make your app enterprise-ready, with SSO, SCIM, RBAC, and more Jira AI SDLC https://jira.dev—Get your tokens’ worth with Jira — In this episode, we cover: (00:00) Introduction to Ryan Carson (02:55) Ryan’s update: Untangle, the divorce PMF pivot, and B2B growth (07:35) Ryan’s current Devin stack: folders, P0 threads, and the paper list (16:29) Watchdog playbook: account monitoring across every firm (18:09) Managing agent decision fatigue: Ryan’s method vs. Claire’s (20:32) Producing more output does not make a better product (22:47) Using cloud agents for ops beyond just code (25:14) When to use Codex vs. Devin vs. Claude Code (27:15) Merge Mommy recap (28:15) LAN PR skill: review loops, video walkthrough, and auto-merge (30:20) Slack vs. Devin threads for async team communication (35:58) Claude Design plus Codex for building a technical design system (39:00) EA tools: Polly the OpenClaw vs. Claude Code on a Mac Mini (42:09) Quick recap and final thoughts — Tools referenced: • Devin (Cognition): https://www.cognition.ai/ • Codex (OpenAI): https://openai.com/codex • Claude Code/Claude Design (Anthropic): https://www.anthropic.com/claude • OpenClaw (Claude-based desktop client): https://openclaw.ai https://openclaw.ai/ • Cursor: https://www.cursor.com/ • BugBot (Devin’s built-in PR review): https://cursor.com/bugbot • Sentry (error monitoring referenced in Watchdog): https://sentry.io/ • Ugmonk (analog to-do system): https://ugmonk.com/ — Other references: • Devin playbooks/skills documentation: https://docs.cognition.ai/ • Jack Dorsey/Buzz (async-first communication referenced): https://buzz.new/ • Merge Mommy (Claire’s Eve agent for PR risk scoring, deployed on Vercel): https://www.lennysnewsletter.com/p/build-an-ai-code-review-bot-in-30 — Where to find Ryan Carson: X: https://x.com/ryancarson Untangle: https://untangle.us https://untangle.us/ — Where to find Claire Vo: ChatPRD: https://www.chatprd.ai/ Website: https://clairevo.com/ LinkedIn: https://www.linkedin.com/in/clairevo/ X: https://x.com/clairevo — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

• 0:00Speaker 00

  If you're out there listening and you are doing engineering, you're doing your work locally, you really need to open your eyes. I think the future is pretty much 100% cloud agents. I went all in, so much so, I spent $5,000 on Devin, and then it went up to $20,000 one month. You know, I think one of my big messages for today is that all of us have to up level our ability to manage agents. Like, that is our job. How do you effectively manage 10, 20, 100, 500 employees? This is the whole reason that organizations were built as pyramids. Like, humans can't scale and have 1,000 directs.

• 0:36Speaker 01

  What is in your stack right now? And how have you structured stuff to make you successful and build things effectively?

• 0:43Speaker 00

  So what I'm doing is figuring out how to manage, you know, throngs of agents. And in Devin, what you're seeing is folders. And you'll notice P0 is the big stuff. These are the main things I need to move forward as the business, new features, P1, P2. So our priority zero, we've got bugs, let them work, usually give them a goal, and it's just such a crazy different world we live in now.

• 1:08Speaker 01

  Welcome back to How I AI. I have a super special guest, Ryan Carson, one of our earliest How I AI episodes, who is here to show us what has changed in the last 12 months in the age of coding agents. We're going to wax poetic about our favorite little AI intern, Devin, talk about all the ways you're under utilizing your coding agents, and give maybe the most anti-AI episode of How I AI where we talk about where AI can't solve your problems, how you need to get out in front and actually talk to people, and how pen and paper still have a place in his stack. Let's get to it. This episode is brought to you by WorkOS. AI has already changed how we work.

• 1:52Speaker 01

  Tools are helping teams write better code, analyze customer data, and even handle support tickets automatically. But there's a catch. These tools only work well when they have deep access to company systems. Your copilot needs to see your entire codebase. Your chatbot needs to search across internal docs. And for enterprise buyers, that raises serious security concerns. That's why these apps face intense IT scrutiny from day one. To pass, they need secure authentication, access controls, audit logs, the whole suite of enterprise features. Building all that from scratch? It's a massive lift. That's where WorkOS comes in. WorkOS gives you drop-in APIs for enterprise features, so your app can become enterprise-ready

• 2:37Speaker 01

  and scale up market faster. Think of it like Stripe for enterprise features. OpenAI, Perplexity, and Cursor are already using WorkOS to move faster and meet enterprise demands. Join them and hundreds of other industry leaders at workos.com. Start building today. Ryan Carson, number one episode of How I AI from our early, early days is back again. I am so happy to have you here, my friend.

• 3:10Speaker 00

  This is, it's so fun to reconnect. And the reason why the show is good is because of you. And just everybody listening, I'm not just kissing ass here. Everybody that I meet, I say, there's one show, if you're stressed about keeping up on AI, just listen to Claire's show.

• 3:25Speaker 01

  Oh, thank you. Well, I'm stressed keeping up with AI, so I will take all that stress on behalf of the audience and pass on what I know. What I also am so excited about this conversation is, you know, we did, we did our first episode a year ago, or even a little more than a year ago. And like, it's so different now. We were such babies.

• 3:45Speaker 00

  I know. It's, it's, it's kind of funny, like, what we chat about now. And it'll be funny what we chat about now in a year, right?

• 3:52Speaker 01

  Yes, totally. And for those who have not gone in the archives, we did a really, really popular episode about how to manage your agents with like, it was like pre-skills even. It was like three-step marked, like you invented skills from first principles. It was like three-step markdown file. We were in an IDE, which I haven't opened an IDE in like months. We were in an IDE and we were really, you were very prescriptive and directive of your agents.

• 4:24Speaker 00

  Very hand-holdy.

• 4:25Speaker 01

  Very hand-holdy. And now, you know, I don't know about you, I'll speak for myself. What the agents are doing is none of my business.

• 4:33Speaker 00

  No, let them, let them work, you know, usually give them a goal. And yeah, it's just such a crazy different world we live in now. It's so exciting.

• 4:43Speaker 01

  Well, catch us up, because there's also been a lot of changes on your side in the last year. So what are you up to? And how has it, you know, impacted how you think about and work with AI these days?

• 4:53Speaker 00

  Yeah. So I basically had this idea to make divorce better. So I have three amazing sisters, we're like best friends. I have a sibling tattoo. It's hard to show. It's right there. You know, we talk every week. And then two of them started to go through a pretty gnarly divorce. And my parents got divorced too. So, you know, I was like, divorce, hmm. Like, no one seems to be talking about divorce in AI. What, why? Like, and so I just started working on that. And was, of course, empowered by Thor's hammer, you know, which is large language models. And, and I started to build it. And this is when we chatted last, you know, I was in Cursor, you know, like you said, really hand-holdy on the agent. And since then, thankfully, it's just blown up.

• 5:37Speaker 00

  I basically figured out product-market fit. And everyone who's listening, there's probably a ton of startup founders out there. You know, you're trying to find product-market fit. You're pivoting fast. You're, and I did what all founders do, right? You know, launched a version of Untangle, which is the product, thought it was for consumers, went out, nobody, turns out, nobody wants to use AI for divorce. They, they want lawyers, people. Just, I'm just gonna say it. And then I basically thought, okay, not PMF. I'm old enough to know, like, not hang on to things, just pivot. And I was like, okay, I'm gonna get a meeting with a lawyer and just see what they think of this thing. And she met with this amazing lawyer named Renee Bauer. And I was like, do you love this?

• 6:22Speaker 00

  Do you hate it? Is this like an enemy? Is it a friend? And she said, I love it. You know, I, there's a paralegal shortage. This discovery thing we do in divorce is a nightmare. I love it, and I want to pay for it. And I was like, oh, you, how much would you pay? And she said, a great number. And now we're, we're B2B, and we sell to family law firms. And it's just exploding. So I've been trying to hang on to this solo founder, I'm gonna do it all myself thing, and I'm completely overwhelmed.

• 6:51Speaker 01

  You and me both. And for folks that can't wait for the punchline, the punchline is, Ryan Carson and Clair Vo, famous, famous solo founder, I can do it all myself, I need no friends other than AI, did our pre-show conversation talking about our regretful and imminent hiring of a bunch of humans.

• 7:13Speaker 00

  Of humans. I'm gonna have to hire humans. And if any of those humans I'm talking to are listening, I love you, and I'm excited about working with you now. But, but the AI, yeah, it's, you get spoiled hiring AIs. I mean, you know, you give them 45 pieces of feedback an hour. You know, you're, anyway, so yeah, gonna hire some humans and grow the Untangle team. Very excited.

• 7:35Speaker 01

  But you haven't yet. And so, I mean, the, the, the good, the good part of the story is you were able to get to rapid iteration, lots of prototypes, product-market fit with you and our mutual intern, Mr. Devin.

• 7:53Speaker 00

  Mr. Devin.

• 7:54Speaker 01

  Mr. Devin. And so, you know, when you were starting out, again, you were like really coding, doing requirements, documents, all of that with AI, but you were very human in the loop. And as I've watched you over the past year, I've seen you slowly turn into AI bro who runs agents overnight. So tell us about, like, what is in your stack right now? What is like very different from what we talked about a year ago? And how have you structured stuff to, to make, make you successful and build things effectively?

• 8:29Speaker 00

  Totally. I like to say I'm in max queue right now, for those of you that are, you know, SpaceX fans. It's like the maximum pressure that I need to hire somebody, I'm doing all the work. And so, yeah, what am I actually doing day to day? So I'll walk you through it. I started off, you know, in a more specific agent, like a Cursor, like an AMP. And, and then I, I saw cloud happening. I'm like, okay, I'm really limited because I have to be at my machine. And there's a lot of things I need to, you know, quick PRs, like, here's a bug, need to sort it out. Or what about this piece of data? Need to, you know, and I was like, why, why isn't cloud?

• 9:12Speaker 00

  Like, I feel like I should be working on the cloud. And I thought, what about Devin? Like Devin, you know, they've been doing cloud for like years now. It was kind of crap in the beginning, right? But maybe it's good now. And I tried it, and I was like, wow. This, this thing's really good. This is not.

• 9:28Speaker 01

  Real good.

• 9:29Speaker 00

  It's really good. And, you know, and I, I thought, okay, I'm gonna try it. And I was just blown away by how mature the cloud agent was. And, and if you're out there listening, and, you know, you are doing engineering, you're doing your work locally, you really need to open your eyes. Like, I, I think the future is pretty much 100%, you know, cloud agents. There's gonna be some local dev, but, so I started looking at that and was really impressed. So now, I went all in, so much so, I spent, at one point, I was like spending $5,000 on Devin. And then it went up to $20,000 one month. And I'm like, whoa, wait a minute. I have a big Twitter account. I'm gonna go talk to the cognition guys and tell them to give me free credits.

• 10:11Speaker 00

  And they did. So, so anyway, I, you know, get 20 grand of credits per month in Devin. Great. But the point is, I live in Devin. I live in the cloud, right? So all of my engineering, almost, and it's Devin's right here on my screen, I've got probably, you know, 10, probably 10, 15 threads, and I bucket them in folders. Because I remember one of your shows, you know, it was either you or your guest was saying, it's sort of like four or five concurrent tasks you can actually keep track of. And I agree with that. So what I use is folders.

• 10:47Speaker 01

  Is there any way to show it?

• 10:50Speaker 00

  I can show it. I'll show you one thing, which is this piece of paper.

• 10:54Speaker 01

  Perfect.

• 10:55Speaker 00

  And.

• 10:56Speaker 01

  This is the AI podcast we're doing where we say we're gonna hire humans, and we're just gonna show you pieces of paper we scribble.

• 11:05Speaker 00

  Okay, the reason, but there's a, there's a, I'm doing this for a reason. Okay. So the reason why I have folders for threads and the reason why I have a piece of paper is because it, the skill is managing throngs of agents. Like that is what I do now. And to, to keep it straight in your head, you, you basically have to bucket these things. And so I have one bucket that I'll, I'll try to show this to you. Let, let's see how, let's see how dangerous this is going to become, everybody. Okay. So everybody, this is Devin. This is my real setup that I'm literally working on today. And, you know, I think one of my big messages for today is that all of us have to up level our ability to manage agents. Like that is our job, right?

• 11:50Speaker 00

  And so how do you do that? How do you effectively manage 10, 20, 100, 500 employees, basically? This is the whole reason that organizations were built as pyramids, right? Like humans can't scale and have a thousand directs unless you're like Jensen Huang, then you can, but. So what I'm doing is figuring out how to manage, you know, throngs of agents. And in Devin, what, what you're seeing is, is folders. And you'll notice we've got bugs, right? I've got P zero, P one, P two. So it's sort of priority zero. What, what are the things that I, I really need to move forward, no matter how distracted I get today? And so I'm going to show you again, my piece of paper. Ta-da.

• 12:32Speaker 00

  Okay. So I, I have two methods for this. Don't laugh, it works. So the way I stay focused is number one, have a P zero folder. But then, you know, I've got a Dell 52 here. I've got eight screens concurrently open, right? And so the way I stay focused is by also having this piece of paper, which has weekly priorities on it.

• 12:55Speaker 00

  Okay. So it's all about weekly. What is important? And I, and I'll, and I'll cast my eye down here at this beautiful UG Monk system I have, which I'm not being paid to say that. It's a fun little to do system. And that just keeps me grounded. Okay. So let's dig into this a little bit. Bugs is like, okay, there's some stuff going on. I want to get it pretty fast. It's not necessarily P zero. You know, let's look at a couple of those, you know? That's the one we're looking at here. Investigate red error, retries, right? So Devin's cranking on that. There's a couple of other bugs in there. P zero is the big stuff. I'm not going to show you that, but that's just like, these are the, these are the, the main things I need to move forward as the business. New features usually.

• 13:39Speaker 00

  P one is like, ah, stuff that's important. Anyway, it goes down. Investors. By the way, tip, oh my God. Like if you're not using an agent to help draft your investor updates and help you keep track of the business, like clearly do that. But here's the key, build a skill to do it. And this is probably obvious to everybody, but another thing I love about Devin is this idea has playbooks and playbooks are, are sort of, there's this weird mix of skills, playbooks, you know, sort of thing going on, but I've built an investor update skill, basically, which does a lot. So this is most of where I do work. And then you'll find like, then that means I do a lot of work here.

• 14:23Speaker 00

  Like I'll be on my phone. And this is the other reality is like, unfortunately, that means I can work like 21 hours a day now.

• 14:33Speaker 01

  And so you do.

• 14:35Speaker 00

  I do. And then, you know, and how do you deal with this? How do you deal with like the, the, the decision fatigue?

• 14:44Speaker 01

  Yeah. So I want to reflect, I'm going to answer that question in just a second, but I want to reflect for folks what I'm seeing here because you and I are different AI managers. And every time you come on, I'm like, no, I'm like, what an organized, what an organized individual. I am so impressed by you. You're like, where's my three steps to managing my, managing my agents? And I'm like, you don't just like float in the tokens, see what comes out the other time. You're like, here are my like prioritized triaged folders for Devin. And I'm like, I just slack Devin. Like he belongs to me. I just, like a sentry link WTF question, question fix.

• 15:27Speaker 00

  And that works. There's nothing wrong with that.

• 15:30Speaker 01

  It totally works. We're, we're so different. But I do think, you know, something you said earlier, I want people to hear, which is like, the job right now is to figure out how to manage agents. And the reason why, I know both you and I are 27. So of course we're, we're young, we're spring chickens.

• 15:46Speaker 00

  I have infinite energy.

• 15:48Speaker 01

  The reason why I think folks of a certain level of experience are like ready to cook right now with AI is all those management skills that you have developed over your career.

• 16:00Speaker 00

  Oh my God.

• 16:01Speaker 01

  Can be applied right here.

• 16:04Speaker 00

  Right now.

• 16:04Speaker 01

  How do you structure? How do you team? How do you prioritize? How do you delegate? When you have a team of 500, how do you still stay anchored on like the three things that are most important for you to do?

• 16:16Speaker 00

  Yes.

• 16:17Speaker 01

  Where do you micromanage? Where do you let go? Like all that stuff still.

• 16:22Speaker 00

  Pays off.

• 16:23Speaker 01

  Yeah. And it pays off. Yep.

• 16:25Speaker 00

  I, I think in multiples, like, so another practical tip I want to talk about is I have, it's a playbook, but it could be a skill if you're not using Devin. It's called watchdog. And, and, you know, we're starting to pick up a lot of family law firms now and they have a lot of cases, right? And, and I'm literally doing customer success and engineering and sales, right? And so it's like, whoa, how do you keep track of actually what's going on in these cases? You know, where, where are clients hitting UX bugs? Like where, what, oh my God, how do you keep track of it? So watchdog, what it does, and I, this is multiple iterations, everybody. So what you want is like, when that moment where you feel like, I need to know what's going in the business right now. I feel stressed. I don't know what's happening.

• 17:11Speaker 00

  This is when I run watchdog. And, and what it does is it basically, it, it has a skill for, that, that's repeated for each firm. So think of these as customers, right?

• 17:24Speaker 00

  And it goes into each account and says, what's going on with this account? Like, what was the activity since we last did watchdog? Where are the errors? Where are the sentry errors? You know, where does all this happen? But then, but then you don't want just data. You want it to be transposed and filtered and sorted for you. So then it comes up with the top three things that are going wrong. And then either like, well, wait, have they been fixed? Because we're shipping a lot of PRs. Like I'm shipping probably 40 PRs a day now. So did it get fixed or is it currently being fixed? Or is there an open PR that never got merged, right? And it just gives me all that. And then I talked to that thread about, okay, let's dig in a little bit.

• 18:07Speaker 00

  And that's how I manage the chaos.

• 18:10Speaker 01

  Yeah. I, you know, you asked me, how do I manage the, the decision fatigue? And I feel like I start a business for every child I have. So like three kids, three businesses. I have a lot of decision fatigue. There's, there's a couple of things that, that I do. I approach it very differently, which is I think about bucketing my time more than I think about bucketing the tasks, which is like, there's time in the day that I do like very, very specific tasks where I'm co-piloting with AI. There's other time where I'm just like, I dedicate to spinning off everything or checking in. So I have to, but as a solo founder, I have to bucket my own time.

• 18:47Speaker 00

  Interesting.

• 18:47Speaker 01

  On things. The other thing that I use is I use decision fatigue as a prioritization heuristic, which is functionally, if there is a feature that I'm working on that I let die on the vine, that is my.

• 19:03Speaker 00

  That's your symbol.

• 19:05Speaker 01

  That's my symbol for maybe this one is not the one I really care or believe in. And so I, I sometimes think that people, you know, don't, don't tell anybody, I know I'm a token maxer and you should let your agents run 24 hours a day and ship as much code as humanly possible and software everywhere. And I really don't ship more than I think the market.

• 19:29Speaker 00

  Yeah.

• 19:29Speaker 01

  Wants. Like I really try to like constrain my output capacity. Not on quality, not on bugs, but like, because I don't, I don't think I get multiples of quality off of multiples of output.

• 19:44Speaker 00

  I totally agree.

• 19:44Speaker 01

  And so I would rather reserve that creative energy for something else, or at least keep it inside till there is something there. And so I do, I run less overnight. I run like bugs and triage and all that kind of stuff. I also work with an engineer, so we kind of ping pong overnight. But I don't, I don't do the sort of like seek out ideas, solve them and always ship them. Cause I just don't want it. It puts too much on me and two, I don't think it actually adds value.

• 20:13Speaker 00

  I agree. I actually think we're nowhere near any frontier model having the intelligence to know what to ship.

• 20:20Speaker 01

  I, I agree.

• 20:21Speaker 00

  It's, it's like a million miles. I mean.

• 20:24Speaker 01

  I agree.

• 20:24Speaker 00

  So, so yeah, I think, you know, I think we all wanted these sort of like automatic improvement loops to, and they just don't work with product. That's kind of a joke.

• 20:33Speaker 01

  And I, and I tell people, it's also because all of this intelligence has not magically created markets where there were none before, other than AI markets, right? It's like, it's not like all of a sudden there are these like buyers that have shown up for solutions that somebody, yeah, for stuff. And so I do think there's this like mismatch between the quantity of output of code and.

• 20:59Speaker 01

  Like commercializable product.

• 21:02Speaker 00

  Amen. So I want to double click on that. I think what, what, what is happening is people are not getting out of their chair enough and actually talking to real people. So, so, you know, the way I figured out PMF for Untangle was, you know, I went outbound with an email. I did as a Google meet with a potential customer and I talked to them. Boom, that was step one. But then step two is like, as soon as we landed them as a customer, I was like, can I come to your office? And I just want to sit and I want to talk and I want to, and this is all like obvious product stuff, but I think because we can ship so much and build so much digital product, we're, we're not getting out of our chairs enough.

• 21:43Speaker 01

  I totally agree.

• 21:45Speaker 00

  Go talk to humans, for real.

• 21:47Speaker 01

  Humans, yep.

• 21:48Speaker 00

  And, and build something neat.

• 21:50Speaker 01

  This episode is brought to you by Jira by Atlassian. The teamwork graph in Jira delivers 44% more accurate agent results with 48% less token usage. That's a huge difference when working with AI coding agents like Claude, Cursor, Codex or Copilot. The hardest part of shipping with AI isn't the code, it's the context. What's the right ticket? What did the spec say? What got decided in Slack? The teamwork graph pulls all of that from across your entire stack, from Jira and Confluence to GitHub, and feeds it directly to your agents before they write a single line. You assign the work, the agent gets everything it needs and a PR surfaces when it's ready. No digging, no context

• 22:34Speaker 01

  switching, no broken flow. Same team, smarter agents. Try them free at jira.dev. That's J-I-R-A dot D-E-V. The other thing I want to touch on, because I think you breezed past it, but it has been a game changer for me in terms of using background agents and in particular Devin, which is I think people have still bucketed these cloud coding agents as coding agents, which is like their only job is to write code for a product or software product. And in fact, Devin runs Deal Desk for me. So Devin's doing quoting, custom quoting for me.

• 23:16Speaker 00

  Yeah.

• 23:17Speaker 01

  Like setting out operations, documentation, customer triage. Like I, I always think, you know, what would you do with somebody who both knew your code base and could write code to solve anything in your business? That's how I use these, these background agents. And I do think that people really don't ask enough of their background agents and especially ones that are very good at coding. I think that's why it's so interesting. Cursor just yesterday came out, or Cursor or Grok, whoever, whoever they are at these, came out with Grok Bot. And I was like,

• 23:57Speaker 01

  you know what? The Cursor chat experience as a, as a background agent was a great one. It was like one of our first, first episodes. And so I'm not surprised to see these like coding platforms start to generalize into business agents in general. And I also feel like more than engineers need to get hands on with, with background agents and cloud, cloud agents, because there's so much that they can do for your business pretty autonomously.

• 24:26Speaker 00

  A hundred percent. But let's talk about the why probably both of us are still using local agents for some things. So it's interesting. So right now as well, I have Codex running. And I have, and I have it using Goal. And I find that in particular, like the low latency, plus the browser on my machine pairing, like plus like basically infinite tokens, you know, from, from OpenAI.

• 24:54Speaker 01

  Yeah, thanks.

• 24:55Speaker 00

  Yeah, cool. I'll take the subsidy. That, you know, plus Goal really, really is good for when you have this kind of, I've got a feature, it's a lot of front end UI.

• 25:06Speaker 01

  Yep.

• 25:06Speaker 00

  I just need you to grind for a lot and I need to be here and kind of hold your hand.

• 25:10Speaker 01

  Yep.

• 25:10Speaker 00

  I find Codex unmatched for that.

• 25:12Speaker 01

  Yeah. So if I were to, because what we said we would do is compare and contrast our solo founder AI setup. So I would say I am personally spending my time in Codex when I'm pair programming or, or managing a product. It's, it falls into a couple of categories. It's going to be like large feature initiatives that span front end and back end that both like, again, need like a lot of code written and you have to grind on it and needs that like intervention every now and then to narrow it to the experience that you want. A lot of like long running refactors and Hectet stuff, like again, I think the latency is unmatched. It's Goal is really, really effective.

• 25:57Speaker 01

  Subagents are really, really effective. And then I use Codex a lot for, I think it's very good at verification. And so a combination of Goal plus a CLI tool inside your code base that can verify core flows, or I use Goal and browser use to do front end verification a lot. So I say, write user stories, use Chrome, pull up the preview branch in Chrome, go through it as a user, list which user stories pass, identify bugs, fix them until you're good. Those are kind of what I'm using Codex for. When I'm using background agents, it's a lot of like auto fixes on bugs.

• 26:38Speaker 01

  One out, like when I'm on calls with, with customers and they say, oh, do you have this MCP? I'm like, I do now. So one off things. And then sort of operations, as you said, like I use a lot of agents for, for operations. And then, you know, I do have a couple like purpose built agents in the stack. So I use Bugbot a lot for PR review. And then I just built this thing delightfully called Merge Mommy that, that does risk assessment on my PRs and auto approves low risk ones. Cause I am also shipping a lot of PRs. And so that's.

• 27:15Speaker 00

  And is that, tell me about Merge, tell me about Merge Mommy. Like that's good. And then I'll tell you about my land PR skill. I want to compare and contrast.

• 27:23Speaker 01

  Yeah. So I did an episode on this a week or two ago. It's an Eve agent. So it's deployed on Vercel. And whenever a PR is opened and then passes all its CI checks, including Bugbot, Merge Mommy gets triggered. And then what she looks at is she scores the PR on like five aspects of risk, like how big, blast radius, security, whatever it is. And then if it gets scored as a low, Merge Mommy stamps it with an approval and we can merge without a human approval.

• 27:55Speaker 00

  Cool.

• 27:56Speaker 01

  But if it's medium or high, then it pings us in Slack and says, it pings either my colleague or I, or somebody else in Slack and says, this one's ready for you to review. Cause for our SOC 2 compliance and all our controls, we require kind of like alternative review. And it just allows us to like get through our PR queue.

• 28:15Speaker 00

  A lot. Okay. First of all, annoyed I missed that episode. That sounds amazing. I'm gonna go watch it. But number two, interesting. So I have something called LAN PR, where basically it's, it's a playbook in Devin, but you can think of it as a skill. And it's usually it's when the agent comes back and says, okay, the PR is done, you know? And, and then I trigger it. And then what it does is it basically makes sure that number one, it runs a fresh Devin review on the PR, which, and Devin review is like an actual product inside of Devin, which is another reason why I think people should just pick, I'll sort of rant about that later, but the code factory, like it should have

• 28:55Speaker 00

  a good, a good built in review that is a real review product, right? So it triggers a review and then it basically allows up to two loops on that, right? You know, okay. All right. Find all the, all the bugs, check that all the comments are, are resolved and then, and then there'll be some more that come up, do it again, does it twice. And then, then the thing is, is it records a, a video walkthrough. And, and this I find Devin is just really good at, like, so what it does, it records in the browser and it actually kind of narrates over, it's got captions about what it's doing. And it's got this cool little, little test, you know, list of tests it's doing and whether

• 29:37Speaker 00

  they're red or green. And you get all this out of the box. Like I tried to build my own code factory and get all this stuff. And I was like, I don't have time to do this. And so the LAN PR, basically it has to show me that video. And then I, then I say, video approved, land it. And then it will merge. So that really works for me. It's kind of that, I think this is where CICD and DevOps, like now you have to be even better at it, like, and really care about it. And, and we use merge queues and GitHub as well. I'm loving merge queues. So a couple of the nitty gritty, this is what I'm actually doing stuff.

• 30:13Speaker 01

  Have you, okay. You all know, this is how you know, not, not sponsored by Cognition, but you all are welcome. We love you.

• 30:20Speaker 00

  Yeah, come on.

• 30:21Speaker 01

  Um, I can't get Devin video verification to work. I can occasionally get it to screenshot.

• 30:30Speaker 00

  Really?

• 30:30Speaker 01

  And like on front end stuff, it taps my, um, my credits for the session and then just falls over. I don't know if it's probably a problem. It's probably a me problem. All AI things are probably a me problem.

• 30:43Speaker 00

  No.

• 30:44Speaker 01

  But I have not been able, but I actually, I do want to show this for folks that want to know exactly what it looks like, because I found an example of it. So I'm claiming that it doesn't work, but then I'm gonna make it work.

• 30:57Speaker 01

  Devin will actually go ahead and test it and record a little video of it using. And you can see these little captions down here.

• 31:10Speaker 00

  Isn't that cool?

• 31:10Speaker 01

  It's exposing its reasoning. It's super cool. It's very useful. I can just only get it to work.

• 31:16Speaker 00

  Okay, so a couple of interesting comments I have that. Number one, interesting that you're in Slack.

• 31:21Speaker 01

  I'm never in Devin UI, never.

• 31:24Speaker 00

  So interestingly, I'm never in Slack.

• 31:28Speaker 00

  I am trying to cut Slack completely out of Untangle and the way the company is going to exist. And I was laughing because you have so many sidebar, you know, notifications and exclamation points. And, and we were saying, you know, you're seeing the Devin preview in Slack. And, and I have a very kind of perfect experience with the video playback testing of Devin. And that's because they use the web app. And so here's what I'm thinking. And again, this is probably going to change, but as we hire people, I'm actually going to require like multi-user interaction to happen through Devin.

• 32:06Speaker 00

  So we talk in threads about what is going on. And that's thing one, because, because then you're not trying to always figure out, where's the context for this thing, you know? Is it in your agent? Is it in Slack? Is it in Google Docs? Like, where is it? So I'm going to try to do that. Now, obviously, that doesn't work for DMs and channels and all that stuff. But I also think, and Jack Dorsey, I think, was saying this, you know, they launched Buzz. It's kind of interesting. I think he banned DMs or somebody banned DMs in their company.

• 32:37Speaker 00

  And I think, you know, as we scale, I'm going to do the same thing. Like, I don't want any one-on-one chats. Like, if we're having private conversations, I'll call you. So that's why I just think Slack is not built for the AI age.

• 32:52Speaker 01

  Well, let me, let me tell you why. I love Slack.

• 32:56Speaker 00

  Okay.

• 32:57Speaker 01

  I like to.

• 32:58Speaker 00

  Commence me.

• 32:59Speaker 01

  I like to person, first, I like to personify my agents. I like, as a solo founder, I need my colleagues to be fun.

• 33:07Speaker 00

  Fair. I'm just boring and mean.

• 33:09Speaker 01

  It's, it's marginally more fun in Slack to use Devin than in the web app. Although they're interchangeable, actually. You're just seeing a reflection of the web app. The, the reason why I think I had been so committed to Devin in Slack is actually not because I think it's a better experience. It's that very early on deploying background agents inside companies. This is where we're going to agree to agree. Inside companies. So when I was at Launch Darkly and we were first deploying background agents in particular, Devin, it was such an effective change agent move to put these background agents in public Slack threads.

• 33:47Speaker 00

  Interesting.

• 33:48Speaker 01

  And so like, it was one of the places, actually it was the only place I had experienced up to that point, where you could do real-time coding work in public.

• 33:58Speaker 00

  Cool.

• 33:58Speaker 01

  And I think that changes people's perception of, oh, what is AI capable of doing? How do I work with AI better? And then I do think, you know, in particular, Devin's pretty good at multiplayer chat. It's one of the few ones that like, you don't trip over your colleagues when you're both chatting in a thread.

• 34:19Speaker 00

  In Slack.

• 34:19Speaker 01

  But I think it's, yeah, in both, in Slack and in, in the web app. And I think it's just, I'm a creature of habit, having told everybody to shove their Devin threads in public channels.

• 34:32Speaker 00

  Yep.

• 34:33Speaker 01

  That that's why I'm using it.

• 34:34Speaker 00

  Okay.

• 34:35Speaker 01

  You know, I, I do think that chat's not going away because I think that's how humans like to work, like communicate. So it's hard to feel like chat's going away, going away, but I do think more and more people, if they don't want to get rid of Slack, want to get rid of Slack culture, which is like, DM culture, private channel culture, like.

• 35:02Speaker 00

  I don't want any of that crap.

• 35:02Speaker 01

  Needless, needless conversation culture that doesn't result.

• 35:06Speaker 00

  It doesn't do work.

• 35:07Speaker 01

  Anything. Yeah. And so almost like every thread in Slack should result in work.

• 35:12Speaker 00

  Yep, but it doesn't.

• 35:14Speaker 01

  Better. It doesn't.

• 35:15Speaker 00

  It devolves into just, you know, and I love humans, trust me, but I don't, listen, you know, I've, I did a stint at Intel where I was an employee and I learned a lot about being, what it's like to be an employee. You know, and I just realized you spend a lot of time talking about stuff that doesn't matter. You're not empowered to do anything. And it's like, you know, what's the point of that? So, and, and you're, the agent is always going to need the context of whatever you're talking about. So, so have, have the conversation there. So before we leave, I know you have a hard stop. I do want to talk about the third plane that I have, which, which is a little bit of Claude code. So I, I, yeah, so I'm number one, it's like, it feels like infinite tokens.

• 35:58Speaker 00

  But I just can't get around how good Claude design is. Now, I don't mean in coming up with new design. I think it's bad. But I think for carrying forward, you know, a design system, it, it does something magical that Codex is hilariously bad at. Devin is terrible at, at design. And so I always have to go back to Claude design. So am I missing something? Or do you do something magical design that's working for you?

• 36:25Speaker 01

  Well, this is a, this is a great question. I was telling you before this, I'm launching a new business. We've got a new brand, new design. And so I went through this process very recently, so I can tell you what I did. I think Claude Designs, Create a Design System, is like unmatched right now.

• 36:43Speaker 00

  Okay.

• 36:43Speaker 01

  It's, it does, it does a great job of decomposing either an example site or a Figma or something into relative non-slop design tokens you can reuse across, across, across a property. Now, that being said, my most recent process is I went to Claude Design. I created the, the kind of like design.md and the, the design file in, in Claude Design. I downloaded that, gave it to Codex and said, take this plus what's in the Figma and actually build me a technical design system.

• 37:23Speaker 00

  Oh.

• 37:24Speaker 01

  And it built all the shared components. It made this nice, like, interlinking in our mono repo.

• 37:30Speaker 00

  Oh, wow.

• 37:32Speaker 01

  And then I, I also tested with Codex using the Figma plugin to then go pull all those design tokens back into Figma, into components that I could use.

• 37:46Speaker 00

  So once you had the design.md and ported it to Codex, then you were locked.

• 37:51Speaker 01

  Yeah, that's exactly, exactly what I did. I mean, my problem with Claude, infinite tokens, very intelligent, my blood boils reading Claude output. It like actually raises my blood pressure. It makes me so.

• 38:08Speaker 00

  So I, the, the one podcast I listened about the, like, the, the opinion on Opus 5 was yours.

• 38:15Speaker 01

  Yeah.

• 38:15Speaker 00

  Because I, I love your take. And I couldn't agree more. I'm so sick of Opus 5. Like, it, it's just so frustrating. It's terrifying in one way because it's like, oh my God, it's way worse than we thought.

• 38:28Speaker 01

  Yeah.

• 38:28Speaker 00

  You know, about everything. And so you're like, okay, you know, which is actually, what's wrong here? Is that, and I, so it's like super smart, but I really am sick of Opus 5. Yeah.

• 38:39Speaker 01

  Yeah, but I do think I, the, the applications in some like very specific deep spikes, cybersecurity design, like kind of unbeatable on the, on the execution side still in Claude. But I am, I just, the, the harness experience of Codex is just a lot more effective and enjoyable for me.

• 39:00Speaker 00

  It's so good. So last question I had before we go is, we can both answer, you go first. Like, what's your current actual EA? Like, who's your actual sidekick that does all of your agent work as an executive assistant?

• 39:14Speaker 01

  Polly the Open Claw.

• 39:16Speaker 00

  Interesting. Okay, you're on Open Claw.

• 39:18Speaker 01

  Still live in Open Claw.

• 39:19Speaker 00

  Okay.

• 39:20Speaker 01

  Just pod committed there.

• 39:21Speaker 00

  Yeah. I love Open Claw. I love Pete. I love the team. I love it. I just moved off. I was like, screw it. I'm just gonna run Claude code on my Mac Mini. And I actually want to switch to Codex because I just love, love, love the UI of Codex. They're so, it's the, it's wonderful, the whole thing.

• 39:41Speaker 01

  Yep. Yep. Yeah, no, I use, I still Polly the Open Claw is still, still running. But I do now have my emails constantly open in the browser in Codex and Codex functionally runs my email. It, it, it ran through 2,000 unread emails in my Gmail the other day and just changed my life.

• 40:04Speaker 00

  This is, this has been a game changer. So like you, I've hired a lot of people in my life, like hundreds and hundreds of people and, and let go lots of people and, and all this kind of stuff. So I, the way I'm running my hiring process right now, I really love it. And we'll see if it works out. But, but I think, I think this is interesting. So what I decided to do is I basically on Twitter, I was like, okay, we're hiring our first engineer. You gotta be like, super duper duper AI forward, you know, of course. And I don't really care where you live. And all I want you to do is record a video, full screen, everything, your, your whole desktop of you building a new feature for an app that already exists. And I just want to see the whole thing.

• 40:49Speaker 00

  And I don't want to talk to you. I don't have a meeting with you. I don't want to get to know if I like you. I just want to see how good of an agent manager you are. And the videos I'm getting are fascinating and very revealing about whether I would want to work with this person. And I'm liking it. So I've got, you know, a number of people that are probably going to move to phase two, where then what I'm doing is say, okay, give them access to Devin and actually build a real thing. And then, but record a video of that. And then basically at the end, I'll do some sort of quick, well, we should meet, you know, and chat. And then I'm going to hire them. And then I'm just going to watch their agent output. And that's how I'm going to manage them.

• 41:28Speaker 01

  I mean, well, one other thing is if they work with Devin, you get the replay video anyway of how the Devin computer works. So saved you, saved you a step.

• 41:37Speaker 00

  Thank you.

• 41:37Speaker 01

  The, the one meta observation I have on this is you're almost hiring a human like you would hire an agent, which is like, I love you, like, we'll get along, I'm sure. But like, really, I just want to see how you do the, you're, you're doing a verification loop is what you're doing.

• 41:56Speaker 00

  And think about how much better this is in this insane way we used to hire.

• 42:00Speaker 01

  Yeah.

• 42:00Speaker 00

  You're like, I like this person. And then you're like, we're going to kind of ask you to do some weird technical project and then, so very excited to see this shake out.

• 42:09Speaker 01

  Okay, well, just to recap for everyone, the, the two most AI-pilled people on your timelines, spent most of our time talking about how too much AI code doesn't really get you great product. You should just go outside with your human body and talk to customers.

• 42:27Speaker 00

  Agreed.

• 42:27Speaker 01

  Keep your priority on a piece of paper.

• 42:32Speaker 00

  It works.

• 42:34Speaker 01

  It works. But, you know, on, on the technical stuff, set up your background agents, prioritize their work in a way that you can manage. Everybody's becoming agent managers. Your coding agent can do more for you than just code. And the new interview is just let them cook and, and do a screen share.

• 42:55Speaker 01

  And also to conclude, the solo founder AI agent managers are both hiring. So Ryan, where can we, we, I know we can find you online, but where can we find you? And how can we be helpful? Who are you hiring for?

• 43:07Speaker 00

  Thank you. So I'm super active on X, so just Ryan Carson, hit me up anytime. And then if you want to help people have better divorces, because people are going to get divorced, okay? Go to untangle.us, check us out. And then just shoot me a DM with a video of you working. And I think we're going to hire someone pretty fast. So, so move fast and, and say hello.

• 43:31Speaker 01

  And I'll say, if you impress Ryan with your video, you're probably going to be an upcoming guest of How I AI, where you will show us how you use your background agents. Awesome. Well, Ryan, thank you so much for joining.

• 43:43Speaker 00

  This has been a blast. Thank you.

• 43:45Speaker 01

  Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube. Or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.