Grok vs. Dot vs. Muse vs. Claude Duet: which one crushes the competition?

I have personal-agent fatigue, so I ran the first How I AI personal-agent benchmark: Grok Bot, Dot, Muse, and Claude Duet across nine real tasks, from a Slack reply and a Sentry bug to a Figma thumbnail, inbox triage, a family newspaper, physics homework for a fourth grader, shoe shopping, and a ChatPRD launch video. I used computer use to operate them: Claude ran Dot, Codex ran the rest and scored the results live. Here’s the scoreboard—and which agent I’d actually trust for work vs. life. — Brought to you by: WorkOS https://workos.com/?utm_source=lennys_howiai&utm_medium=podcast&utm_campaign=q22025—Make your app enterprise-ready, with SSO, SCIM, RBAC, and more Firecrawl https://firecrawl.dev/howiai—Give your AI agents web data and beyond — In this episode, we cover: (00:00) Introduction (02:13) The personal-agent benchmark setup (05:00) The nine tasks and the prompts I used (07:20) Scoreboard overview (10:06) Task 1: Draft a Slack reply as me (11:39) Task 2: Cut a short-form video from raw Dropbox footage (14:56) Task 3: Fix a bug from Sentry (17:30) Task 4: Build a How I AI thumbnail in Figma (19:25) Task 5: Triage my inbox for to-dos (21:06) Task 6: Build a family newspaper (23:30) Task 7: Explain a physics homework question to a fourth grader (24:54) Task 8: Shop for shoes (26:46) Task 9: Make a ChatPRD Product Intelligence launch video (30:01) Final scores (32:38) My takeaways — Tools referenced: • Grok Bot: https://x.ai/bot • Dot: https://chatgpt.com/features/dots/ • Meta Muse: https://ai.meta.com/muse/ • Claude Duet: https://claude.com/product/claude-duet • Codex: https://openai.com/codex • ElevenLabs: https://elevenlabs.io/ • Figma: https://www.figma.com/ • Sentry: https://sentry.io/ • Linear: https://linear.app/ • Dropbox: https://www.dropbox.com/ — Where to find Claire Vo: ChatPRD: https://www.chatprd.ai/ Website: https://clairevo.com/ LinkedIn: https://www.linkedin.com/in/clairevo/ X: https://x.com/clairevo — Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Claire Vo?

  I told you all I have personal agent fatigue, but I am here for you. So today I am going into the personal agent minds and I am running the very first how I AI personal agent benchmark, which goes across four agents and nine tasks to tell you what personal agent is actually the best. I am going to try to run this in the most unbiased, equal way. I will tell you how I set up the benchmark, what is in the benchmark, how I ran the benchmark. And then we're going to look at the results and at the very end, get a summary of what agent I think does best on which task. This was a really fun and also hard one to run.

• 0:45Claire Vo?

  I spent definitely several hours today figuring out how to run this and getting it done. I hope that you all find this helpful and I am going to be along the ride with you because I really haven't looked at many of these results. And so we're going to look at them live. We're going to see how it goes and I'm going to tell you which personal agent is the best for you. This episode is brought to you by Work OS. AI has already changed how we work. Tools are helping teams write better code, analyze customer data, and even handle support tickets automatically. But there's a catch. These tools only work well when they have deep access to company systems. Your copilot needs to see your entire code base. Your chatbot needs to search across internal docs.

• 1:28Claire Vo?

  And for enterprise buyers, that raises serious security concerns. That's why these apps face intense IT scrutiny from day one. To pass, they need secure authentication, access controls, audit logs, the whole suite of enterprise features. Building all that from scratch? It's a massive lift. That's where Work OS comes in. Work OS gives you drop in APIs for enterprise features so your app can become enterprise ready and scale up market faster. Think of it like Stripe for enterprise features. OpenAI, Perplexity, and Cursor are already using Work OS to move faster and meet enterprise demands. Join them and hundreds of other industry leaders at workos.com. Start building today.

• 2:14Claire Vo?

  Okay, before I go into the results of the benchmark, I just want to tell you how I came up with this bench. So the first thing I did is I went into all my agents. I went into DOT, Grokbot, Muse, and Duet, Claude's Duet. And I said, I'm going to run a benchmark. Give me like the top tasks that you do for me. And so each of them gave me the top 10 tasks and then I put them all in a Claude and I said, okay, come up with a way to run a personal benchmark against this, picking the tasks that were most common across the agents. Because to be honest, I use different agents for different things. And so I wanted to find sort of the most cross cutting use cases as well as use cases that would test different components.

• 2:56Claire Vo?

  And then the way I was going to run this is I was going to have one operator agent, which was going to use computer use to run all the other agents and then keep the benchmark going and so on and so forth. The one problem with my operator agent plan was I also needed to run DOT and Codex is my favorite computer use agent. So just to be very clear, I had Claude run DOT and then I had Codex run Claude, Grokbot, and Muse. So there's like a little bit of diff in the operator agent, but I thought this was pretty clever. The other thing that I wanted to make sure was that all of my agents had access to the same things. And this was a really good tip for you all that are suffering like me, where you have

• 3:40Claire Vo?

  all these agents and they all have access to like a couple of different email addresses, a couple of different connectors, like one's connected here, one isn't connected there. So one tip I have for you all is I had Codex use browser use to check that all my agents were connected to roughly the same stuff and had access to the same skills. So just to set the baseline, we picked a couple of tasks across the top ones from my agents. I used Codex to make sure they all had access to the same things, although I forgot one of my agents, Muse, and that they all had access to the same skills. So they were like as baseline as humanly possible, although of course they all benefit from memory and all the different things that are in each of those agents.

• 4:25Claire Vo?

  So then I had this even kind of like plan. And then what I did is I had eight, actually nine, use cases that I was going to test each of these agents against and created very specific lightweight prompts for each of these. I didn't want it to do that like sort of AI prompting where it does like a paragraph for you. I was like, keep it in my voice, do exactly what I would do. And so I plan, there's actually nine here, I'll walk you through them, run sheet of tasks with the prompt. If an agent asks for something, what you should say and how you can nudge it. And so the tasks that I came up with were drafting a reply in Slack to my friend at

• 5:08Claire Vo?

  lovable that asked me for something this morning, cutting a short form video, fixing a bug from Sentry, building a how I AI thumbnail, triaging my inbox for to do's, building a family newspaper, explaining a physics question answer to me because I'm going to explain it to my fourth grader, going shopping for shoes, and then making a chat purity launch video. So those were the tasks. And then, as I said, I had Claude computer use, run DOT, and then I had Codex run computer use against Claude Duet, Grokbot, and Muse. And then I had it run and time all the things and do all the results.

• 5:52Claire Vo?

  Before we get into the full results, just a couple of observations. One is, man, I'm now going to have computer use run all my agents, running other agents. Like the parallelism of this exercise was thrilling to the heart. Two, I did forget to hook up Muse to a bunch of business tools. I just don't use Muse for business stuff. And one of the things I noticed is as I was like, the bench was running, I was like, oh, I forgot to hook up Slack. Slack isn't in the connector list. And so it was just kind of annoying. So it's very clear that Muse is sort of like the one off the side. That's not really meant for business necessarily. It's really more of a personal agent.

• 6:39Claire Vo?

  And then Claude computer use is just hot garbage. So I was like super frustrated having Claude computer use, but I needed something other than Codex to run Codex. And so we get what we get and we don't get upset, but I did not love Claude computer use. So let's dive into the results. I'm actually recording my audio right now. So as I talk through the results and tell you what I think, we're going to put that into Claude at the very end. It's going to score all these things and come up with which one I like the best. So this sort of live human in the loop scoring, but I am going to go through all these, tell you what I think, and we can see if you agree as well. So let's go for it. All right.

• 7:21Claire Vo?

  So this is how I personal benchmark DOT, Grok, Muse, and Claude. Call out here, Muse and Claude could only get seven of the nine tasks done. We will see if that was human error or agent error, but DOT and Grok completed all the tasks. Okay, so how we ran it, again, the prompts were just one line. They were very human driven. They weren't those like big AI prompts. The agent did not drive itself, so the agent was driven by an operator agent. I had to like babysit things, so I had to nudge things, computer use broke, all that kind of stuff. So I was kind of in the loop. So we did a little zigzag and then, actually judge blind is fake. I am judging it not blind. I'm going to judge it live on this episode and you all are going to come along with me on my judging journey.

• 8:05Claire Vo?

  Okay, here's the entry scoreboard and then we'll do the final score at the end. Just a couple of things to see right off the bat. Muse is very fast. Muse was the fastest in four out of the nine tasks, even though it had to be steered the most. Grok and DOT also had a couple of tasks where they finished very fast. And then Claude was overall and in incremental tasks, the slowest. I called this out in my recent Claude duet review, just super slow. So I like forgot what it was doing, kind of frustrating. And then you can see here, this is really funny. Claude asked for the most permissions, like Claude was the most timid.

• 8:46Claire Vo?

  So you can see it asked four times for permission where all the other ones only asked once. So very interesting kind of meta view on safety and stuff like that here with Claude. So let's look at minutes per task. So you can see DOT and Claude are generally the slowest. And then Muse is consistently the fastest. Muse was the fastest on building a thumbnail, going through my inbox, building a family newspaper, and building a launch video. Grok was fastest at drafting a reply as me and fixing a bug. Interestingly enough, it was the fastest at coding.

• 9:27Claire Vo?

  And then Codex was the fastest at making a video and actually shopping for shoes for me, though we'll see how good of a job that it did. DOT was also the slowest on a lot of tasks. And then Claude, for whatever reason, was very slow on replying as an email. So you know, this impression that I had in my Muse review where I said Muse is very fast, actually held up in this experiment. And then my impression that Claude was very slow also held up in the experiment. So while you all may think that my reviews are not scientific, they are in fact backed up by this very tremendously accurate data science. Okay, so let's go through the tasks.

• 10:11Claire Vo?

  So the first one was replying to me. It was replying to a Slack. It was replying to a Slack to my friend at lovable who asked me to do a talk. So I will say all of these were fine. Muse couldn't do it because it didn't have access to Slack again, because I don't really hook it up to Slack. It's just not what I'm using it for. All of them were great. All of them said, yes, I will do this event. But I did like DOT in particular because it asked for me to be paid and it asked if it was remote. And if you don't know me, I like to be paid to do things and I like to not leave my house. So it did feel like ChatGPT Codex DOT knew me the best and knew about the things that

• 10:56Claire Vo?

  I cared about. My schedule is busy. I need to get paid if I'm using my time, all this kind of stuff. I promise lovable, I'll do it for you. I'm happy to do it. But it was pretty interesting to see.

• 11:10Claire Vo?

  Dot was the only one that did it. All the other ones were fine. I would call this, you know, Claude is the shortest, but actually did the most inaccurate response. So if I'm just eyeballing this, Muse didn't do it. Claude did short, but okay. I would say Grok is most in my voice and Dot did the most accurate job. So again, I think any of these tools is fine. They all did fine. This is not a very hard task. Okay. Now here's the fun one. I had these agents make videos for our Monday episode with Blair from Reddit.

• 11:53Claire Vo?

  And so it actually, all it did was go into my Dropbox, find the raw video, pull it down, cut it and make the video. Claude could not do it. Duet could not do it. It would not download a video. So Claude totally face planted on this one. Gets a big fat donut on Claude Duet for video editing. Bot Dot, Grok and Muse did them. And we're just going to go ahead. Why not? Let's watch a little of them live. Not listening. What is your prompting strategy? How do you get it back on track?

• 12:23Speaker 01

  I don't yell at them. I know some folks do. But what I do is I take a day and then I get more vague. And we all have bots with projects and

• 12:30Claire Vo?

  Okay, that was Dot. What is your prompting strategy? How do you get it back on track?

• 12:34Speaker 01

  So I don't yell at them. I know some folks do. But what I do is I take a day and then I get

• 12:40Claire Vo?

  Uh, that was Grok. Grok has bad overlays, but it did the like zoomy in, zoomy out. Video's better. Let's look at Muse real quick. What's your prompting strategy? How do you get it back on track?

• 12:49Speaker 01

  I don't yell at them. I know some folks do. But what I do is I take a day and then I get more vague. And we all have bots with projects and context. And I pop out of a project and I describe the thing large, like a movie director. So here are all the facts and then here is all the intents. And then I pull that back into the project and then it knows enough to apply it to what our goals are.

• 13:11Claire Vo?

  Okay. So this is what I will say. Very, very hot takes. Again, Claude didn't do it. These are all like kind of impressive in that I didn't touch them. The, the design of the one that I like the most is Muse. So Muse is very simple. It's got the how AI tag in the front. The, um, the caption layout is ours. There's no funky stuff. The cuts are punchy. They don't do the zoom and zoom out, but it's okay. The design of Grok is terrible. Um, and it's got this like overlay on his forehead, which is very unattractive, but it did that zoom and zoom out. And then I would say Dot looks very good. It looks just like the Muse one, but it again, just doesn't have those punchy cuts. So I want to give it to Muse. I'm giving it to Muse.

• 13:55Claire Vo?

  I think Muse would do the best. And then I think, you know, Claude failed. Grok probably the worst and Dot second choice. So again, I didn't really give it much to work with. I gave it access to the transcript or raw video. I didn't touch it and all of these cut. I think with a better skill, it could have done a better job. But I was actually really surprised by Muse because I have not had Muse do this task before and it really nailed it out the gate. So I don't know if you all know this, but I've been using Muse a lot for social, specifically posting social. But I may start to think about Muse for creating these social posts as well because it did quite a nice job.

• 14:40Claire Vo?

  The other thing to call out is Grok did the worst and took the longest. So again, I don't think it's really optimized for this video cutting use case. And then Dot was, it can't tell how long it did. I think it took about seven minutes, was good, not great at this short. Okay. The next one was fixing a bug. So classic coding use case. Again, another one that had one of these agents fail. Muse did not do it. I don't have GitHub access to Muse. I don't actually have any business system access in Muse. So Muse did nothing. I asked it to maybe assign it to Devon in Linear and it thought Devon was a person. Devon is not a person. Devon's an agent. So Muse fell off the board. But let's look at the other ones.

• 15:24Claire Vo?

  They all took about 10 minutes to get this done. And I actually had Claude Opus 55 go grade the fixes. And we'll just look at them right here. So Dot Codex got closest to the actual fix. So that one did the fix that we thought was the most accurate for the bug. Claude Duet though, found a second bug that we didn't find, which we really, really love. Grok did the worst job, which is it loosened validation. So it was just like, we're going to solve the bug by not validating this bad issue. And then Muse didn't do it. So I'm actually going to give Claude and Dot the wins here. I'm going to give Claude the extra thumbs up because not only did Claude find another bug and fix it,

• 16:09Claire Vo?

  it also identified that Dot had already fixed it and said, I don't need to ship any more code. So I would say for coding, Codex and Claude or Dot and Duet are truly kind of the best. I actually don't code that much with Grok. So I'm not surprised that this wasn't as good. I really only do my coding in Claude and Codex. So I'm not surprised these worked, but I was surprised at how bad the coding from Grok was. So kind of my instinct was right on these. OpenAI Anthropic much better at coding than Grok Bot, although Grok Bot's good at a lot of stuff. This episode is brought to you by Firecrawl, which gives your AI agents data from the web and beyond.

• 16:53Claire Vo?

  Your agents can search and scrape the web while accessing Alexandra's rapidly expanding library of over 100 million index sources and more than 200 data providers, all through one connection. Connect Firecrawl to ChatGPT or Claude to find people in their work emails, pick your fantasy football lineup or analyze trends for your next marketing campaign. Firecrawl is building the library for superintelligence, bringing more of the world's knowledge into the tools you already use. Go to firecrawl.dev/howiai to get started. Okay. Next one, let's look at episode thumbnails. So we did episode clips.

• 17:36Claire Vo?

  Now we'll do episode thumbnails. And I will say, um, we did the pre-work here. So our thumbnail templates are actually like quite lovely. So it's actually pretty easy to use the thing that MCP and create new thumbnails. So I'm just going to do a hot judgment here. Muse did the worst job. Muse pulled our existing thumbnail and then did this really ugly one where he's like, Blair is like peeking through creepy curtains. We hate this. I'm sorry, Blair. I would never do this to you. And so Muse is going to fail here. No good.

• 18:16Claire Vo?

  Grok though, Dot and Claude did perfectly fine jobs. Although I will say Dot and Claude just put new content in existing templates that we had. So didn't really get super creative. I would say Grok actually wins this one in that it gave me really good new ideas for these thumbnails and headlines, ones that we hadn't done, that I also think look really nice. Actually, maybe we should test these while they're live. They have like that YouTube vibe for them. And so this is like a fun tip for you all is I have consistently tested Grok, Dot and Claude Duet on Figma.

• 18:59Claire Vo?

  And Grok is definitely the best at Figma. Grok is like 100% the best at using Figma. I don't know if this is John at SpaceX SIs, like Figma bro, magic touch. But if you're using Figma, use Grok Bot, man. It is so good. So that is my preference. I'm not surprised this one. Grok takes the cake for Figma and maybe I will have it create the thumbnails for this episode. Okay. Something that might not be obvious from the high level benchmark that I want to call out about how all these agents did things is they all have different interfaces for giving you to-dos. And so while they all found the to-dos fine, I suppose,

• 19:44Claire Vo?

  and they all gave me reasonable things to focus on. I am going to say that Grok did the best job because it gave me a Grok document. And these just came out, I think, last week. And they're so nice. It's just this like beautiful list of things that I need to look at with little checkmarks, super organized, really nice to work with. And so, you know, all of them did a fine job finding the tasks. I would just say Grok gave those tasks to me in the best format. Muse only had access to my Gmail, but found all the things I need to do as a parent ahead of Halloween week, which is a busy week for parents.

• 20:26Claire Vo?

  And then Claude wanted to put my to-dos in its little like task follow-up list so I could track them. Dot did the worst job here for sure. It just gave me a message with like five things I should focus on. So Dot gets the thumb down for me on like the personal agent, personal agent. I would rank these as Grok, best interface, best job. This is what I use Grok Bot for. My chief of staff is in Grok Bot. I'm not surprised that it does a really good job there. Muse did a perfectly lovely job with the information it had. Claude was fine, but again, I just don't like that I have to be in the Claude app for this. And Dot really, really failed here. Okay, family newspaper.

• 21:11Claire Vo?

  So this was a really interesting one. This is a use case if you've used my Grok Bot, TradBot, that I really like, where we print out a newspaper in the morning for the kids to look at about what's going on for the week, what our schedule is like, what events, what news would be interesting. Claude did such a trash job that I feel like I should actually call it out. It made like a markdown file. It made me so, so, so sad. And Claude only had access to my work calendar. And so it was just like, I can't do anything. I made you an outline. And I had this experience a couple of times with Duet, where like it couldn't do the job and then just gave me like a markdown file that was like, this is how you can do the job. This is not what I want at all.

• 21:56Claire Vo?

  So Claude definitely failed the family newspaper test. Bot Dot, Grok and Muse all did different jobs. And I was shocked, sadly shocked, how poorly Dot did. Like, look at this design. It is trash, trash. And again, these are not my children's names. I just like had it swap out different kids' names. But this is so ugly. Like, what model thinks that this is a good idea? This is terrible. Grok did a really nice job. I actually run mine consistently through Grok. So you can see here, it's got Big Week at School. It's got a Don't Forget List. There's Bay Area Sports.

• 22:40Claire Vo?

  There's Halloween stuff. There's nice weather across the top. But Muse, per my earlier review, just absolutely crushed the family newspaper. It generated this nice little image. And I do have three boys and they're adorable like this. And so it has this beautiful image at the top, a headline. It did take two pages, but I'm okay with it. If it's really cute and it has our schedule on one page and then a cute day-by-day weather ideas inspiration on the second page. I just think it's really cute. I think if I tuned it with a skill, it would do an excellent job. So if you're looking for family newspaper, yes, I love TradBot. Please download it for your Grok Bot.

• 23:23Claire Vo?

  But maybe you tell your Muse to do it because it does a really nice job with family-oriented artifacts.

• 23:32Claire Vo?

  Okay, one kind of like side task I had these agents do, which I occasionally do on the go, is take a screenshot or take a picture of my kids' advanced homework, which I have forgotten how to do, and ask it, how do I explain this to my kids? So I haven't done physics in a long time. Like, how do I explain a velocity over time graph to my kid? Now, I do know how to do this, but we just used it as a good example. I just opened up the workbook. Okay, so I picked this like, explain this little exercise and asked them all to complete it. They all got it right. The answer is C. They all explained it fine. Who won, though? Muse won. Why did Muse win?

• 24:14Claire Vo?

  It took approximately a half a second to get this right. And so this was one where like the information was correct across the board. Muse was so lightning fast. And we didn't record it here because it was so fast. It was so fast and gave me the answer. And so I've actually been moving a lot of my ChatGPT, like Googly question and answer style chats to Muse because it does such a good job and it's so fast. So like sort of my go-to, like, explain how this works. What is this? Look this up. I've been moving those a lot to Muse again because of this experiments where it's really, really fast. Okay, another personal use case I did here.

• 24:59Claire Vo?

  Shopping for some autumn shoes. I don't know what's going to make me cool. I'm an old lady, but I want those like red Adidas Tokios. I hear that a slim shoe is really in right now, but I'm cheap. So I want to find a good deal. So I asked Grok, Dot, Muse and Claude to do this. Claude totally got blocked by almost every shopping site. I don't know why. It was like consistently blocked over time. It also looked for white shoes with red stripes instead of red shoes with white stripes. Like, I don't know why it did it. And it found the most expensive ones.

• 25:44Claire Vo?

  And so like Claude really failed this personal shopping task.

• 25:50Claire Vo?

  Grok found great ones. So Grok did a really, really good job, as did Muse, although Muse needed a little bit of steering. So Grok was super fast and did a good job. I have Grok do a lot of this like price comparison shopping for me. And this has been my experience where it's like really good at using navigation, shopping and price compare. Dot did not do a good job here either. So while Claude did like a bad job, Dot like barely tried. Dot did not do comparison. It didn't look at different colors. It didn't like say like, I can't find red, but I can find maroon.

• 26:33Claire Vo?

  So I would rank this as Grok best, Muse almost as good. Claude, not good, but at least it tried. And Dot did not try at all. So if you're using shopping, I would say Grok and Muse are your buddies.

• 26:48Claire Vo?

  Here's the laughs. I'm already laughing because I can tell one of these are two irables. So the last one, a launch video for a ChatPRD feature called Product Intelligence that I'm working on and launching. And you can just see here, they just really went lots of different directions. Now, I have a skill for this. And so it should be able to follow the skill and do what it's supposed to do. But we're just going to look at the differences between these. We're going to start with Dot, which is not good. It's like a PowerPoint. It's no, no, no, no, good.

• 27:27Claire Vo?

  I mean, this is very soothing, but this is terrible. Okay, so no, fail, I hate you, Dot, bad video, no good. Kind of like the newspaper. It's like, what are we even doing here? Grok crushed, though. Look at this.

• 27:43Claire Vo?

  Now, it actually used my skill. So where, oh, there's like a little error here. So the layout isn't perfect, but at least used my skill and looks like our video is looking. But the layout isn't very good at all. And there's a lot of overflow. So better, but not great. Muse ate it. Muse doesn't have access to my repo, so I did not set up our Sweet Muse for success.

• 28:09Speaker 02

  Every product decision has a paper trail. It is just scattered. Then Klerkly link shows what how confident together. Evans first, inference save.

• 28:16Claire Vo?

  Hold on, I'm pausing because although it hadn't access to nothing, it made up a bunch of stuff and gave a nice little British accent overview.

• 28:31Speaker 02

  Uncertainty out in the open.

• 28:33Claire Vo?

  Ooh. So I would say it's bad, but bad in an ambitious way. So I'm going to give Muse a couple of extra points for effort, a couple of points for effort. And then Claude, I don't know. This is why I use Claude to do this. Claude did a very, very good job and the most complete job.

• 28:52Speaker 02

  Customers are telling you exactly what to build. It's just scattered across support chats, call notes, bug reports, issues and poll requests. Meet Product Intelligence from ChatPRD. Connect the tools where customers talk and your team ships.

• 29:08Claire Vo?

  So what I will say is like Claude nailed the value prop. It used our branding. There were no layout issues. It actually went and asked and used the 11 Labs MCP to come up with a voiceover. I would have loved for it to be a female voice or even my voice that I have recorded in 11 Labs. But Claude absolutely crushed this task. Duet crushed this task. And so what I would say is Claude for video, Claude for video. Jen, it's just, it's just the best. Maybe not funny TikTok-style vertical video, but like these product walkthroughs, those sorts of things. I actually didn't think Claude Duette would do a good job and it did a really good job. Muse was terrible, but like terrible because of me.

• 29:53Claire Vo?

  Dot was terrible because of Dot. And Grok tried, like Grok followed instructions. So I can't give it a hard time. So that is the final kind of like my analysis. Let me pause. I'm going to throw this transcript into Claude and we're going to get the final ranking and scoring. All right, party people, here we go. Here are the results of the very first Hawa AI Personal Agent Benchmark. And...

• 30:25Claire Vo?

  Grok won. Grok Bot. Grok Bot's the best. And we love her. She's the best. So A+ SpaceX SI team. You cooked. The potato cooked. Grok Bot is freaking awesome. So Grok Bot got 93% done, high-quality fixes. I don't actually know how these scores work, but Claude told me it's true. It won three tasks, including doing my thumbnail, doing a to-do list for my inbox, and shopping for shoes for me. So that just shows the breadth of the Grok Bot. But as predicted, and this is why I tell people I use these two agents, Muse coming

• 31:10Claire Vo?

  up with all the personal use cases. We gave Muse a break on the tasks that it couldn't do because I didn't give it GitHub and I don't give it access to work stuff, so it can't do those tasks. But on the tasks that it had access for, it crushed especially creative ones. The short-form content, it did the best. Family newspaper did the best. It's super fast at explaining homework. It is my personal agent. I'm not surprised to see it up here. Dot coming right up behind. I think Dot got kind of like a little sandbagged. So I would actually say that Dot scored closer to Claude Duet than it did to Muse because Dot really like got scored for writing a Slack like me. Like, that's not amazing.

• 31:55Claire Vo?

  And it did a good job of code and it did a good job of orchestrating all the other agents. So Codex does orchestrate all the other agents, but Dot's not Codex. But it totally failed on anything creative. It wasn't great to work with. So I would say like Dot is a little bit more towards the Claude end of this ranking than the Muse Grok end of the ranking. And then poor Claude Duet pulling up the rear, 79%, super slow. It did fix the bug the most completely and it did my launch video the best, but otherwise either did not complete or lost against all these other models, unfortunately. So Duet's got

• 32:37Claire Vo?

  some work to do. Okay, so what are my takeaways? Pick your agent for your job, but if you're picking work jobs, probably give it to Grok Bot. And if you're picking life jobs, probably get it to Muse. And OpenAI, Anthropic, get your act together. You're losing to these other agents. Access really matters, so make sure that you have consistent tool access across your agents if you're trying to benchmark them. I like speed, so fast, I used to say all the time, fast speed's right, but fast is a very good part of an agent experience. Using artifacts, using different ways to communicate between human and agent, those like quality of life affordances that I call them really do matter. Grok's documents are

• 33:18Claire Vo?

  very nice. And so the way that the agent is actually designed and presents you information as a human is really important. Number five, safety styles differ. Claude's still a nerd. It's still asking for permission. So like when you have a personal agent, striking that right balance between permission giving and proactivity is really important. I really like Muse's sidebar with all the permission asks, so I can don't forget them. But Claude was just way too conservative. And then at the end of the day, there are still two agents racing for my coding workflows, Dot and Duet, which means OpenAI and Anthropic, Codex and Claude. I think this race will continue because this is where a lot of the tokens

• 34:03Claire Vo?

  are being spent, bought and used. But I would still go to those tools, but I actually wouldn't go to those agents. So what's interesting about this is while those two agents did the best job on coding, I would not use those agents to code. I would just use the underlying tools and models to code. So that is it. That is the How I AI agent benchmark. I am so happy to have done this. We now have a framework. When new agents come out, I can definitely run them. And now I'm going to go back to hanging out with my Grok Bot, hanging out with my Muse, and ignoring Dot and Duet. Thanks for joining How I AI. Thanks so much for watching. If you enjoyed this show, please like and subscribe here

• 34:46Claire Vo?

  on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.