Damn, the Cluely ragebait got us

What if cheating was just…the future of work? That’s the pitch behind Cluely, the viral AI startup that claims its stealthy browser overlay is “undetectable” and can help users bluff their way through everything from job interviews to exams. The company has raised $5.3 million and sparked a wave of backlash from startups building tools to catch cheaters. Cluely’s response? They’ll just build smart glasses or brain chips. Today, on TechCrunch’s Equity podcast, hosts Kirsten Korosec, Max Zeff and Anthony Ha are getting into the week’s headlines, including whether Cluely’s viral strategy is genius, gross, or both, and what it says about the future of work in the AI age. Listen to the full episode to hear about: Sam Altman’s latest World event in San Francisco where eyeball scans met privacy concerns Why Shein’s IPO is under threat from new tariffs, and how companies like Amazon are bracing for 100%+ duty increases on Chinese goods Waymo and Toyota’s agreement to explore autonomous tech integration The messy world of AI benchmarks and which major companies are allegedly gaming the system with LM Arena Equity will be back next week, so stay tuned! Equity is TechCrunch’s flagship podcast, produced by Theresa Loconsolo, and posts every Wednesday and Friday. Subscribe to us on Apple Podcasts, Overcast, Spotify and all the casts. You also can follow Equity on X and Threads, at @EquityPod. For the full episode transcript, for those who prefer reading over listening, check out our full archive of episodes here. Credits: Equity is produced by Theresa Loconsolo with editing by Kell. We’d also like to thank TechCrunch’s audience development team. Thank you so much for listening, and we'll talk to you next time. Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Anthony Ha?

  This episode of Equity is brought to you by Schneider Electric. Schneider Electric, your energy technology partner.

• 0:06Speaker 01

  I was this close last night to letting Sam Altman scan my eyeballs.

• 0:12Kirsten Korosec?

  Never.

• 0:12Anthony Ha?

  And then what happened?

• 0:13Kirsten Korosec?

  No.

• 0:14Speaker 01

  I came to my senses. I decided against it. I was at Sam Altman's World event last night in San Francisco, and there were all kinds of people there who are getting in front of these big metal orbs and scanning their eyeballs to sign up for their identification service. And they even unveiled a mobile device where you could take this scanning orb on the go and scan your eyeball wherever you are. And Anthony, I'm just curious, how likely are you to get your eyeball scanned by Sam Altman?

• 0:43Anthony Ha?

  I would say that the odds are pretty close to zero. I mean, world is such a complicated thing that we probably don't have time to explain, but like, it's also connected to Web 3, a bunch of stuff that I'm not super interested in. We're also at this moment where I feel like there was a period where people were saying, hey, maybe don't give your genetic data to 23andMe. That's being vindicated right now. So I would say scanning your eyeballs is not exactly the same, but kind of in the same category.

• 1:07Kirsten Korosec?

  I think scanning your eyeballs and putting it on the blockchain to confirm whether you're a human or not is potentially very bad. And the description that the company gives for its purposes is so vague and so open that it just gives me concern. So I would say this, unless someone is holding me down and opening my eye, they will not get my eyeball scanned.

• 1:34Anthony Ha?

  Right, you're doing a gesture right now. Would you say for our listeners, you're sort of showing what that might be like?

• 1:38Kirsten Korosec?

  Yeah, exactly. But you know what? I think we're going to talk about Sam Altman and not scanning eyeballs, but other stuff a little bit later. Welcome back to Equity, TechCrunch's flagship podcast about the business of startups. It is Friday, May 2. I'm Kirsten Korisak, Transportation Editor.

• 1:56Speaker 01

  I'm Max, Senior Reporter.

• 1:57Anthony Ha?

  And I'm, wait, are we using phone names? Which is first name? Sorry, I got thrown.

• 2:05Anthony Ha?

  And I'm Anthony Ho, Weekend Editor.

• 2:07Kirsten Korosec?

  Cool. We're just going by Max today, so just for the audience. He's like Madonna now. Just one name.

• 2:12Anthony Ha?

  Well, he's already shortening it from Maxwell to Max. And then we're just, yeah, Max Z, Max.

• 2:18Kirsten Korosec?

  Next week, it'll just be M. On the show today, though, we do have some other deals to talk about from Shein and Waymo and Toyota. And then we're going to get into Cluely and AI cheating, which I know Anthony can't wait to talk about, and AI Labs Gaming, their benchmarks. So I want to kick off this first deal by throwing it to Anthony, and it's to catch us up on Shein and what they're doing with their IPO.

• 2:42Anthony Ha?

  Right. Well, this is part of the broader, just every international company, any company that's relying on global shipping is kind of being like, what are we doing right now? And in the case of Shein, which is a Chinese retailer, but has a huge business in the US. In fact, the US accounts for, I think, 1/3 of the company's annual revenue. So reportedly, they're trying to figure out, what does our business look like if we face these huge Chinese tariffs? Because again, part of the whole point of their business is that the goods are supposed to be affordable. If you've got tariffs of more than 100%, that's going to be pretty difficult.

• 3:16Kirsten Korosec?

  145%, by the way. So this isn't surprising to me at all. I don't know if you were surprised by this. I mean, it's like a double whammy. On the one hand, a lot of companies are questioning whether they should go on IPO because of volatility. But specifically for Shein, they're so exposed.

• 3:34Anthony Ha?

  Almost every company must have some version of this conversation. And Shein is one of the first companies where it's like, oh, this is going to be pretty serious. They're looking at apparently trying to do manufacturing outside of China so that it doesn't get hit with the same level of tariffs. But like, they just don't apparently have the supply chain to do that in a meaningful way anytime soon.

• 3:55Speaker 01

  Right. Shein is one of the few companies that has really benefited from the de minimis exemption, which is going away in these tariffs. And for people who don't know, that's basically this exemption that's existed for a while where you can get duty-free imports into the US that are under $800. They are exempt from these tariffs and from these import fees. And that's really allowed Shein and companies like Timu and even Amazon now is starting to break into this, to get into these ultra-cheap goods where you can get hair ties for like $2 or something, or like a couch for like 15 bucks. Like, we've seen crazy things on these websites. And now that business just doesn't make sense anymore. Like, if that goes away, these businesses just don't work.

• 4:34Anthony Ha?

  Right. And we might think of this as something that's going to particularly hit companies like Shein and Timu. And there have been stories about already starting to see these like massive, like 366% price hikes on some products, which you just say is not the normal product. It's that's the extremely high end. But it's also something that Amazon is considering a huge amount of their products are also going to be affected by the Chinese tariffs. And there was a whole other blow up about the idea that not the main Amazon site, but one of the Amazon sites was going to show the actual additional tariff cost in their prices. And the Trump administration was not happy about this.

• 5:12Kirsten Korosec?

  No, they were not happy at all. I don't know if you read the column in The Verge that basically was like, "Amazon, now you have to do this." But it's very interesting to me how companies are navigating the business side of tariffs, but also the political side of tariffs, which is how do they give some price transparency? Or how do they find a workaround around tariffs without pissing off the administration and getting a lot of blowback? I was really compelled with how Rivian has been dealing with tariffs. And I don't know if you saw this, but Bloomberg had a report and we followed up on it, but they built up a stockpile of batteries for its trucks, SUVs, and commercial vans

• 5:55Kirsten Korosec?

  before the election and then right after in anticipation for the tariffs. So that was how they dealt with it. They basically stockpiled a bunch of stuff. And now like Amazon, we'll see how they navigate it. They can't stockpile. They couldn't have stockpiled things. So now they're kind of faced with the present day, like, how do I get consumers to continue to buy my products, not make the administration mad, but also let people know, hey, these prices are more expensive because of tariffs, not because of something on our end?

• 6:24Anthony Ha?

  So I think one of the things about that stockpile strategy, right, is also this question of what is the status quo? Is there even going to be a new status quo? Because I think a lot of businesses have taken this attitude of we're going to sort of go along with whatever the Trump administration wants. These are temporary measures. But even if you've got a stockpile, the stockpile runs out eventually. Either the tariffs are going to stay in effect in some form, or also there's just potentially just incredible ongoing chaos and no one knows what's going to happen and can't plan in advance.

• 6:53Kirsten Korosec?

  Yeah.

• 6:54Speaker 01

  Right. There is a lot of changes going on every week, it seems, with the tariff situation. I think this is a real inflection point for tech and the Trump administration. I think we do have to move on here because we have another really interesting deal to get into between Waymo and Toyota. They announced a partnership that is very fascinating to me. And Kirsten, I would love if you could kind of break down what exactly they agreed to here.

• 7:17Kirsten Korosec?

  Yeah. Let's be clear. I don't know if I can call it a partnership. I described it kind of cheekily as dating because the official announcement was like as non-committal as possible. It was like a preliminary agreement to possibly collaborate on something that might happen. But in Toyota speak, that actually is serious because they're a very careful, cautious company. They don't come out in public and talk about any even potential deal unless they really feel confident that it's going to happen. And in this case, Waymo, which is an Alphabet-owned self-driving vehicle company, has robotaxi services in a number of cities right now, including San Francisco, has signed a preliminary agreement with Toyota on two fronts.

• 8:00Kirsten Korosec?

  One, looking at building a new autonomous vehicle platform, which would be acting as a robotaxi, so a vehicle. And on the other one, which is actually the more interesting piece for me, is that Waymo tech may end up in next-generation Toyota vehicles that are sold to customers. So that means that Anthony, a few years down the line, you might, you know, be able to pick up that Toyota Corolla and it could have some Waymo tech in it.

• 8:29Anthony Ha?

  So you talked about how the language is like very vague and cautious. When you sort of read between the lines, do you feel like, well, that's just sort of like corporate speak, but like this is actually like a pretty serious deal and it's pretty likely we're going to see that Corolla with Waymo technology? Or is this sort of a pilot that might end up going nowhere?

• 8:49Kirsten Korosec?

  I actually don't think this is just a pilot that's going to go nowhere. I do think that this is going to be like a more serious relationship. Because again, as I mentioned before, Toyota's not one to, they certainly will trial things and create MOUs with small startups and things like that. But this is Waymo we're talking about. I don't think that they're going to publicly talk about anything unless it's serious.

• 9:10Speaker 01

  There was a hint about this happening on Google's earnings call. Sundar Pichai said that he was considering letting people use Waymos for personal ownership. This is kind of what he hinted at without really getting into the details more. And then this announcement came a couple of days later and we were like, oh, okay. Got it. That's what you're talking about. Tesla's stock took a hit on this announcement. Kirsten, I want to ask you, like, how do you see this kind of speeding up the race to, you know, give people cars that could be autonomous vehicles themselves? Give people these kind of like personal robo taxi future? We've been hearing about this for so long and it still seems like such a far out pie in the sky idea to me.

• 9:49Kirsten Korosec?

  Well, I mean, it's far out-ish. So to be clear, Tesla has been promising, and specifically when I say Tesla has been promising, Elon Musk has been promising, that people are going to be able to, through a wireless software update, that their Model 3, if it had FSD, you know, full self-driving software, would suddenly become quote unquote "level four autonomous vehicle", that then you could use the vehicle and make money off of it by turning it into a robo taxi service. That type of thing is far off and it continues to be promised and it's not happening. Tesla vehicles today have what is considered a level two system. This is like an advanced driving assistance system, but the driver, the human, has to remain engaged.

• 10:34Kirsten Korosec?

  Other companies like GM, GM CEO Mary Barra, have talked about potentially having personal autonomous vehicles with some dates that have slipped here and there. And then you see WAVE, another buzzy startup that has made deals, like specifically with Nissan to put their tech in. But the question is this, what are we talking about when we talk about autonomous vehicle technology? In vehicles, is this a I'm going to go to sleep and I'm not liable for anything that happens while my car drives me around? Or is it some features in the vehicle will be automated, but you're still kind of expected to be paying attention and engaged? And I think it's the latter.

• 11:15Kirsten Korosec?

  The holy grail, I can go to sleep in my vehicle, I think is many, many years away, in part because the liability question. So the robotaxi business model is kind of a safer way for a company to really test driverless tech without getting a consumer involved in like a purchased vehicle.

• 11:33Speaker 01

  Curious, can you talk to us about what Waymo's vision for their role in this ecosystem of autonomous driving vehicles? I mean, in San Francisco, they kind of control the whole operation right now. You go onto a Waymo app, you're kind of in these cars that are very clearly branded as Waymo's. But is that what they always want to do? Or do they really just want to supply the autonomous driving part of this and let someone else handle the rest of it?

• 11:57Kirsten Korosec?

  Yeah, I mean, I think that they eventually want to do what you just said. But I mean, this was a former Google self-driving project that then became a company that really has acted as a research and development program, slowly transitioning into a commercial program. And they've talked about this publicly and also just as I've spoken to other Waymo folks. The intention never was to run the entire operation in all of these cities. But I think that what that business model looks like has continued to evolve. They have talked, for instance, about the potential of licensing their tech to other automakers, just like we mentioned. They have explored a self-driving truck program, which currently doesn't exist.

• 12:39Kirsten Korosec?

  And they have talked about not controlling all the operations around robotaxis. The important thing to note is that they're starting to make moves to not control everything around robotaxis. So specifically their partnership with Uber. So Uber is handling all the network routing, everything's running through the Uber app. And then Waymo, of course, is handling like the tech side of things. But the maintenance and the charging is handled by Uber through a third party. So you're starting to see the robotaxi side evolve as a business. Waymo, I think, is looking for ways to reduce their outflow, like costs and overhead, so that they can maybe actually get close to profitability. I mean, right now they're making revenue, but they're, as far as I know, they're not close to profitability in any way.

• 13:24Anthony Ha?

  You mentioned the liability issues and my sense of the Waymo rollout, it's sort of also just getting approval to run these tests and then run robotaxis on a like city by city or state by state level. Do you think that that's how things are going to continue? That it's going to be this sort of piecemeal regulation and approval? Or do you think like there's actually going to be some sort of national law around this?

• 13:45Kirsten Korosec?

  Way to ask the like light question at the end there, Anthony. But I will say this, the new administration has indicated a desire to do a federal AV framework. They've signaled as such, they've put out a new standing order. But when you look at the language, it doesn't change too much. But we'll see where this goes. So for now, I would say absolutely going state by state. And that's why you're seeing a lot of activity in Texas. So a company called Aurora just had their first like driverless commercial self-driving truck launch that just happened. You see Waymo being active in Austin. You see Tesla being potentially active in Austin, starting in June. You're seeing self-driving truck company Kodiak active in Texas. There's a reason.

• 14:27Kirsten Korosec?

  A lot of talent there, but also not many regulations very much open for business. So I think in the near future, absolutely it'll be piecemeal. It'll be all about regulations and also market fit. Now we have to move on. When we come back from the break, we're going to get into Cluely, which again, as I noted before, Anthony absolutely has lots to say on that. And then Max is going to catch us up on AI Labs gaming their benchmarks.

• 14:58Anthony Ha?

  What moves our world forward? Energy. History has been shaped by it, revolutions won with it, and progress dependent on it. For 190 years, Schneider Electric has been a leading force in this story, creating the technologies that harness energy and transform the competitive landscape. And today, in the dawning era of energy and industrial intelligence, Schneider is helping customers build more efficient, resilient, and sustainable operations across every sector, from data centers to homes, from buildings to industry. Schneider Electric, your energy technology partner. Discover more at se.com.

• 15:36Kirsten Korosec?

  And we're back. Okay, Anthony, I was teeing you up twice now on this show. So Cluely.

• 15:43Anthony Ha?

  Cluely.

• 15:44Kirsten Korosec?

  What is Cluely?

• 15:45Anthony Ha?

  It's a startup. There's a lot to unpack here. The short version of this is that it's a startup that announced some funding, kind of announced the vision, and the vision in short is to use AI to help you cheat on everything. There was actually this promotional video which showed the founder using AI on a date so that he could lie about his age and what he knew about. There's been a lot of reactions to this company, a lot of them, I think, fairly negative, which I think was fairly intentional. I think that their goal was to go viral and be controversial.

• 16:18Kirsten Korosec?

  Controversial, yeah. I mean, hey, it's like the idea that like, there's no bad news. There's no bad publicity, right? But they actually raised money off of this.

• 16:27Anthony Ha?

  They did, and I think a lot of the initial coverage was really focused on this vision. The founders were Columbia students, and in fact, one of the founders used this project called Interview Coder to cheat on, I mean, again, that's the word he would use, like I think in an internship interview with Amazon. And I think that eventually actually led to an investigation at Columbia. The founders actually dropped out of Columbia. So that's kind of a whole thing, which, again, they're highlighting in their story. They're like, we are like, bad boys who dropped out. So it seems like they have actually built, or at least used, tools like this that have achieved something. At the same time, The Verge was like, let's try this out, and I'll use it both in my interview with the Cluely founders, also use in other contexts. And I think it seems like the really big problem

• 17:12Anthony Ha?

  is that this has incredible lag right now. And so if you are trying to like, have a job interview, or if I was trying to like, use it to like, coach me through this podcast recording, there would just be these long silences. And if you were trying to catch me as a cheater, it might be pretty obvious.

• 17:29Kirsten Korosec?

  Ah, Max, what has come out of this, though? Because there's other startups now launching products to catch people?

• 17:40Anthony Ha?

  Yeah.

• 17:41Kirsten Korosec?

  Using AI cheating? So is this the result of Cluely, or was this already happening?

• 17:45Speaker 01

  Yeah, I mean, I think that this was happening more broadly, like the thing that Roy, the co-founder of Cluely, I think he is smart in a certain way to capitalize on this idea where AI is going to help people cheat. I think his idea is that we are not in the same world anymore. For these big tech internships and jobs, you don't need to study for 100 hours to be able to get one of these jobs. You can use an AI tool to like mostly navigate through it. And his software is supposedly undetectable by like the spyware that these companies put on their applicants, computers to see what they're doing. But these other startups have now launched products to be more advanced about catching people using AI cheating tools.

• 18:27Speaker 01

  I think that what he's doing is part of a broader thing that's going on in AI. I do think that his strategy here is kind of the most interesting part, though, where he is just rage baiting people.

• 18:37Kirsten Korosec?

  And here we are.

• 18:38Speaker 01

  We're talking about it on Equity. I mean, tons of people are talking about it. And I think that he's gained like 100,000 followers on X. They actually just announced that Cluely is creating like a house full of 50 interns in San Francisco, and they're paying them all like 50 bucks an hour to work on Cluely. I think it's going to be like a creator house, like a TikTok pipe house sort of thing is what it sounds like. He's clearly very entrepreneurial in a certain way to be able to understand the social media landscape, to like just rage bait people, create all this content around your business. And now it's like the eyes are on him, like there's attention. It's not the worst idea ever, and he's got enough following to raise some money. It is the worst idea ever.

• 19:18Kirsten Korosec?

  No, I'm maybe poking the bear a little bit, but creating a creator house and getting a bunch of interns and getting a bunch of publicity isn't the worst idea. I don't know about the premise of the company. Let's see how it evolves. More broadly speaking, though, I'm very interested to see how other startups are created or other products are created, or that if there's a market for that, to catch the folks using AI or lying about using AI. And I know that you cover a ton about this subject. It's pretty hard to catch someone using like a chat GPT or something, right? Isn't that, it's very difficult, any product out there that actually says it can, it's sort of mixed reviews right now?

• 19:59Speaker 01

  Yeah, there's a couple. For text, Pindrop is one company that has made a business out of catching AI voice tools. They make a business selling to insurance companies to make sure like if someone's calling them, they can verify that it's not an AI voice chatbot. But all of this, it's definitely a business that's being created here to catch all this stuff. But it's all still kind of very nascent and we have to see how it plays out.

• 20:23Anthony Ha?

  Right. I mean, one of the differences is that at least the way they're selling this. And to be fair, they're a very early startup, but also I think there's some sleight of hand going on. The launch video they had has almost nothing to do with the product they have right now, I think. Like, that is so far beyond anything they can do right now. And so it's really important to emphasize that even the Roy Lee has said like, oh yeah, the product we have right now is like a proof of concept, which that's true of most early-stage startups. Because the promises are so big and so dramatic, you're just more aware of that gap. So I think it's worth underlining, the tech isn't there, at least not yet, doesn't seem to be there yet. And then also just that I don't like that, and I also don't like the vision, like that they have this manifesto that like includes this line like,

• 21:05Anthony Ha?

  "Why memorize facts? Write code, research anything when a model can do it in seconds." And I just think that is such a bleak and terrible vision for the future.

• 21:15Speaker 01

  Okay, I think I need to respond to that. I agree with you on a very like broad level, Anthony, but I also think that the media ecosystem that we're in, the venture space that we're in, like seems to reward this kind of behavior of like, set a goal way farther out, especially in AI. He said on a podcast recently, Roy, that he wanted to create an AGI chip that lives in your brain and helps you cheat on like everything in the world.

• 21:42Anthony Ha?

  This guy will say anything. He'll say anything.

• 21:45Speaker 01

  Well, exactly. But like, this is like, look at the top of the AI ecosystem with OpenAI, and they're like, "We're gonna make AGI." And then they raise an ungodly amount of money. And now it's like, at the bottom, that is, I think, what's inspiring these kinds of startups right now. And it's, I agree. Like, I agree with all the reasons you said, like, this is kind of a dangerous thing to be doing when the product is just not there at all. But that's 2025.

• 22:09Kirsten Korosec?

  I will say this. So he has taken a page like so many other founders before from the Elon Musk playbook, which is major bravado, promise a ton, get a lot of attention. And the question is, will it be successful? Because a lot of founders have tried this and has absolutely backfired for them. So there is a trick to being Elon Musk, which is you promise a ton, you deliver on just enough and count on people not going back in history and like comparing like, oh, actually you missed all these other things. That's the trick. So we'll see how that plays out. Can he deliver something? Like, sure, go ahead and promise the brain chip, but can you deliver on the most basic stuff?

• 22:54Kirsten Korosec?

  TBD on that. Max, I do want to talk AI benchmarking, which is actually kind of a controversial subject right now. What exactly is going on in the world of AI and with benchmarking?

• 23:06Speaker 01

  So a study came out earlier this week from researchers at Cohere, which basically showed that this very popular AI benchmark called Chatbot Arena, two chatbots compete against each other and then users vote on which answer they like the best. And then over time, those votes contribute to a score. And then there's a leaderboard of which chatbot people tend to like the most. And this study came out from researchers at Cohere that basically said that the organizers of Chatbot Arena are helping major AI labs game this benchmark. And this is a very complicated study that gets into the nitty gritty of all of it. But basically, one of the major findings of it was that Meta privately tested 27 models before launching their Lama 4 models.

• 23:50Speaker 01

  Meta launched one model and that model did very well on this arena. It jumped to the number two spot on this closely watched AI benchmark. And what the study basically said is that there's a lot of gaming going on behind the scenes to achieve those impressive scores and for AI labs to look like they're ahead of the game. I don't want to get too into the details of this because Elmarina denies a lot of the facts in this. And they say that the analysis of the study is wrong. They say this private testing is actually a good thing for the System and there's a debate to be had there. I think the bigger picture of this story is that there seems to be more and more evidence that AI labs really are putting a lot of energy and resources around gaming these benchmarks and making it seem like they are the most advanced and the cutting edge.

• 24:34Speaker 01

  And I think there's an interesting question around like, is this just a big distraction? Is this the right thing to be focused on right now? Or are there more areas of innovation that we could be using in AI?

• 24:45Kirsten Korosec?

  Well, and I also wonder if everyone is gaming the system, then what's the point of this? Like, you know, it's like, if this actual thing becomes known that all of these companies, because it sounds like it's not just meta that's being accused of gaming the system, but it's also like OpenAI and Google, Amazon, others. If they're all gaming the system, then the benchmarks have no value. So then everyone will just ignore it, right? Is that the endpoint here?

• 25:10Speaker 01

  Yes. I talked to the Cohere's VP of AI Research, Sarah Hooker, and she was, that was exactly her point, was that there is no scientific integrity to this stuff. If all the companies can basically privately figure out their scores in advance and then just make sure that their models are at the top of this before it comes out, like, that's not scientifically based approach to this stuff. And the AI research world is very much founded in science. Like, it is a research field. So people care a lot about this stuff, and I'm curious to see the reaction to this study.

• 25:43Anthony Ha?

  Right, and it seems like this also speaks to this very awkward moment that the AI world is in, and it's something that started as like R and D, started in like the academic world. And OpenAI's structure itself just speaks to this awkwardness. And suddenly there's like crazy amounts of money pouring into it. There is so much at stake. And so the way I feel like I've seen it as a reporter is something like these incredibly arcane benchmarks that I never paid attention to, now having to read stories, edit stories, that I'm just like, what is this? I'm so confused. But like, these things are actually like a big deal to the companies involved.

• 26:17Speaker 01

  Yeah.

• 26:17Kirsten Korosec?

  Yeah, they're validating, now there's money involved.

• 26:20Anthony Ha?

  There's a lot of money involved.

• 26:21Kirsten Korosec?

  It used to be just the science involved. So anytime that a benchmark is then used, whether it was created to be used for this purpose or it evolved for this use of validating a company, which leads directly to them potentially either raising money or being able to charge more, anytime there's money involved, all of a sudden. We need to step back and take a look about the validity of these benchmarks and like how they're being used. And if all of these companies, to my earlier point, are just gaming the system, it starts to really, the value of them becomes diluted. And I'm not saying that that is the case right now, but that is generally what will happen in any industry when it's not really clear whether the benchmarks are even followed or if they have any value.

• 27:03Speaker 01

  There's also something interesting here about LM Arena itself. I mean, this is a test that everyone in the AI industry is super focused on. And they recently announced that they were launching as their own company. And there's a lot of these startups that are trying to evaluate AI models as a business. This is the latest example, but we've seen a lot of examples of how there's a conflict of interest here potentially. How can you be independent when you are also relying on these AI labs for your revenue, for your business? I think it's a tough balance, but we'll see. LM Arena has already agreed to make some changes in response to this study and operate in a little bit more of a balanced way. But they haven't agreed to anything. I think their reaction here will be very telling.

• 27:43Kirsten Korosec?

  Well, we will have to see how that all plays out. For now, though, thanks so much to Max and Anthony for joining and thanks to our audience for listening to another episode of Equity. We'll be back next week. Until then, you can find Equity under the handle at EquityPod on X and Threads.

• 28:03Rebecca Bellan?

  Equity is produced by Teresa Locan Solo with editing by Kel. We'd also like to thank TechCrunch's audience development team. Thank you so much for listening and we'll talk to you next time.