Are AI Benchmarks Telling The Full Story? [SPONSORED] (Andrew Gordon and Nora Petrova - Prolific)

Is a car that wins a Formula 1 race the best choice for your morning commute? Probably not. In this sponsored deep dive with Prolific, we explore why the same logic applies to Artificial Intelligence. While models are currently shattering records on technical exams, they often fail the most important test of all: **the human experience.** Why High Benchmark Scores Don’t Mean Better AI Joining us are **Andrew Gordon** (Staff Researcher in Behavioral Science) and **Nora Petrova** (AI Researcher) from **Prolific**. They reveal the hidden flaws in how we currently rank AI and introduce a more rigorous, "humane" way to measure whether these models are actually helpful, safe, and relatable for real people. --- Key Insights in This Episode: * *The F1 Car Analogy:* Andrew explains why a model that excels at the "Humanities Last Exam" might be a nightmare for daily use. Technical benchmarks often ignore the nuances of human communication and adaptability. * *The "Wild West" of AI Safety:* As users turn to AI for sensitive topics like mental health, Nora highlights the alarming lack of oversight and the "thin veneer" of safety training—citing recent controversial incidents like Grok-3’s "Mecha Hitler." * *Fixing the "Leaderboard Illusion":* The team critiques current popular rankings like Chatbot Arena, discussing how anonymous, unstratified voting can lead to biased results and how companies can "game" the system. * *The Xbox Secret to AI Ranking:* Discover how Prolific uses *TrueSkill*—the same algorithm Microsoft developed for Xbox Live matchmaking—to create a fairer, more statistically sound leaderboard for LLMs. * *The Personality Gap:* Early data from the **Humane Leaderboard** suggests that while AI is getting smarter, it is actually performing *worse* on metrics like personality, culture, and "sycophancy" (the tendency for models to become annoying "people-pleasers"). --- About the HUMAINE Leaderboard Moving beyond simple "A vs. B" testing, the researchers discuss their new framework that samples participants based on *census data* (Age, Ethnicity, Political Alignment). By using a representative sample of the general public rather than just tech enthusiasts, they are building a standard that reflects the values of the real world. *Are we building models for benchmarks, or are we building them for humans? It’s time to change the scoreboard.* Rescript link: https://app.rescript.info/public/share/IDqwjY9Q43S22qSgL5EkWGFymJwZ3SVxvrfpgHZLXQc --- TIMESTAMPS: 00:00:00 Introduction & The Benchmarking Problem 00:01:58 The Fractured State of AI Evaluation 00:03:54 AI Safety & Interpretability 00:05:45 Bias in Chatbot Arena 00:06:45 Prolific's Three Pillars Approach 00:09:01 TrueSkill Ranking & Efficient Sampling 00:12:04 Census-Based Representative Sampling 00:13:00 Key Findings: Culture, Personality & Sycophancy --- REFERENCES: Paper: [00:00:15] MMLU https://arxiv.org/abs/2009.03300 [00:05:10] Constitutional AI https://arxiv.org/abs/2212.08073 [00:06:45] The Leaderboard Illusion https://arxiv.org/abs/2504.20879 [00:09:41] HUMAINE Framework Paper https://huggingface.co/blog/ProlificAI/humaine-framework Company: [00:00:30] Prolific https://www.prolific.com [00:01:45] Chatbot Arena https://lmarena.ai/ Person: [00:00:35] Andrew Gordon https://www.linkedin.com/in/andrew-gordon-03879919a/ [00:00:45] Nora Petrova https://www.linkedin.com/in/nora-petrova/ Event: Algorithm: [00:09:01] Microsoft TrueSkill https://www.microsoft.com/en-us/research/project/trueskill-ranking-system/ Leaderboard: [00:09:21] Prolific HUMAINE Leaderboard https://www.prolific.com/humaine [00:09:31] HUMAINE HuggingFace Space https://huggingface.co/spaces/ProlificAI/humaine-leaderboard [00:10:21] Prolific AI Leaderboard Portal https://www.prolific.com/leaderboard Dataset: [00:09:51] Prolific Social Reasoning RLHF Dataset https://huggingface.co/datasets/ProlificAI/social-reasoning-rlhf Organization: [00:10:31] MLCommons https://mlcommons.org/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Formula One cars are the absolute pinnacle of engineering, right? Everything is perfect, huge top speed. If you used it as your daily commuting car, you'd have an absolute nightmare, right? And I think the same can be said for these models, right? A model that is incredibly good on humanity's last exam or MMLU might be absolute nightmare to use day to day. Most reporting on benchmarks is done on technical benchmarks these days, right? And that is where you get the model. You you give it a set of evaluations, maybe on one theme or maybe on an exam and then you get a score and humans aren't really involved in that loop. Hey, my name is Andrew Gordon. I'm a staff researcher in behavioural science at Prolific. So I work on the sciences team tackling questions related to humans in research, specifically online research.

• 0:45Speaker 01

  My name is Nora Petrova. I'm an AI researcher at Prolific. I'm tackling questions around how do we include humans, development of AI models and evaluation of AI models? How do we align them to human values and how do we fully understand what they're capable of?

• 0:59Speaker 00

  How helpful do people find models? What's the communication like? How adaptive do they find it? What do they think of the model's personality? And that, when you get ratings on those kind of factors, what you actually get is an actionable set of results that say, okay, your model is struggling with trust or your model is struggling with personality. And we had a first go at this with what's called the Prolific User Experience Leaderboard. That was a proof of concept with 500 participants from the US, a representative set of participants where they evaluated a single model at a time and gave their feedback in a Likert scale format, you know, how helpful did you find the model, one to seven and so on. We've actually moved now and taken the learnings from that initial leaderboard and built it into what we're calling Humane, which is our main leaderboard.

• 1:45Speaker 00

  And that uses a much, a similar approach to Chaparral Arena in the sense that we have comparative battles between between models, which actually allows us to much more clearly differentiate which model is performing better.

• 1:58Tim Scarfe?

  What if we could have a fairer approach where we actually diversely sampled and stratified folks based on how old they are and where they live and what their values are? What would be a fairer approach to understand the behaviour of models and to do evaluation? So this is what Andrew and Nora have been doing at Prolific. How do we know whether these models are actually good for the humans that are using them? How can we make evaluation metrics which are fairer? This is Andrew and Nora.

• 2:26Speaker 00

  But the problem is at the moment, I mean, the the field of kind of evaluations and benchmarking these models is incredibly nascent, right? It's only been around as long as L L M's have been around in the last couple of years. And because of that, it's a kind of like a fractured field, right? There's no, there's no standard playing field for how these labs report data on on on benchmarking. You know, some may emphasize that with Grok 4 recently, we saw huge amount of emphasis on humanities last exam, right? And less so on other benchmarks. And some models will come out without any benchmarking data at all, right? And without some kind of, essentially, there's a lot of heterogeneity in how these labs report the results. And what it, what it leads to for

• 3:05Speaker 00

  me is a situation where I think we're at risk of struggling to actually compare the models on any even playing field. But there's of course, bigger questions as well about, you know, models are often lauded for doing the high score in humanities last exam. So we know from a technical perspective, the model has advanced above its rivals. But for me, and I guess my kind of core argument in this space is, if you just rely on those technical metrics, you miss half the point, right? These models are designed for humans to use. At the end of the day, most of the users are humans. And simple performance on these exams doesn't necessarily correlate to a good user experience. All the frontier labs really need to start kind of having a human

• 3:49Speaker 00

  preference lead towards a little more front of mind alongside all the technical metrics.

• 3:54Speaker 01

  People are increasingly using these models for very sensitive topics and questions for mental health, for how they should navigate problems in their lives. And there is no oversight on that. And in any other area where these topics are discussed, there is a lot of regulation and a lot of kind of ethical conduct built into it. Whereas here is kind of the Wild West at the moment. And some companies are taking it more seriously than others and trying to study the ways in which humans are using the models for for more personal topics and and problems. And we've seen some pretty stark examples recently with with Grok 3 and Mecha Hitler. And it does raise questions about how how thin of a veneer is the safety training on top of some of these models?

• 4:39Speaker 00

  Well, there is no leaderboard for safety. Right, like, there's no metric, like, we don't grade LLMs by how safe they are. In fact, it's not really even in the question, apart from some researchers. So, I mean, I would argue that should be just as important as how fast or smart the model is. You know, how safe is it for people to use?

• 5:00Speaker 01

  There's been a lot of interesting research coming from Anthropic in that direction with regards to safety, with regards to alignments of the models and using constitutional AI and various approaches that they've explored. And also around mechanistic interpretability, just peering kind of behind the curtains of the models and understanding how an input produces a certain output, which can feature its concepts, which circuits get activated along the way and kind of tracing the thoughts essentially of these models and trying to isolate where potential problems may emerge. So the work of this kind is very important in raising the confidence that these models will be able to handle novel situations in safe ways.

• 5:45Speaker 00

  I think the one other thing I'd say is, given that Chatbot Arena is the number one and frankly, pretty much the only human preference leaderboard out there for LLMs, it's really important that we actually understand what's going on behind the scenes. So obviously, you know, Chatbot Arena is entirely open source. People go in, they put in a prompt, they get a response from two different models. They then say which one is better. And that paper found that actually what was happening in the background is that some companies are getting a lot access to a lot more private testing in the background than others. For instance, before Lama 4 launched, we saw Meta release 27 models on the arena. But of course, only one was actually reported in the end, which obviously undermines the integrity of the arena.

• 6:26Speaker 00

  Because the more comparisons you have for your model, the more access to prompts you have, the more data you have to refine a better model that's better at the arena. And it adds an element of bias into into the data, which is very, very hard to get around. There are other issues that were called out and issues that we've seen ourselves, which we think kind of dictates the need for a more rigorous and and methodologically sound approach to doing these kind of human preference data sets. Amongst us, I think we had a pretty good idea. Beyond beyond the criticisms in the leaderboard illusion paper, we think that it actually that paper didn't really touch on some of the other things we we think should be cared about when you're doing human preference valuation. I think for me, there's three big areas, I think, where where we've sought to improve. First of all, as you mentioned, sample.

• 7:10Speaker 00

  So obviously, the sample for the chatbot arena is anybody, right? We don't know anything about them. We don't collect any demographic data. So they are just people going there anonymously, prompting the models and giving their preference data. Now, obviously, that's great. You get a huge amount of data, which is fantastic, but you know nothing about the people giving the data, which is fairly suboptimal. Then in terms of specificity, for anybody that's used chatbot arena, all you're doing is saying, I like this response more or I like this response more, right? In the real world, that kind of data is is useless, in a sense, right? It it gives you a really nice way to make a nice leaderboard of AI models, but it tells the companies nothing about why that preference has been has been given.

• 7:54Speaker 00

  So in in our approach, we sought to mitigate that by actually splitting preference down into its constituent parts. So things like, how helpful do people find models? What's the communication like? How adaptive do they find it? What do they think of the model's personality? And that, when you get ratings on those kind of factors, what you actually get is an actionable set of results that say, okay, your model is struggling with trust or your model is struggling with personality. That's where you need to be focusing to really actually build a model that is good for real users in the real world. But there's no QA in the sense that, okay, I could go in and I could just say hello or I could say absolutely nothing or I could have a multi-turn conversation and completely wonder from, you know, how big is the sun to how long is a snake?

• 8:36Speaker 00

  You know, like just topic wandering, which I don't think is a really good nuance for your models. So we built in to our structure where participants come in and they have multi-step conversations with models. We built in QA that actually says, you know, if if you put lower low effort into your question or you start wondering, we're going to penalize you through three of those and you're out. So. Those are the kind of principles, I guess, we built the leaderboard around.

• 9:01Speaker 01

  I would just touch upon the methodology that we've used, which is TrueSkill. It's a framework, if you will, developed by Microsoft for estimating skill levels of players on Xbox Live. So they take into account things like randomness in games, changing skill levels across time, whether someone is having kind of a fluky win streak versus a seasoned player that consistently performs well. So all of these things that we thought would be good to take into account. And it's a very flexible system that estimates probabilities with Bayesian distributions, with kind of a mean and a variance that gets narrower and narrower over time

• 9:42Speaker 01

  as the system kind of learns about the outcome of these battles or these comparisons. Most importantly, it's based on information gain. So the way we pick the next pair that should occur in the tournament is based on how much we will learn from these models going head to head. How much information are they giving us? How much are they reducing the uncertainty? And we kind of order the queue of pairs according to that. And that gets us to a place of minimized uncertainty as fast as possible, as fast as we can. It's a really flexible approach. We can run separate tournaments, like we've done with our demographic groups. We have around 20 demographic groups and we've run separate tournaments for them.

• 10:24Speaker 01

  And we can consolidate the findings for each tournament to obtain kind of an overall leaderboard that is much less uncertain than any of the individual tournaments or leaderboards that we can produce from any of the demographic groups. So it really allows us to slice and dice data in any way we want. And we can easily add more demographic groups, more models over time. We're, yeah, developing in the open and welcoming feedback.

• 10:51Speaker 00

  I think one of the things they pointed out in the leaderboard illusion paper was that actually some models are sampled considerably higher than other models. And I believe that the folks behind Chatbot Arena said that's because people come to the arena to play with the latest models, right? Which is all well and good. People want to play with the state of the art, right? Which is great. But it doesn't lead to an efficient sampling method. It essentially means that some models get a lot more battles than others. They therefore get a lot more data. They therefore get a lot better in the arena. And there's a pretty strong relationship between the number of battles and the place on the leaderboard. We only ever do battles based on the need from the data. So if the uncertainty is high for a specific model against another specific model, we conduct a battle for that to lower that uncertainty.

• 11:36Speaker 00

  So it's all driven by the data. It's very computationally sound because we don't actually make any more comparisons than we need to. And it allows us to really get to a point where models are strongly differentiated based on uncertainty.

• 11:48Speaker 01

  If we have a certain goal with regards to uncertainty, in order to fully differentiate the models at the confidence interval that we're interested in, we can just conduct more battles until we get there. The control is in our hands. In essence, we just need to recruit more participants in order to get to that level of certainty.

• 12:04Speaker 00

  The way we've sampled for this study, we're obviously using our own participants from the prolific platform. But we've sampled effectively in, based on the census data that we have for both the US and the UK. So a long-term vision of this would obviously be a more global product. But at the moment, what we do is we stratify our sample. Our participants who are giving us this feedback by demographics like their age, their ethnicity, their political alignment. And we have an awful lot of data from censuses that tells us, you know, each country is made up of this certain proportion of these demographics, which essentially allows us to say that when we've amalgamated all these findings and we find that leading model,

• 12:44Speaker 00

  we can very confidently say that that model is preferred by as representative a set of the general public as we can possibly get. So hopefully in that sense, it's a lot more related to like the real world preferences of people in the world, rather than a very potentially skewed and biased subset that might be responding to the chatbot arena. Because we we ran our first one as an MVP, a proof of concept, that was a lot more about kind of proving that we can do this in a rigorous and methodologically sound way. When we actually ran that, we only ran it with 500 participants. It gave us a lot of insights about how we build Humane, which is our leaderboard that we're working on at the moment. Now, that leaderboard is actually running as we speak in the background.

• 13:29Speaker 00

  We're still having battles, so we expect to be able to have more data from that. But what I can say about the the first, the first round that we did, models tended to perform across the board of the six models we tested, which were leading models at the time, they performed a lot worse on personality metrics and background and culture metrics as opposed to things like helpfulness, communication, adaptiveness. What that really signals is that there's there's some, I guess, more subjective aspect of these models, which people are less impressed by, potentially just they were doing tasks that don't elicit a personality in the model, or they don't elicit the model talking about background and culture. Also, the model doesn't know their background and culture, so it's very hard to align with them.

• 14:11Speaker 00

  But the other possibility is that potentially models are just not very good at that. And that would be potentially an effect of the data they've been trained on, right? Because we know very little, I mean, obviously, models are trained on the entire internet, but when you train a model on the entire internet, do you get a personality that really represents what people want, right? And from this testing, we found that generally people were less impressed with model personality or its ability to have an understanding of their background and culture than with more kind of, I guess, objective measures.

• 14:45Speaker 01

  And obviously, a lot of these models have undergone extensive fine tuning to tailor their personalities or tailor how they approach answering questions that are different across the different companies. But we've observed recently that there has been an increase in sick of fancy or this kind of people pleasing behavior of models, and people generally don't seem to like it. One thing that this, the results of this experiments or and the leaderboard data sets will allow us to answer is what is the correlation? Between telltale signs of second fancy and a down vote in the personality metric. Does that influence people's decisions on which model they prefer?

• 15:30Speaker 01

  We can perform various types of post-processing and analysis of the data to identify the levels of second fancy that's observed in the data set and to try to identify interesting relationships between the feedback that people gave, kind of more model-driven, kind of Elam as a judge-oriented analysis of the conversations and the kind of classification of various patterns and what the models exhibit. So it's quite interesting to see what we'll find.