Pod Insights Summation with Auren Hoffman Astroten Founder Richard Shotton - why your A/B tests fail and how behavioral science can fix them About this episode Richard Shotton is the founder of Astroten and author of several bestselling books on marketing, including The Choice Factory, The Illusion of Choice, and most recently Hacking the Human Mind, co-authored with MichaelAaron Flicker. He specializes in applying behavioral science to marketing and has worked with brands like Google, Meta, BrewDog, and Barclays. In this episode of World of DaaS, Richard and Auren discuss: Why A/B testing often steers product wrongSocial proof and scarcity in marke...
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:02 Auren Hoffman? Hello fellow data nerds. Welcome to World of DAS. I'm your host, Orin Hoffman, CEO of Incubate and GP of Flex Capital. Discover more episodes, get weekly data as a service news, original content, articles on data and more at worldofdas.com. That's worldofdas, D-A-A-S, dot com. Hello fellow data nerds. My guest today is Richard Shotton. Richard is the founder of Astro10 and author of two bestselling books, The Choice Factory and The Illusion of Choice. And he has a new book coming out now called Hacking the Human Mind. Richard, welcome to World of DAS.
0:39 Speaker 01 Well, thank you for having me. Glad to be here.
0:42 Auren Hoffman? Very excited. Now, a lot of companies run A/B tests constantly, but you argue that many tests are just the wrong things entirely. Where's the blind spots as companies approach these type of experimentation?
0:55 Speaker 01 A/B testing is a huge improvement on directly questioning customers. I think that's the first thing. It's certainly a big improvement. If there's a theme of behavioral science, which is my forte, it's that what people say influences them and what actually influences them are very different things. So there's an amazing psychologist at University of Virginia called Timothy Wilson. And he has this lovely phrase. He says, "We are strangers to ourselves." And what he means by that is people don't really know why they do the things they do. So if you're like most businesses and you question people, send them a survey, put them in a focus group, they'll give you long, logical, sensible reason about why they buy your products. But most of the time, those
1:40 Speaker 01 reasons are very misleading. We don't have introspective insights. What we say is often a really misleading guide to what we actually do.
1:49 Auren Hoffman? Some things are testable, but some things are like hard to test. If you're building a product and it takes two years to build this thing, it's a little bit hard to A/B test that thing. If it's a button on a website and it's red or blue, that's a little bit easier.
2:01 Speaker 01 Absolutely. But I would argue when people are thinking about tests, Especially in world of apps and digital, and many of those companies, as you say, have this culture of A B testing. But what they tend to do is test the final thing. Should we change the colour of the text or the size of the button or the shape of the button? Now, that's incrementally beneficial. But what I would say is you should take that same A B testing mentality and do it much earlier in the process. So what you really want to do is take a behavioural science principle, let's say, social proof, the idea that we are deeply influenced by what others do. You want to find out whether this idea that
2:42 Speaker 01 has been proven in lots of different categories and markets and places, you want to find out if it works for your particular challenge. And that's where you should use A B testing. And then, once you know this broad insight is powerful, then you can start iterating on how best to use it. Yes, A B testing, but bring it earlier in the process.
3:04 Auren Hoffman? Are there particular types of things like social proof that companies don't think of enough? Every time I go into a company, almost always they have like a deficit of social proof. That seems like an easy one for a lot of companies to fix. But are there other ones where they just don't do enough?
3:21 Speaker 01 There's some biases like social proof that I think are partially applied, but not enough. Of all the experiments that behavioural scientists have looked into, probably the one with the most robust data was social proof. So it's this argument that if you make a course of behaviour look like it's popular, you'll make that course of behaviour more appealing and more people are likely to do it. Now, many listeners might be rolling their eyes and thinking, well, we see this all the time. You go to a website and it says 5,000 customer reviews or 1 million happy customers. And that's true. Those are examples of social proof, but they're pretty basic. If people are applying that basic level, you can always push it further. So for example, people like Robert Cialdini at Arizona State
4:06 Speaker 01 University have shown that we're not equally influenced by other people. We are most influenced by people like ourselves. I'm a Londoner, Briton. I'm going to be more influenced by knowing what other Londoners do than what other Brits do. So rather than saying you have a million customers, what you should be doing is identifying facts about the audience, where they live, type of business they own, and then tailoring the message of popularity accordingly. So social proof is one of those experiments where it's applied in a basics degree, but there's so much more opportunity. You very rarely see that level of tailoring.
4:45 Auren Hoffman? I saw a bunch of things to get people to vote. And there are a bunch of things like 60% of people vote, that helps people. But if it was both your neighbours voted already, really you're going to vote.
4:57 Speaker 01 It's fascinating, some of the original studies into tailoring this social proof message, it is really trivial linkages with the audience. So knowing your neighbours, most people on your street are going, that would be, yeah, absolutely super powerful. You could apply these principles quite subtly. There's a lovely study by Peterson at the University of Texas, gets a giant sample, I think it's over a thousand people, and he randomises them into two kind of e-commerce journeys. So some people are shopping and they see that an item they want to buy isn't there. And then after they've done their e-commerce journey, they're questioned by Peterson and he finds out how irritated they are.
5:37 Speaker 01 Now, sometimes those people see the product that isn't there labelled as unavailable or out of stock. Other times they see it labelled as sold out. When it comes to irritation scores, people are much, much more irritated if it's labelled out of stock or unavailable than sold out. Now, according to Peterson, this is about social proof. If you say something's unavailable or out of stock, what you're drawing attention to as a product is that you've been inept, you haven't managed your supply chain, you've had logistical problems. If you say it's sold out, completely different set of associations. You're emphasising loads of people wanted this product and you were overwhelmed by demand.
6:18 Speaker 01 Yes, people apply social proof in this basic way, but there's far more sophisticated ways you could apply.
6:24 Auren Hoffman? Often when I'm searching on internet, I'll see something like only two more available at this price. And I'm like, oh, I gotta buy this now before it goes away.
6:32 Speaker 01 It's very closely related to social proof, but psychologists would say it's scarcity. So if you emphasise there isn't much left, exactly the same product will become more appealing. That is a real powerful bias. When psychologists try and rank the relative impacts commercially of some of these experiments, scarcity along with social proof, they always come out towards the top. So one of my favourite applications, and I think retailers in America are much better at doing this than in Britain. We don't really do it much in Britain. Let's say you go in either to a digital site, it could be a supermarket, and a product is discounted.
7:13 Speaker 01 What you should do to apply scarcity is say to people, maximum number of products you can buy is two per transaction.
7:20 Auren Hoffman? I see that all the time. You're not allowed to buy more than four. And I'm like, I wasn't even thinking of buying one, but now I'll buy four.
7:27 Speaker 01 Exactly, because people are making these snap decisions. They're inferring things from the body language of the promotion.
7:34 Auren Hoffman? I better get this canned soup now. You never know, and I might need canned soup.
7:39 Speaker 01 Who knows if they'll keep on making smart decisions. It kind of suggests that the retailer either thinks this thing's going to sell out at such a low price, that implies social proof, or it implies that the retailer is losing money at such a good discount.
7:55 Auren Hoffman? Because back in the day, when you had a super hot item, like the Beanie Babies in the nineties, you could only buy one. You could go in the store, wait in line, and you could only buy one. Or if you're buying Taylor Swift tickets, you can maybe only buy two of them. So there are these things that are for real scarce. It's easy to fool us when there's things that are pseudo scarce.
8:14 Speaker 01 That's a really nice way of putting it. There are all these biases, there's an element of truth to them, but sometimes they become so wed in our mind that the consumer over-applies them. Often, a very valuable item will be scarce, but we use this rule of thumb that scarcity was high value, even in situations where it's not applicable.
8:34 Auren Hoffman? What are some other behavioural biases that influence our purchasing decisions?
8:39 Speaker 01 One of them I thought might be interesting to your listeners is one called the illusion of effort. So the original study was run in America, Andrea Morales at the University of Southern California, back in two thousand and five. She recruits a group of people who are in the market for an apartment, trying to buy a house. And she shows them ten apartments that meet their requirements. Now, everyone in the study gets exactly the same list of houses. And then they are asked to rate the caliber, the quality of the real estate agent. Now, the twist in the experiment, and there's always a twist in these experiments, is that sometimes people are told the real estate agent took one hour to generate the list,
9:23 Speaker 01 and they used a computer. That's the low effort condition. Other people are told the real estate agent took nine hours and they generated this list manually. Now, when people rate the real estate agent, you see a very clear difference. Scores are all out of a hundred. Low effort group, they rate the service provider at fifty out of a hundred. High effort group rate the service provider at sixty-eight out of a hundred. So even though people are getting exactly the same product, there is this thirty-six percent variance in rate levels.
9:56 Auren Hoffman? In the tech business, I think it would be the opposite. So I think if we saw someone who spent nine hours and you got like some random list, you're like, what the heck? Even one hour to get me a list of Ted Diggs, that seems like a lot of time. I'd be like, this person's a complete dunce. I don't want to work with somebody like that who took nine hours to get me like ten items here. They're clearly stupid. I want to work with a person who takes nine seconds to get rid of it.
10:21 Speaker 01 If you ask people, any customer, if you ask them directly, they're never going to say, oh, I wish my real estate agent took a little bit longer to produce this stuff. But this is when claim data becomes a little bit problematic. Because when you run these test and control experiments, you see this happening again and again. And I think it's this rule of thumb that generally in life, the more effort someone puts into something, the longer they take, the higher quality is. There's this argument from Daniel Kahneman. Won the Nobel Prize back in 2002. He says, when people face complex questions, what they tend to do, they don't even realise it, but what they tend to do is replace the complex question with a simpler one that gives them an almost as good answer.
11:04 Speaker 01 And when you are judging the quality of a real estate agent, that is a complex question. So people think to themselves, well, can't deal with that. A really simple thing is how much effort did they take? So we use effort as a proxy for quality.
11:18 Auren Hoffman? I still think in our industry, it's opposite where if there's a high effort, that means lower quality in our industry, but I could be wrong. When I see an engineer and they're like, this took me 32 hours to build this thing. And then I see another engineer took me like one minute. I clearly think the one minute engineer is way better than the 32 hour one.
11:38 Speaker 01 But you could make that same argument about an estate agent. If I thought an estate agent could do this same product in one hour, nine hours. But I think the customer's looking at it a slightly different way. They're thinking, what's likely to have the better quality? The thing that people put effort into or the things that they didn't. There's always a gap between an experiment and a different situation. But with AI, I think you're starting to see something quite interesting. The number of experiments on AI from behavioural scientists are a bit smaller because it's a more recent invention. But there are some, there's a great academic at a VR university in Amsterdam called Kobe Millet. And in 2023, he does a lovely simple test, shows people a poster and he changes the labelling.
12:21 Speaker 01 So sometimes this poster of a skull is referred to as being created by an AI robot. Other times it's referred to as being hand drawn. So everyone sees the same poster, the labelling changes. Now, when Millet asks people about the artistic merit, the creativity, their purchase intent, you see this wild swing.
12:41 Auren Hoffman? You want the handcrafted leather bound thing rather than the machine made. Even the machine made one might be like way higher quality. You just presume the handcrafted one is better.
12:53 Speaker 01 He attributes this back to the illusion of effort. We think AI equals low effort. Our experience of chat GPT as consumers is you put in a prompt and 10 seconds later, it spat out a blog post. I would say it would be ridiculous of me to say, don't use AI because people don't want things to speed it up. Of course, you can't do that. But what you've got to be aware of is if your product is being fired out more and more quickly, there is a danger the same product is rated as low quality. So what you need to do is shift people's focus from the speed of delivery to how long it may be took to set up the protocols and processes. So even though I can generate these answers that you're talking about really, really
13:35 Speaker 01 quickly, it's because I've got 15 years of experience or 25 years of experience or we've invested a million pounds. And I think it's that subtle shift that all of us could use.
13:45 Auren Hoffman? There are all these anecdotes of people, a lawyer or something like that, billing somebody a million dollars for an hour of work. And they're like, that just took you an hour. It's like, no, actually that took me 30 years to get you this one million dollar thing.
13:59 Speaker 01 Exactly. And it's interesting. It comes up in all sorts of different categories. I've heard it attributed Picasso, I think Whistler, there's a Paula Scher one who was a graphic designer. So always the same thing. Yes, it took me 30 seconds to deliver, but it took me 34 years to learn how to do it in 30 seconds. It's exactly that principle.
14:18 Auren Hoffman? How do you think B two B marketing feels different from consumer marketing?
14:22 Speaker 01 One of the big challenges I always face is people say, okay, yeah, this stuff might work when it's cans of Coke and bags of crisps, but it's not going to work when it's really big decisions, either massive consumer decisions like buying a car or big professional decisions. And interestingly, when behavioral economics, which I would say is a subset of behavioral science, when that first came to the fore seventies, eighties, there was an English economist called Ken Binmore, who was very dismissive. He said, oh, all these biases, they're not going to affect really important purchases. Now, if you're buying a million pound product or you're buying a car, you've got a vested interest to think things through very clearly. And Richard Thaler, who went on to win the Nobel Prize in 2017, he responded and said,
15:07 Speaker 01 look, Ken, you've got this completely the wrong way around. When people are buying very small purchases, they do it so regularly, they know their preferences. When you buy very expensive products, car or a house or engagement ring or many business purchases, you do it so infrequently that you know that you don't know. And it's in these situations of uncertainty where we rely more and more on biases. The theoretical argument from failure is business to business will be even more affected by these biases. All very well and good that he makes that theoretical argument, but the experimental evidence backs him up.
15:47 Speaker 01 So we talked about social proof, lots of evidence that works on consumers, but it works on professionals as well. There's a 2018 study from the Australian government. So they had a very serious challenge. Australian doctors were over prescribing antibiotics. So they send out 6,600 letters trying to persuade doctors to give out fewer antibiotics. And sometimes they just try and educate the doctor. Here are all the reasons why over prescription is so bad. And it makes barely any difference on prescription rates. They go down by 3.2%. Other doctors, though, they get that same educational letter, but they also have a social proof line at the top.
16:29 Auren Hoffman? You prescribe more than 90% of doctors or something like that.
16:33 Speaker 01 Exactly. And then they tailored it to the city. So yeah, 90% other doctors in Sydney, that reduced prescription rates by 9.3%. It's a far more effective. You would think many people would argue, well, doctors, they are logical decision makers. These are life and death decisions. Surely they will just weigh up the facts. But even this group, even they are deeply influenced by social proof.
16:55 Auren Hoffman? Now, what about where there's like a double decision being made? Let's say you're a CFO, you're rarely looking to go to a new company. And then your company, you're rarely hiring a CFO. And there's a double decision that needs to be made. How do all these biases affect when there's, it's not just me wanting to buy a product and the product trying to sell to me. They'll sell to anybody. It's more like a dual rare decision being made.
17:19 Speaker 01 Both parties can harness these biases. They're almost a war of who does it best. So if you're one of 10 candidates being interviewed for a very rare position, your way of making yourself appealing is to use these biases more effectively, more creatively and more laterally than your opponent in this situation.
17:40 Auren Hoffman? What about in the dating market? Again, that's like a double decision being made for marriage or something like that. Marriage just tend to work out pretty well in general. How do those decisions get made?
17:52 Speaker 01 Dating's a fascinating one, and if people have got a deep interest in that, I would strongly recommend a book. And often the books have different names in England and America. So in England, it's called Dataclism. But the thing that will be consistent across markets is it's by Christian Rudder, who was the founder and chief technology officer at OkCupid. And he started by writing this blog. And I think the subtitle was how people behave when they think no one's watching. And he analyzes all sorts of interesting quirks of behavior. And the main thing he shows is that what people say is a very, very bad guide to what they actually think.
18:35 Speaker 01 On the dating sites, there's an awful lot of lying. So what he shows, for example, is you can look at American heights for men, and it's essentially a bell curve.
18:45 Auren Hoffman? Five eight, five nine is probably, yeah.
18:47 Speaker 01 Yeah, okay, let's say five eight, five nine is where it should be. What he shows is on OkCupid, the heights of American males mirrors the bell curve almost perfectly, but it shifted two inches. The average height is now five eleven.
19:01 Auren Hoffman? Shifted two inches. Yeah, okay. It's a sleigh line. It sounds like you're five eight, you're five ten or so. And then everyone discounts that. Everyone kind of knows, okay, well, if this guy says he's five ten, he's probably five nine or five eight.
19:13 Speaker 01 Yeah, exactly.
19:15 Auren Hoffman? If he says he's five ten, he's probably not five three. He's definitely not taller than five ten.
19:20 Speaker 01 I think you're absolutely right. There's people are lying, but only enough so they've got plausible deniability. And then you get this paradox of if you actually tell the truth, you aren't properly communicating your height. Because if you're five foot nine and you say you're five foot nine, people will think you're five foot seven. So a truth becomes a misleading answer.
19:39 Auren Hoffman? Are you optimistic for the first date or the second date? Because you're optimistic for the first date, yeah, you might want to lie. But if you're optimistic for the second date, then they're going to be pleasantly surprised when they see you. They'll be like, oh, great, he's even better looking than I thought. This is fantastic.
19:54 Speaker 01 I really like your angle, because one of my favorite experiments is one that I think could be applied. More by marketers, which is my field, but definitely more by daters. So there's an idea called the pratfall effect. We don't use this term in UK, but I think in the 1960s in America, it meant a small blunder. So what happens in the original study? There's a guy called Elliot Aronson, who's at Harvard. He gets his colleague to take part in a quiz. He then gives his colleague all the answers to the quiz questions. So the guy, absolute steamroller is the opposition, looks like a genius, wins this quiz by miles, gets 92% of the questions right. But then as the quiz is finishing, he makes this pratfall, he makes a small blunder.
20:38 Speaker 01 And he spills a cup of coffee down himself. Now, Aronson records this and he plays an audio version of this clip to listeners. But sometimes the listeners hear the entire incident, great performance and mistake. Other times they just hear the great performance. So the mistake is airbrushed out, edited out. And what he finds is that people are significantly more, we're talking 45, 55% more likely to find the contestant appealing. If they hear the mistake and the great performance. So he calls this the pratfall effect. So the argument I think would be, if you're thinking of using this commercially, most advertisers brag. And actually, most people on dating sites brag.
21:23 Speaker 01 If you admit any type of flaw, you will stand out.
21:28 Auren Hoffman? This is the Avis word number two type of thing?
21:31 Speaker 01 Exactly. It's exactly that. It's the Guinness, good things come to those who wait.
21:35 Auren Hoffman? Heinz ketchup.
21:36 Speaker 01 Yeah, yeah, really slow. The other thing it does is I think makes you feel a little bit more human. It makes your other claims more believable because admitting a flaw has tangibly proved your honesty.
21:51 Auren Hoffman? World of DAS isn't just a podcast. World of DAS membership is a private invite only community for founders and executives of important data and AI businesses. It's a place for off the record, high trust conversations focused on the biggest questions data leaders are facing. The community is about peer learning. Curated intros and access to private events. Everyone in the group is building or leading something meaningful in data. If that sounds like you, apply to join at world of DAS dot com. That's world of D A A S dot com.
22:32 Auren Hoffman? We know everyone and every product has flaws. If you don't know what those flaws are, you're always like, ah, something. If they shine a lantern on their flaw, then it's like, oh, great. I can live with that. He's a smoker. Okay, I can live with the fact that he's a smoker because he's got all these other great things about him or something.
22:50 Speaker 01 I think you're absolutely right. We all know every product person has a flaw. And if we're uncertain about where it lies, we might assume that it's a really important thing. If you can admit something, it can't be a humble brag. It's got to be meaningful enough, but not too important. And then the really clever bit is if you admit the right flaw, you can emphasize a mirror strength. So if you're a product and you say, we're really expensive. What people assume is, well, if it's really expensive, they're probably going to be high quality. Often humans think that weaknesses and strengths are two sides of the same coin. If you pick the right flaw to admit, you can then emphasize a core strength.
23:34 Speaker 01 That's what Avis did in your example.
23:37 Auren Hoffman? I work too much, which might mean I'm rich.
23:39 Speaker 01 So I think what you've got to do is think, what is the genuine thing I want to land? And then is there a flaw that I can admit that might emphasize that? My favorite example, are you familiar with Buckley's?
23:50 Auren Hoffman? No.
23:51 Speaker 01 It's a cough syrup in Canada. Back in 1990, tiny little brand, 13th or so in the market. No one was really buying it. They then changed their creative direction and the new line they have is it tastes awful and it works. And the ads are very comic and for 35 years they have run that line. They emphasize how foul this thing tastes because they know people will assume if it's that bad tasting, it's going to be really potent. They shot some number one in market share.
24:21 Auren Hoffman? That makes sense. I would think something that tastes awful would be better for me.
24:25 Speaker 01 Exactly. I mean, same liquid as it was six months ago, but drawing attention to it. Yeah, emphasize that potency.
24:31 Auren Hoffman? I think politicians do that too sometimes where they'll shine a lantern on their problem. And then that helps the other stuff or it humanizes them in some sort of way as well.
24:42 Speaker 01 That's a really interesting one about politicians. I mean, I wonder in Britain, we've had stagnant economy, barely grown since the financial crash. What's that 17 years? So people are increasingly angry with mainstream politicians. So one of the most successful tactics, Boris Johnson did it and became conservative leader and then prime minister. Nigel Farage is doing it currently, topping the polls for his Reform Party. What they do is they make themselves very, very different from other politicians. They know that people will say, I'm angry with cookie cutter standard politicians. This person, because they drink loads or they make jokes or are a bit scruffy. They've defined themselves as being not politicians and therefore their popularity shoots up for a while.
25:26 Speaker 01 I think you're right. It does work in politics.
25:29 Auren Hoffman? Price is kind of an interesting issue. There's a price quality thing. How does one think about price when they're trying to sell their product?
25:38 Speaker 01 There's a hell of a lot of research on price. You mentioned price quality. Absolutely true. There's a Stanford psychologist called Baba Shiv. That's a lovely study. Serves people five different bottles of wine and each bottle has a prominent price label. But there are only four different liquids in the wine. One of the wines is repeated. Now, people begin this study and they have a little sip of the first wine, a Merlot, which they think is coming from a $5 bottle and then they say it tastes awful. Couple of wines down the line, they are getting another bottle of Merlot, which has a prominent $45 price label, same liquid. And when they rate it, not only do they describe it with far more positive adjectives,
26:22 Speaker 01 the quantitative score is 70, seven zero, 70% higher. So according to Shiv, partly, not completely, but partly what we experience is what we expect to experience. And one of the things that sets up our expectation is price, because people have a rule of thumb in their head that you get what you pay for. And the interesting thing is that becomes self fulfilling. So exactly the same liquid is rated wildly differently, dependent on the cost. So if someone isn't buying your products, economists would say, well, you know, you need to reduce it because you haven't got enough value and you're not providing enough utility in their mind. But before you do that, why don't you test the far more profitable idea?
27:06 Speaker 01 Maybe I'm charging too little.
27:09 Auren Hoffman? I saw a study somewhat of like a placebo effect where it was Advil and then people rated how much Advil worked. And then it was the same exact bottle of Advil, nothing different, except they on the Advil, they wrote 30% off. And that one actually worked less. It was just like less effective for some reason. These things are really like messing with people.
27:31 Speaker 01 Exactly. Even the world of painkilling, exactly the same pharmacological pill will have a different impact, whether we think it's expensive or whether we think it's cheap. You've got to be very careful about radical discounts on your products, reducing price, promotions that are too strong, because you're not only training people to think your product's poor quality, the actual experience of the product will be worse. That's, I think, the interesting behavioral science bias, the actual experience deteriorates as well.
28:02 Auren Hoffman? This is a data podcast and a lot of cognitive biases affect the way we executives interpret our own company's data. What are the dangerous ones that we should be aware of as leaders?
28:14 Speaker 01 I would say one of the big ones is overconfidence. And this is an area that's been studied hugely. So whether it's looks, humor, intelligence, it's not 50% of population think they're above average. It's more like 70 or 80%. Again and again, and all sorts of different categories, people are overconfident. Two of my favourite studies in this area, there's one by, I think it's Ola Svensson, Swedish psychologist back in the eighties, he was interested in driving. Now, it won't shock anyone that most people think they're a better driver than average. But what Svensson did, he went to hospitals where people are laid up, injured, they're in bed, they've been in car accidents, even that group, they think they're better than drivers than average.
29:00 Speaker 01 Well, there's another one in Britain by Konstantin Sedikides, the University of Southampton. He goes to prisons, these are young people's prisons, and even the prisoners, most of them think they are more moral than average. So this is a consistent finding that affects all walks of life. If people recognise that and they recognise it affects them as well, they could save themselves an awful lot of problems.
29:27 Auren Hoffman? And some of the things, they're quite hard to measure, it's quite hard to know, I don't even know how moral I am, and then I don't necessarily know how moral the average person is. These things are a lot harder to do, but there are other things where I think it is a little bit easier to have a sense, like I think probably most people have a probably a pretty good sense where they fall in the IQ zone.
29:47 Speaker 01 I think intelligence people tend to overestimate because a lot of people won't have done tests, or if they do a test, they think, oh, I was hungover that day, they'll come up with some explanation.
29:57 Auren Hoffman? It would be pretty rare, someone who's in the bottom 25% out of intelligence to think they're in the top 25% out of intelligence.
30:05 Speaker 01 I don't know this one off the top of my head, because there's not such a commercial application, but there is this idea called the Dunning Kruger effect, which is essentially the more stupid that you are, the more likely you are to exaggerate your intelligence. And I think some of it is if you don't know anything about a topic, you don't know how much you're missing out on. Now, there's this dangerous thing, is as you get better at a topic, it's like an island, the sea's receding, you kind of realise how much else there is to know, your coastline of ignorance grows, you're more aware of that. So I think actually the more stupid people are, there is this bigger gap.
30:46 Auren Hoffman? What are the biases where people think they're worse than average?
30:49 Speaker 01 I've only seen studies like intelligence, humour, looks where people are overconfident. And remember, we're talking on average, there will be some people who are overly underconfident. I've heard people make an argument that there's a gender variation. I've asked marketers, how good at their job are they versus their peers? Both men and women are overconfident. Ninety percent of men thought they were better than their peers at their job. It's more like eighty percent for women. So both genders were overconfident, but women less so than men. But I haven't seen something recently where there's a repeated finding of people being underconfident.
31:27 Auren Hoffman? I often ask people what percentile they're in for given traits. And I find that for many traits, people are quite accurate. They're like roughly good. It'd be my assessment of them. But I said, the trait where people are the worst at assessing is EQ.
31:44 Speaker 01 I think something interesting there though, Warren. How are you asking these people? Because if you ask people on an anonymous survey, and stress that it's anonymous. Let's say eighty five percent on average people say they are better than their colleagues at their job. I've done that study where I've stood up in a conference hall and asked people who's better at their job. And of a hundred people, you get three sticking their hand up. Because there is a very different thing about telling everyone, I believe that I'm better in a confessional anonymous situation. But I can't put my hand up and let other people know that I think I'm better.
32:18 Auren Hoffman? So it's a good point. When I ask people, people will often only admit to things that they are really good at. Where I found that where people just have no idea how good they are is EQ. People will be like, I'm very high in EQ. Or someone will say they're very low in EQ. And I don't even know if it has any relationship to the truth in that one.
32:35 Speaker 01 That would probably be one of the areas where I would be happy admitting that I'm probably below average on. Mainly because my wife keeps on telling me.
32:43 Auren Hoffman? Yeah, I'm definitely below average in EQ. But then who knows? Maybe some people are so bad at it.
32:49 Speaker 01 This question of the difference between what people say in public forums and private ones, I don't know if you've seen, there's an amazing book by Seth Stephens Davidovich called Everybody Lies. And one of my favorite examples is if he compares Facebook data with search data. So he looks at what queries are people putting into Google and then he compares it on a similar topic to what people are posting about on Facebook. So he looks at something like my boyfriend is, and the most likely words to come up on Facebook after that phrase are things like generous, wonderful, loving. On Google, it's things
33:32 Speaker 01 like cheating, an idiot, selfish. It's a really powerful demonstration of what people say when they know others will see it is about a airbrushed manufactured version themselves. To me, as a marketer, what it means is people who do social media listening should never take those statements as a verbatim truth. They are the image people want to project themselves in no way accurate insights what people actually think and do.
34:01 Auren Hoffman? AI is going to get better and better at personalization. Hopefully, I still get terrible recommendations on Spotify and Netflix and stuff like that. But hopefully it will get better and better and better at personalization. Does that make the traditional behavioral insights more important long term or less important long term?
34:18 Speaker 01 I'm a skeptic about psychology and behavioral science going out of fashion. There's a legendary creative director from the 1960s called Bill Bernbach. And he said, you know, it's taken millions of years for human nature to evolve. It'll take millions for it to vary. It may be fashionable to talk about the changing man, but what we as communicators should be concerned with is the unchanging man. Yes, technology changes, yes, culture changes, but these drivers of what people want, they are remarkably consistent.
34:51 Auren Hoffman? I can make an argument that behavioral science types of things will become more important in AI world. I can make an argument it will become less important. Surely they'll still be important, but on the relative importance, do you think it changes? Or do you think it's roughly the same?
35:04 Speaker 01 I think there's two ways you could look at this. Firstly, the more choice people have, the more powerful these biases are. Because, take a supermarket example, if you were standing in front of the shampoo aisle, If you try to logically work out which shampoo would maximize your utility per dollar spent, you would be choosing for an hour, maybe two hours. People don't do that. They just pick the one that's the mid-priced one or the most expensive one, the one they've seen on TV. They have a quick rule of thumb that they use to make a fast decision. One thing that is happening in society is we've got more and more information at our fingertips. As that goes on and on, I say it makes these biases more powerful. I think I would put my chips in that camp.
35:49 Auren Hoffman? I often just choose based on the packaging, what looks cool, kind of like judging the book by its cover. I have actually found that that's been a great rule of thumb. Just read whatever has the best, coolest cover is actually quite good. Or there are like weird sayings like that that you think are just like wrong. You always say, don't judge by your book by its cover. Maybe we should judge the book by its cover.
36:10 Speaker 01 If we extend that argument beyond books to products, the serving vessel, the aesthetics of those elements.
36:19 Auren Hoffman? Yeah, that beautiful bottle of vodka.
36:22 Speaker 01 Exactly. And it goes further than people expect. But you could go to the cutlery. There's a wonderful psychologist called Charles Spence. You know, I think he's the most interesting person when it comes to the psychology of taste and food. He did a study in Edinburgh where they served dinner at this conference and people are randomized into different tables. And some people get this very fancy, heavy cutlery. Other people get cheap, light cutlery. There's, I think it's 11, maybe 12% variance in terms of the ratings of exactly the same food. People have this kind of assumption that heavy equals quality and it becomes a self fulfilling prophecy.
37:02 Auren Hoffman? Of course, yeah. If I got heavy color, I'd definitely feel like it would taste better.
37:06 Speaker 01 All sorts of brands apply this. So yes, in the food world, expensive bottles of wine will be a bit heavier. But even you go through to fancy speakers, sound systems, Bang & Olufsen, their remote control could be super light, but they artificially put weight in it because they know this heft gives it a sense of quality. Certainly judging products by their aesthetic attributes, touch, sounds, weights, absolutely right.
37:34 Auren Hoffman? In the last 25 years, there's been just so many books and so many people have been talking about these cognitive biases. And I would have thought that because so many people were talking about it, it would mean they would be less effective or less involved. But it doesn't seem that's the case. It seems like we're still just as susceptible to all these things as we were before we learned about them. Why is that happening? Or am I right about that?
37:57 Speaker 01 They still affect us. There are some experiments where they run a charity donations, where you say things like, how much do you want to donate? That's the control condition. Then they use a principle like anchoring for another group of people, which say, well, lots of people give £50. How much do you want to give? What happens is people end up giving more money. So let's say they give £20 in the first group, £30 in the second group. Now, the principle of anchoring is you throw out this high number and people take it as a starting place. They adjust, but not enough. So it ends up affecting their end behaviour. Third group of people go to the donation page and it says, many people give £50. Please be aware that being told this figure can inflate people's donation levels. I'm making the exact numbers up. I know this study brilliantly, but they'll
38:42 Speaker 01 end up giving, let's say, £28. It does drop a little bit, but it's still much more than in the control condition. So even being told about the biases doesn't negate them completely.
38:53 Auren Hoffman? But it does lessen it potentially.
38:55 Speaker 01 It was a trivial lessening, though. That's the only thing to say. So yes, it does lessen it, but it's trivial. I would argue one of the reasons they don't disappear with knowledge is they're useful. It's the danger of people referring to them as irrationalities. Because if we go back to our shampoo example, who's being irrational? Me, who maybe cares about my hair a lot. That's obviously not true, but if I cared about my hair a lot, just picking very quickly the most expensive one. Or you, who is an econ and trying to, I'm not saying I don't offend you, but you go and weigh up all the pros and cons and you spend two hours there. Which of us is being irrational? The biases and these shortcuts
39:35 Speaker 01 and these rules of thumb are quite good ways of making very fast decisions. Most consumers, most people would prefer speed over accuracy in most situations.
39:47 Auren Hoffman? That makes sense. What is the next frontier of behavioral science that maybe is coming?
39:52 Speaker 01 I think one of the really interesting areas here is understanding far more about how different demographics or personality types are affected by biases, because most studies at the moment are just, on average, this principle works well or it doesn't work well. What's getting interesting is people are starting to test the relative impact on different groups. So on something like social proof, this idea that we're influenced by other people, and if something looks popular, it becomes more popular still, that's been around so long that that kind of work has already been done. For example, young people tend to be more affected by social proof than older people. There are some studies that show the super wealthy aren't as affected by
40:36 Speaker 01 social proof. There's a study by the British Tax Office, HMRC, that showed if you say to people, nine out of 10 people pay their tax on time in Britain, people increase the likelihood of paying on time. But if people were in the top 1% of debtors, actually it was the top 1% or 5%, these are people who owe like 50 or 75,000, $100,000, that group actually became less likely to pay their tax on time.
41:03 Auren Hoffman? Because you're like, oh, 10%, oh, great, I should be in the top 1%, or that's great, 10%, I didn't realize it was as high as 10% that is doing that. So I don't feel as bad anymore.
41:13 Speaker 01 I think the argument from the behavioral scientists was everyone thinks they're not influenced by the average, but this group genuinely thought they were different. So being told what the common man was doing actually made them less likely to do it.
41:27 Auren Hoffman? That makes sense. Two personal questions we ask all of our guests. What is a conspiracy theory that you believe?
41:33 Speaker 01 I think the problem with that question is, if you believe it, you define it as the truth, not a conspiracy theory.
41:38 Auren Hoffman? Well, you might have some sort of sense that other people think it's a conspiracy.
41:42 Speaker 01 It was certainly seen as a conspiracy theory when it first came out. And I'm not saying I was wholeheartedly certain, but I remember when there was COVID coming out and there was the lab leak hypothesis, and that was branded as a conspiracy theory. I always thought it sounded plausible. I'm not going to bet any money on it. I don't know whether there's other plausible explanations out there, probably are. So I always found that one was too quickly labeled a conspiracy theory.
42:09 Auren Hoffman? Well, sometimes when things aren't labelled quickly a conspiracy theory, there's a conspiracy to label them a conspiracy. So sometimes there is a conspiracy there.
42:17 Speaker 01 I think some of it was the us versus them political attitude because someone on the conservative side, Donald Trump espoused that view, therefore everyone on the liberal side didn't think about whether it was true or not. They just thought, well, I disagree with this man, so therefore it must be untrue. And I felt a bit of that went on with that.
42:34 Auren Hoffman? Yeah, that happens quite a bit. Our last question, we ask all of our guests, what conventional wisdom or advice do you think is generally bad advice?
42:42 Speaker 01 Well, we've covered what I was gonna say, which was asking your customers. I think that's awful advice. Tests. Look, prioritise what they do. Prioritise observed data over claimed data. One we haven't talked about is I think there's a myth about the decline of trust in businesses. Everyone's always talking about certainly marketing, like a crisis in trust. Actually, though, if you take something like the Edelman tracking, they're probably the ones that have been doing this for longest. If you actually go back and look how much people trusted businesses 15, 20 years ago, it was really low, just like it is now. People have never trusted businesses because, let's say you were in a Greek market 2000 years ago, and I came up to you and said,
43:26 Speaker 01 my olive oil is the best in the world. What you're gonna think is not, oh, wow, this is a miracle that I've discovered the one man selling the best. You're gonna think this man's trying to pour the wool over his eyes. He's just wants to make a quick denarii or whatever, drachma or whatever they used to buy. And I think that's a really dangerous myth, because if you think you're in a unique historical moment, you don't learn from the past. If you recognise the challenge is the same as it was 20, 30 years ago, well, you can learn from how the best people tackled it 20, 30 years ago. You could apply things like the Pratt-Ful effect. You could apply ideas like speaking very precisely and specifically, the studies that show that's more trusted. Now, if I say I'm 92.3% certain, it's more believable than if I just say I'm quite certain. It might
44:10 Speaker 01 not actually change the validity of my idea, but precision is taken as a shortcut by people for accuracy and trustworthiness. I think this idea about businesses being hit by a trust crisis, I think is a really bogus idea.
44:24 Auren Hoffman? This is great. Thank you, Richard Shotton, for joining us on World of DAS. I follow you at RShotten on X. I definitely encourage our listeners to engage you there and buy your new book as well. This has been a ton of fun.
44:37 Speaker 01 Thank you very much. And I actually post more on LinkedIn now.
44:40 Auren Hoffman? Okay, sorry. Okay, so also, let's follow you on LinkedIn, too.
44:43 Speaker 01 Thank you very much.
44:44 Auren Hoffman? Perfect. All right. Great to chat with you.
44:46 Speaker 01 Thanks a lot.
44:49 Auren Hoffman? Thanks for listening. If you haven't already done so, please subscribe to World of DAS wherever you consume your podcasts, YouTube, Apple, Spotify, and more. And please help us get discovered by leaving a review and check out worldofdas.com. That's worldofdas, D-A-A-S, dot com. And of course, connect with me on Twitter, at Auren. That's A-U-R-E-N. Would absolutely love to hear from you.
Previous The LM Brief: The Energy Bottleneck Behind AI’s Growth Next The LM Brief: The Ethics of Agentic AI - Balancing Autonomy and Trust Page updated November 1, 2026 · transcript and summary are machine-generated