Why Is Behavioural Genetics A Hated Science? - Dr Stuart Ritchie

Dr Stuart Ritchie is a psychologist and science communicator known for his research in human intelligence and an author. The influence of our genes on the outcomes we get in life has been long established and replicated in science. However the public response to this has been very unhappy, making Behavioural Genetics one of the most heated areas of research there is. Expect to learn why some people dislike behavioural genetics so much, what happened with the recent SSRI rug pull, whether Emotional Intelligence is an actual thing, how to be sceptical without becoming nihilistic, which psychological phenomenon were debunked during the replication crisis and much more... Sponsors: Get 83% discount & 3 months free from Surfshark VPN at https://surfshark.deals/MODERNWISDOM (use code MODERNWISDOM) Get 15% discount on Craftd London’s jewellery at https://bit.ly/cdwisdom (use code MW15) Get 15% discount on all VERSO’s products at https://ver.so/modernwisdom (use code: MW15) Extra Stuff: Buy Science Fictions - https://amzn.to/3y666Wl Follow Stuart on Twitter - https://twitter.com/stuartjritchie Get my free Reading List of 100 books to read before you die → https://chriswillx.com/books/ To support me on Patreon (thank you): https://www.patreon.com/modernwisdom #behaviouralgenetics #genes #psychology - 00:00 Intro 00:25 Why People Distrust Behavioural Genetics 11:10 Creating an Uneven Playing Field 19:35 Evidence of IQ Heritability 22:09 Impact of Replication Crisis on Behavioural Genetics 31:33 Is EQ Real? 38:48 Dying Psychological Concepts 45:39 The Placebo Effect in Psychology 51:54 Treating Depression with SSRIs 1:07:55 How to be Effectively Sceptical of Science 1:18:14 Where to Find Dr Ritchie - Get my free Reading List of 100 life-changing books here - https://chriswillx.com/books/ Listen to all episodes on audio: Apple Podcasts: https://apple.co/2MNqIgw Spotify: https://spoti.fi/2LSimPn - Get in touch in the comments below or head to... Instagram: https://www.instagram.com/chriswillx Twitter: https://www.twitter.com/chriswillx Email: https://chriswillx.com/contact/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Candidate gene research. This particular gene is related to depression. This particular gene is related to memory skills. And so it makes you smarter. Endless research on that, loads and loads and loads of papers published across all the top journals, millions of dollars of research funding, people wasting their entire careers, writing their PhD dissertations, doing job talks, getting employed at top universities on the basis of this candidate gene research. And it was all nonsense.

• 0:25Chris Williamson?

  Given your current academic background at the moment, can you explain to people why you think it is that behavioral genetics has so much distaste, distrust, dislike generally?

• 0:40Speaker 00

  There's a lot of different reasons. I think the main reason is some kind of a misconception about what it actually means to say that behavioral traits are related to genetics. So I think when people hear that, especially when the trait itself is controversial, like you can get into a whole debate about intelligence or personality without even mentioning genetics. You know, people, people get upset by just, you know, mentioning those, those traits. But when you say they're linked to genetics, people make lots of assumptions. People think they know your politics. They think they know like what you, what you're trying to say, what you're trying to like slip under the, you know, under people's, under the radar, under people's notice.

• 1:21Speaker 00

  And those assumptions are things like, well, if it's related to genetics, it must be completely unchangeable. If it's unchangeable, then we don't need to do anything about our political situation. And we don't need to, you know, help people out. And people are just stuck like a tram on a tram line. They can't, they can't turn off or change or end the lot. And if you're interested in behavioral genetics, you must be using it to justify our current political situation. So I think, I think that like, that's one of the major assumptions. Like immutability, the fear of immutability. And that's of course not what behavioral genetics says at all. Behavioral genetics is about trying to understand, you know, how things are right now, not necessarily

• 2:06Speaker 00

  how things might be if we change things in the future. Or indeed, if they may have been different in the past. We have lots of interesting studies of how like the genetic contribution to things differs across different times and different places and different political regimes even. There's some interesting research on communist regimes and how that might have affected the heritability of traits.

• 2:29Chris Williamson?

  How so? Yeah, tell us about that.

• 2:31Speaker 00

  Well, there's some research in Estonia, which obviously used to be a communist country. And so, yeah, this is a research using a polygenic score, which your viewers may be familiar with from hearing your interview with Robert Ploman, one of my colleagues here at King's College London. And so the idea is that you look at the genetic contribution to various traits and educational attainment being the main one, before and after communism in Estonia. So that was, it's just this incredibly cool paper that was done. Would be nice if it was replicated and stuff, but this is, you know, it's an interesting, it's an interesting.

• 3:09Chris Williamson?

  You'd need more communist regimes to come and go in order for that to happen. It's going to be difficult.

• 3:14Speaker 00

  You need a few, you certainly, you need to do that same study in lots of other like Warsaw Pact countries, which are now no longer communism. So the idea is that in people who were born after communism was gone, genetics explains more of the variation in people's educational attainment, for instance. So like, and the idea is like the broad interpretation or one interpretation you can draw from that is that a freer political system, one that doesn't, you know, oppress people in the same way that communism does, allows them to kind of reach their genetic potential is one, is one way you might want to put it, more effectively than one which kind of, where the environment really was kind of suppressing people's ability to kind of be who they, who they really were.

• 3:59Speaker 00

  So that's, that's one interesting thing. But that, but you know, in just saying that, you can see how the environment makes a difference to how genetics operate. And that's, you know, research that's done by, you know, behavioral geneticists who are not, you know, criticizing polygenic scores. They're saying, look, we can use polygenic scores to illustrate interesting things about. Our society and how society changes and so on. So I think it's a real misconception to say that genetics means that we're immutable in some way. I mean, the classic thing people talk about, the kind of almost boring cliched thing that people talk about is glasses, right?

• 4:40Speaker 00

  Is eyesight is really heritable. So myopia, short-sightedness, really, really heritable,

• 4:48Speaker 00

  runs of families, all that sort of stuff. But, you know, we can cure it instantly or effectively cure it instantly by just like putting on a pair of glasses at the right prescription. So things in the environment can completely alter the way that genetics has its effect. Now, it's a slight cop out to say that because we don't have the environmental equivalent of glasses for other traits like personality. Like, we don't know the stuff that will massively instantly change your personality. We don't know the things that will massively instantly change your intelligence or your educational attainment and all that. Yet, maybe we will at some point in the future, but we don't have that right now. Maybe it's a good thing that we don't know ways to massively alter people's personalities using

• 5:30Speaker 00

  the environment, because of course that might be used by somewhat nefarious political regimes or cults or whatever else. I mean, cults are already pretty good at controlling people's behaviour and so on. And if they had ways of changing people's personalities more effectively, then that might be rather scary. So, yeah, but that's, you know, one example that's very often used. Another one is height. Like, height is really heritable. We know that tall parents tend to have tall kids. Shorter parents tend to have shorter kids, on average. So it goes. And yet, if you don't feed a kid while they're growing up, they'll be stunted. People in North Korea are like six inches shorter than people in, or four inches, whatever, is shorter than people in South Korea because of all the malnutrition.

• 6:12Chris Williamson?

  No one's criticizing the fact that behavioral genetics suggests that that's the mechanism by which tall parents have tall kids, right? Well, not behavioral genetics, I guess, but just heritability generally, right? The height is heritable. What is it about other elements that causes people, shoe size or eye color? What is it about that that causes it to be such a different ballpark?

• 6:34Speaker 00

  Well, you've put your finger on the thing here, which is the double standard, which is people are very, very happy to talk about, you know, genetic influences on stuff like height or eye color and things like that, that don't have a political valence. But as soon as it, you know, you use those exact same methods, and that's the important thing, like you use the same methods studying twins, studying families, studying molecular, you know, DNA differences between people, you use those exact same traits to study stuff like intelligence, personality, whatever, and people flip out. And that's, I think, a weird double standard. I mean, what would one of them say in response to that? They would say, well, those traits are a lot more complicated. They're socially influenced,

• 7:17Speaker 00

  things like educational attainment is influenced by all sorts of stuff, the quality of the school you went to, the socioeconomic situation that you find yourself growing up in.

• 7:26Chris Williamson?

  Just to jump in there, didn't Plowman find that when you equate for everything else, schools on their own have less than 5% of an impact on someone's educational outcomes?

• 7:39Speaker 00

  That's the, yeah, he used the Ofsted ratings of schools to show that. Yeah, I think, I think probably in the UK, that's probably true. I think in other countries that might, that might differ slightly. Like, I think in countries where the schools really, really dramatically differ, we've done a pretty good job of like equalizing because of things like Ofsted inspections of fairly.

• 7:58Chris Williamson?

  The American people that are listening, Ofsted is a external accreditation board that comes in and makes sure that the school's doing everything right and that the students are getting the appropriate. And we also have a much more homogeneous teaching curriculum, right? Everybody across the entire, maybe UK, but certainly England, has one set of exams at this age, at this age, at this age. Unless you're in some weird like international business schooly type thing. Pretty much everybody has the same stuff. So you have this.

• 8:28Speaker 00

  It is a bit different in it is a bit different in Scotland where I'm from. They have the in Scotland they have the curriculum for excellence, which I believe is not all that excellent. But but anyway, yeah, you're quite right. So so yeah, what what Robert did in that study was found that once you control for the selection into into schools, so private schools, for instance, and

• 8:51Speaker 00

  so yeah, there's two different, there's two different, there's the private school versus versus state school one. And there's the offset ratings one. And in both cases, the quality of the school doesn't make that that that much of a difference. But I think what someone might say in response to my argument is that there's a lot of, you know, I'm just using school quality as one as one potential thing. But like, there's a lot of other stuff that goes into that. And something like intelligence compared to something that goes into height. I'm not entirely sure about that because there's loads of interesting biological stuff going on with, you know, nutritional intakes and all the sort of stuff that happens when you're when you're trying to understand height. And almost inevitably, these traits that we think are really straightforward. When we do more in-depth genetic analyses of them, even something like eye color, it becomes a much more complex story as to exactly how it's going on.

• 9:37Speaker 00

  And there were some papers recently on eye color that they were basically saying like, the story is much more complicated than we than we than we thought.

• 9:44Speaker 00

  But that's also the case for things like intelligence, education, personality, all that sort of stuff. But but I think there's this kind of allergy to even talking about stuff like education as if it could be

• 9:57Speaker 00

  studied in a kind of genetic sense, or even among among some people, like they think that it can't even be studied in a psychological sense as well. Like, it's a purely social thing. Like, it's ridiculous to try and quantify it. And there's a lot of, you know, opposition to quantification and and using standardized tests and all that kind of stuff.

• 10:17Speaker 00

  In in these kind of contexts. So, yeah, I think people have this have this strange double standard when it comes to to that. And also, there's a double standard in the opposite direction, which is they're very happy to. When someone says, you know, this particular, you know, intervention in the environment, whether it's teachers doing something different, or parents doing something different, or the government doing something different, you know, this had a big effect on people's,

• 10:44Speaker 00

  on kids' educational attainment, for instance. Using those same kind of methods, you could, you could look at, you know, genetic influences too, like polygenic scores, using the same kind of statistics and the same kind of stuff. And yet, people, people are not happy to take the same, draw the same conclusions there. So there's double standards everywhere. People are just kind of, really, they feel this real aversion to genetic explanations.

• 11:10Chris Williamson?

  Well, so tell me about where this is coming from, because it seems like, here's my, here's the bro-science explanation, right? That coming into talking about behavioral genetics, group differences and individual differences got lumped in together, group differences got used by people that have some pretty nasty ideas about better and worse types of individuals, and that has caused downstream for behavioral genetics to kind of be lumped in with quasi-racists. That's one reason. Another reason is we live in a meritocracy, and in a meritocracy, if you are your successes and you own your failures, someone that is told that you have a predisposition toward being more successful ahead of arriving into the world, given that success and your achievements

• 11:55Chris Williamson?

  in life are one of the fundamental things that you take your status and your wellbeing and your sense of everything from, that also feels very unfair. You know, in a world that's trying to give people equality of opportunity, realizing that people start, like literally start the race at very, very varying levels, is kind of a hard thing to bring in here. What do you want us to do? Do you want us to try and normalize for genetic predisposition before people get to school? That, that doesn't seem very fair. Okay, what happens if we completely flatten the environment so that everyone gets the same? Okay, so what you're saying then is that the only differences and outcome are going to be exclusively genetic? Well, God, that doesn't feel very fair either.

• 12:36Speaker 00

  Yeah, yeah, absolutely. And the funny thing is that I think people... You know, this is something which the average person understands very, very well. There's that paper that showed that, you know, the people who get, who are the most accurate at predicting the heritability of different traits, so like how heritable things are, obesity, height, intelligence, whatever. The people who are most accurate at saying, yeah, genetics are probably involved in that, are mothers with more than one child, right? So there's this idea that if you've got one child, then, you know, that's interesting. But if you've got another one, you can say, oh, right, I'm not parenting this kid particularly differently. I'm not doing anything massively different. And yet, this child is totally different.

• 13:20Speaker 00

  And to be honest, if you know anyone who, if you know brothers or sisters or, you know, whatever, siblings of any, of any kind, like, you know that they differ dramatically in their personality. And it's probably not that they were like expressly parented in a particularly different way. There are genetics that influence that. So we all, we all realize that. Some, some kids are, you know, starting school with slightly higher aptitude for, you know, sitting down and concentrating and some are, some are not. But the thing is, the mistake is, is to, is to assume that there's only one political outcome from that. There's only one political interpretation of that because the kind of more liberal interpretation of that is, well, if people start off in different things, then you have to think in terms of John Rawls, like the, um, the veil of ignorance. And this is what Paige Harden talks about in her book, the veil of ignorance, which

• 14:04Speaker 00

  is, you know, if you didn't know what traits you were going to have, uh, you could be any, you could be entered into the genetic lottery, as Paige puts it, and, uh, and, and start your life as one of many people. How should we set up the world such that you're, you know, uh, that the world is as fair as, as, as, as it possibly could be to whatever you might be like? Um, and so that implies a lot of leveling off. That implies a lot of, you know, uh, extra resources for kids who are struggling, uh, making sure that things are as equal as possible, making sure that opportunities are, are, are given to kids who maybe wouldn't otherwise get them. So you could draw a very, you know, progressive liberal conclusion from that, just as much as you could draw the conclusion of, oh, well, we shouldn't bother helping people

• 14:49Speaker 00

  because, you know, genetics is having an effect.

• 14:52Chris Williamson?

  Yeah, it does seem like an incredibly illiberal way to view the situation that that evidence over there that could assist us in helping people to get themselves to the kind of life and the kind of world that they want to have, we shouldn't know about that. Well, hang on, whether you decide to believe in it or not doesn't mitigate the impact of it on those people's lives.

• 15:14Speaker 00

  Yeah.

• 15:15Chris Williamson?

  And I think I had a great conversation with Paige and I appreciated the fact that she engaged with, given that she's very from the left, an incredibly difficult circle to square, right? It is very hard to work out how you combine left-leaning politics with a deep understanding of behavioral genetics and the impact that they seem to have on people's outcomes in life.

• 15:40Speaker 00

  Yeah, totally. And that's, you know, what one, you know, we must admit as behavioral geneticists that one reason that people have drawn the kind of more right-leaning conclusion and fear the right-leaning conclusion is that there are lots of figures in history who have drawn that conclusion, right? But if you go right to the very start of when people were kind of inventing intelligence tests and so on, this was very much the idea. So Godfrey Thompson, who was this kind of very well-known intelligence researcher at the start of the 20th century, University of Edinburgh, there's a, there's a building named after him in the University of Edinburgh Education School, where, by the way, they probably don't teach that much about like IQ and so on anymore. But it's somewhat ironic

• 16:24Speaker 00

  that it's in a building named after him, but fair enough. He was of the opinion that, and he didn't do so much genetic stuff, but he was of the opinion that, you know, intelligence differences that kids have. imply that we should spend exactly the same on every single child. So, so like,

• 16:43Speaker 00

  you know, no matter whether they're struggling or doing really well, every single child should get the same amount of resources from, from, from the state. He wanted things to be as equal as possible. And yet, he was one of the pioneering intelligence researchers and made major contributions to our understanding of, of, of exactly what intelligence is and what the G factor is, the general factor for intelligence, all this kind of stuff. And, and you know, the, the, the, the general, you know, influence that he and other people around him at the time had on the education system was that, you know, there are all these people out there who are, you know, would not otherwise be noticed by the good schools. Because of course, at that point, getting into a good school meant knowing people, your parents being rich and all that sort of stuff. And so, we should

• 17:27Speaker 00

  try and have an objective way of doing that. And that's why they influenced the Butler Education Act in the 1940s. And, and that's how we got the grammar school system that we had in the UK for a long time. Now, the grammar school system turned out to be not particularly effective, because the people who went to grammar school had a great time, but the people who didn't pass the 11 plus test, which was the, the test that you, the IQ test that you did, ended up in secondary moderns, which were very, very poorly resourced. And there's all these like hellish stories of, you know, the latter half of the 20th century, the people went to secondary moderns and had a horrible time. And I can totally understand these crumbling buildings and all this kind of stuff. But, but that was not the intention. Like, the intention of the psychologists who set this up was a kind of, was a kind of noble one, which was that, you

• 18:12Speaker 00

  know, we, we were going to try and find smart kids from poor backgrounds and give them, you know, an academic curriculum that was, that is appropriate to them. Not, not give them more resources or anything. Just give them an academic curriculum that includes stuff that's more appropriate to them, and give an academic curriculum to people who are, you know, are not as academically inclined. So, that was the original intention.

• 18:38Speaker 00

  You know, subsequent to that, there came lots of people who did in fact believe in, you know, eugenic theories and, and, and a lot of all the, the, the stuff that we now hear IQ tests, you know, associated with. But, you know, the original IQ tests were, you know, uh, uh, made for noble reasons. The very, very, very first one, Alfred Binet in the nineteen teens, uh, was invented to help kids who were struggling in school. Uh, you know, kids who maybe, um, had particular, you know, what we would now call, you know, special educational needs, um, was invented to have an objective way of diagnosing them and, and, and giving them extra attention and help. So like the, the kind of, um,

• 19:17Speaker 00

  if you believe this stuff, therefore you must believe in like predestination and immutability of traits and you must be, you know, have these, these, these right wing, uh, views on things. It's just completely, it's, it's ahistorical and it's, uh, and it doesn't follow from the science.

• 19:34Chris Williamson?

  Am I right in saying that the best evidence we have at the moment is that IQ correlates point eight by your sixties or something like that with your parents, about eighty, eighty percent? Is that right?

• 19:44Speaker 00

  Uh, well, the, the heritability, so there's this weird, there's this weird, uh, result where the heritability, uh, uh, goes up over time.

• 19:53Chris Williamson?

  Yes.

• 19:54Speaker 00

  So like environment seems to have more of an influence at the start, uh, of your life and then, uh, it gets less important in, in, in measuring, uh, uh, you know, the, the, it's impact on intelligence. And then, yeah, it goes up to the heritability, that is the percentage of the, the, uh, variance in the trait that is associated with genetic differences. Yeah, it's, it's sixty, seventy, eighty percent by the time you do it in older folks. You know, there's studies like, um, the vets, uh, the Vietnam Experience Twin Study where you've got all these twins who were in, they happened to be in Vietnam and that's not the most important, uh, part of it. But you've got all these twins who, uh, were there and they did an IQ test and, uh, uh, uh, and so on. So, um, they've got a younger IQ test and an older IQ test, um, one that they did when they. You know, when they went to Vietnam and they were like 20 years old and one that they

• 20:39Speaker 00

  did when they were later in life. And you can see that heritability increases over over over time.

• 20:44Chris Williamson?

  Yeah, well, I mean, that that's fascinating. The fact that we have so much of ourselves that is a part of our parents is it's kind of something that's beautiful. But yeah, I think the the fact that Plomin said it does not predetermine, but it does predispose was was how he put it to me.

• 21:01Speaker 00

  Yeah, Robert has some of these phrases like like that. And like, it makes it it makes a different it matters, but it doesn't make a difference. It's not like that.

• 21:10Chris Williamson?

  Yeah.

• 21:11Speaker 00

  Is that the best way to express that?

• 21:14Chris Williamson?

  I'm not sure. But there was a tweet that I put out that I got a lot of pushback for when I first had Plomin on. And it was a quote from him from the episode. And he said something along the lines of the single most important thing that you can do for your child's future happiness, educational outcomes and income is the partner that you choose to have them with.

• 21:32Speaker 00

  Yeah, it's I can see why people would be upset by that. But like, that's one of the things that we know will have will make a difference. This other stuff in the environment, like interventions, are we're much less certain about. You know, just, you know, for instance, things like breastfeeding, does that does that improve people's intelligence? Like, that's a big, big thing.

• 21:57Chris Williamson?

  Natal nutrition.

• 21:59Speaker 00

  Yeah, exactly. All sorts of things like that. We just don't know. The studies are not that good on that. Whereas the studies are pretty reliable in knowing that, you know, genetics has an effect on on people's outcomes.

• 22:09Chris Williamson?

  That's a good question. There has been this big replication crisis everywhere at the moment. And it feels like studies that were often used either bro scientifically or really scientifically are being thrown out. How much has behavioral genetics been axed by this recent replication crisis?

• 22:27Speaker 00

  Well, I think behavioral genetics, I wrote about this in my most recent book, Science Fictions, which is about the replication crisis exactly. And I think behavioral genetics. Was one of the very first fields to be completely smashed by the replication crisis in one of its respects. So in behavioral genetics, you've got lots of different, you've got lots of different types of research. So you've got research with twins, you know, do the difference between, you know, identical twins and fraternal twins and like inferring things from that about how genetic particular traits are, that's one thing. But then you've got the molecular stuff. And when I was a PhD student, so like 10 years ago, all the way through that, we had candidate gene research. So it was like, this particular gene that

• 23:09Speaker 00

  we have a theory about is related to depression, for instance, or sometimes more complicated, like this gene, and if you get abused as a child, it's going to relate to depression. This particular gene is related to memory skills, and so it makes you smarter if you have this particular variant of this gene. And endless research on that, loads and loads and loads of papers published across all the top journals and many of the non-top journals, millions of dollars of research funding, people basing their entire careers, writing their PhD dissertations, you know, doing job talks, getting employed at top universities on the basis of this candidate gene research. And it was all, asterisk, not

• 23:52Speaker 00

  all, but like 99%, nonsense. It was all unreplicable, almost all unreplicable. There's like one example I can think of that didn't fail, when people thought, wait a second, should we try like replicating these candidate gene results in big studies? So not just like 100 people, but like several thousand people to see if this particular genetic variant is linked to, you know, variation in memory and so on. And these were big effects, like there were studies published in some of the top journals that were like, 20% of the variation in people's memory abilities are explained by this one, this one genetic variant. And looking back, it seems ridiculous, but at the time, that was very much, you know, The kind of standard study that you would do.

• 24:37Speaker 00

  It all felt a bit, people tried to replicate it, they couldn't find that those genes were in fact related to the disorders and the traits that were related to them. And there was a massive replication crisis. And now we've moved on, instead of doing these candidate gene studies, we've moved on to genome-wide association studies, which instead of looking at like one genetic variant, you look at literally hundreds of thousands of genetic variants. And it turns out that for things that are complicated, even something like height, but also you know, things like educational attainment and so on. It's not that there's one gene that like has a big effect, it's that there's tens of thousands of genes that each have this little infinitesimal effect and it all builds up. So you know, you might have a difference here and I might have a difference here and

• 25:20Speaker 00

  there's so many, many, you know, many, many differences that we have that make me slightly taller or slightly shorter than you and or whatever it is. And that's now how we think about things. And in those genome-wide association studies, we are starting finding like replicable evidence now. But like, behavioral genetics was was completely, you know, trashed by the replication crisis. You know, in the early 2000s, kind of before we started even talking about the replication crisis in psychology, this was this was sort of happening. And you know, the one that I can think of that that hasn't, you know, failed entirely is the Appo E gene, the Appo lipoprotein E allele, which is if you've got one allele

• 26:00Speaker 00

  of this, I think you've got like substantially higher, like twice the risk of getting Alzheimer's disease. You've got two alleles of this, something like 10 times the risk. And that comes out every every time for Alzheimer's disease. That's like the gene that we know about that is related to Alzheimer's disease. Yeah, I've got one of them. I did my 23andMe, I've got one of them, which is, you know, enjoy me while you can. But but you know, we, behavioral genetics has made substantial reforms and we don't do that sort of research anymore, or mostly. I mean, you see the odd paper coming up, like I get asked to review the odd candidate gene study and like the reviews can can just be like. This will not replicate. Bollocks. It's a replicate, it's just, it has to be bollocks.

• 26:43Speaker 00

  And then, so yeah, as I say, the stuff we've moved on to, which is the kind of world of genome-wide association studies, polygenic scores, all that kind of stuff, is much more reliable in its basic associations. However, there's a whole bunch of other things, other like concerns that are around that sort of research. So, and I don't mean ethical concerns and political concerns, I mean scientific concerns. So like, how much does assortative mating, i.e. people having kids with people who are like them, bias our estimates? How much does not having diverse samples mean that we're not learning enough about actual, you know, humanity in general? So like, the vast majority of this research has been done on people from a European background,

• 27:28Speaker 00

  you know, white American people or white people from Europe, basically. And so, you know, and when you try and use those polygenic scores, they don't predict as well when you look at people from other backgrounds. So we need to, we need to do a lot more research on, on, on that to really understand, you know, how these traits are made up. And, yeah, so there's, and there's all sorts of other, you know, statistical and technical and all sorts of other concerns with that. But we are getting more basic, you know, replications done. Whereas in the candidate gene research, we just went way ahead of, of where the science actually was and started saying, well, we found the gene for intelligence, the gene for memory, the gene for depression, all this sort of stuff. And it was completely wrong. And I think we have to constantly remember and have it at the back of our minds, or

• 28:11Speaker 00

  maybe even the front of our minds at all times, like, for decades, for maybe a decade, we were going completely on the wrong track and, and, and everyone was, you know, maybe not everyone, there were, there were some people who were saying, well, for instance, there were some people saying, didn't Ronald Fisher, the geneticist and statistician, write a paper in 1918 that said we should expect really tiny effects of genes and not really big effects? Isn't this research kind of going against that? Some people did point that out at the time, but nobody listened to them. And we had a whole field spending endless money, taxpayers' money in many, in many cases, just wasting it on statistically invalid studies. So we should, we should always remember that. And I think even areas That are now a bit more rigorous, like behavioral genetics, need to bear that in

• 28:56Speaker 00

  mind. It's like the, I don't know if this is ahistorical, but you know, when a Roman general came back and did a triumph in Rome, there was always like a slave standing on the back of the chariot saying like, remember you will die, you know, in their ear. I don't, I mean, I think that might not-

• 29:12Chris Williamson?

  No, I brought that up about Marcus Aurelius the other day, that as he walked through the streets of Rome and everyone was hailing him as this philosopher god-king and he was the benevolent leader, he would have someone behind him the whole time that would say, "You are only a man."

• 29:26Speaker 00

  Precisely. So like, I think we need to constantly have that, like, there was a replication crisis in your field like just a few years ago, you know, so-

• 29:33Chris Williamson?

  Okay, so that's the genetic side of things, or the, looking at the individual genes itself. In terms of the behavioral genetics, environment versus gene side, heritability side, how robust has that been in terms of replicating?

• 29:50Speaker 00

  I think the, you know, if you look back at the twin, the twin stuff and the family stuff, so you know, not using, not actually trying to go in and like measure DNA differences, but looking at family differences, that stuff is much more robust, much more rigorous. Those studies are relatively straightforward to do in the sense that they were doing them, you know, early in the 20th century and so on. Interesting potential fraud case with one of the very famous studies on intelligence and heritability. Cyril Burt, if anyone wants to look up, there's some interesting case there. I'm actually halfway through writing an article about this and whether, you know, Cyril Burt was a fraud or not, because there

• 30:32Speaker 00

  are people that have questioned that. I think probably he was a fraud, but anyway, his results have been confirmed by subsequent research, which is that, yeah, if you ask, you know, to what extent are traits like intelligence related to genetic differences between people, you'll robustly get the answer. And then if you take a step back and just, you know, forget the genetics and ask things like, you know, is there a general factor of intelligence? Like, I- Are people who are good at one type of intelligence test, you know, do they tend to be good at every type of intelligence test? That's one of the most replicable results in psychology. Like, you get that every single time. Tests correlate positively together.

• 31:10Chris Williamson?

  And yet, a lot of people would be very upset to hear that.

• 31:13Speaker 00

  It's funny, isn't it? It's quite ironic that people are upset by one of the most rigorous and replicate findings. But it's because, you know, it's this thing I mentioned right at the start. People assume that when you say that, you must mean all this other stuff. It's like, no, no. When I say that, all I mean is that people who are good at one type of test tend to be good at all the other ones.

• 31:35Chris Williamson?

  Are there multiple types of intelligences?

• 31:37Speaker 00

  Hmm. Well, depends on what you mean.

• 31:39Chris Williamson?

  So. EQ. Is EQ a thing?

• 31:41Speaker 00

  Well, so there's a few things to say.

• 31:46Speaker 00

  When you do an IQ test, you will notice that, if you do an IQ test, whatever, you will notice that there are lots of different types of tests, right? So you'll do a memory test, a vocabulary test, a speed test, all sorts of spatial rotation tests, shape rotators, everyone was talking about that on Twitter.

• 32:03Chris Williamson?

  Word cells.

• 32:03Speaker 00

  Yeah, word cells versus shape rotators.

• 32:05Chris Williamson?

  Are you a word cell or a shape rotator?

• 32:06Speaker 00

  I think I might be a, I think I'm, well, I don't know. If I look at, like, people in the physics department, I definitely feel like a word cell. But, but, but I, the, you know, those types of tests could all be described as, like, you know, specific skills, right? If you look back at Charles Spearman in the 19, early 1900s, 1910s, who was the first person to kind of talk about general intelligence, we remember him very much for talking about general intelligence, but he also talked about specific skills too. So the general factor of intelligence explains, like, 40 to 50% of the variation across all different tests. Like, that's the rigorous finding, that's what comes out.

• 32:45Speaker 00

  But there are specific skills too. So, and what that means is, they can vary independently of the general intelligence. I mean. You know, one silly extreme example, just to illustrate it, would be like if you sat down every single day and like memorized the dictionary every single day, then your vocabulary would, you know, your specific skill of your vocabulary would improve, whereas that's not going to help your, um, your, you know, uh, shape rotation or, or, or whatever. And in fact, there's a whole literature on, on like working memory training, you know, like those brain training games that were quite a thing for a while. Uh, people did research on that for a long time because they, there was this initial finding published in a big journal, uh, uh, proceeding in the National Academy of Sciences, that said that if you do that stuff, it improves your, your general ability.

• 33:27Chris Williamson?

  Turned out to be bollocks, didn't it?

• 33:28Speaker 00

  Everyone was like, oh my God, you know, we gotta, gotta try and replicate this. And then it didn't replicate. It felt a bit when people tried to do studies, they had better controls and all that sort of stuff. Um, but that was the idea, you know, that, that training up one task can transfer to other ones. That, that seems to, at least for the case of working memory, that seems not to be the case. Um, but yeah, so that's one thing to say is that there are all these different specific skills. They tend to correlate together, but they don't correlate together at like one. So they are different to some, to some degree. So if you mean are there multiple intelligences in that respect, there are specific skills, that's the most obvious one. Then there's, is there this thing called multiple intelligences theory that was made up by Howard Gardner at Harvard University? Um, and when I say made up, I actually, you know, use that word advisedly because he

• 34:13Speaker 00

  didn't have any data. He just decided that there were these different types of intelligence, you know. Um, it's not, it's not the same as verbal auditory kinesthetic learning, which is another thing which, you know, schools love, but it's that kind of thing, that there are, there are these multiple different types of intelligence, arithmetic, uh, arithmetical intelligence and, uh, and so on. And over the years, he's added new intelligences. So he's decided that there are new ones. There's existential intelligence now, um, interpersonal, intra-personal intelligence, um, uh, naturalistic intelligence, which is about like how much people know about plants in the garden and stuff. And he's decided that that is an intelligence. Now, Those are skills, those are real things that we that we that we care about.

• 34:56Speaker 00

  It's it's cool to meet someone who knows the difference between plants, you know, when you're out in the woods or whatever, that's that's great. But is that what we would call an intelligence in the same way that we would talk about like, you know, verbal and spatial abilities? Probably not. And does it mean that there's no such thing as the general factor of intelligence? No, because there's actually no empirical content to the to the multiple intelligences theory whatsoever. It's just one guy's opinion. Which is remarkable, like Harvard psychologist had a massive impact on the world and the way they think about how the human mind works, having done no research.

• 35:32Chris Williamson?

  So you're saying you're saying that there's still potential for my academic career to take that one?

• 35:36Speaker 00

  Yeah, you don't need to do any research, just come up with an idea that people like and they will spread it around every school.

• 35:41Chris Williamson?

  Dude, I'm brilliant at doing that.

• 35:42Speaker 00

  Yeah, well, exactly, exactly. So that's a good thing. Then you mentioned emotional intelligence.

• 35:48Chris Williamson?

  EQ, eats IQ for breakfast, that's the.

• 35:50Speaker 00

  Which is like another, which is like another thing. And my understanding of that is a few years ago, there was a meta-analysis done on all the research predicting job performance from EQ, IQ and personality, right? And what they found was, if you just look at EQ on its own, it predicts job performance. People who have higher emotional intelligence do better at at work, right? So there you go, that correlation is there. If you just look at the bivariate correlation between those two things, totally, totally robust there. However, if you put intelligence and personality, like the big five personality factors, you had Christian Jarrett on the show recently talking about that sort of thing, and into the same equation, then EQ no longer has an effect.

• 36:33Speaker 00

  And that's in predicting in predicting job outcomes. And that's because EQ is just a redscription of stuff that we already knew about, right? So so it's just a redscription of being, you know, caring about what people's feelings and emotions are, right? So that's one, that's one part of it. And some of that comes across in some personality factors. And being smart enough to kind of operate in your mind, okay, that person must be thinking this way, you know, I can, I can kind of, if I say this to them, it's going to have this effect. Like, it takes quite a smart person to be able to kind of, I mean, the slightly derogatory way of putting it would be like manipulate people in in in that way. So in predicting job performance, it's just a redscription. So I can't remember which way around it is.

• 37:16Speaker 00

  There's the like, people talk about the jingle and jangle fallacy, right? And one of them is, the jingle fallacy is. Describing the same thing with a different name and therefore thinking it's something different. And then the jangle fallacy is calling different things the same name and thinking that they must be the same. Maybe it's the opposite way around. I can't remember which way, which is jingle, which is jangle. But psychology psychologists do an awful lot of that, like re-describing things with a new name and thinking it's something different. One example is grit, right? So there was this big book in like 20

• 37:46Chris Williamson?

  Angela Duckworth?

• 37:47Speaker 00

  Yes, Angela Duckworth, 2016, something like that, that came out, that was, that was great. Man, it's in every single school in the country. Everyone loves it, like the power of persistence and passion and all that sort of stuff. If you look at the studies, it's literally, you know, it's correlated at point nine on a scale of minus one to one, it's correlated at like point nine

• 38:06Chris Williamson?

  Conscientiousness?

• 38:07Speaker 00

  With conscientiousness. It's just the same thing.

• 38:09Chris Williamson?

  I didn't even, didn't even know that and I picked it up.

• 38:11Speaker 00

  There you go. And, and, and, you know, there's meta-analyses that have come out saying that it really is just a redscription. So it's not wrong. It's not that grit is a load of nonsense. And it's also not that emotional intelligence is a load of nonsense. It's just like

• 38:22Chris Williamson?

  It's a jangle.

• 38:23Speaker 00

  It's just a new way of talking about something that we already knew about. And so, you know, the contribution there is, I'm not clear what the contribution is apart from like, you know, maybe popularizing old stuff.

• 38:33Chris Williamson?

  Well, that's what I would say. I would say that given the way that pop psych works at the moment, that create a branding problem, a catchy branding problem is 90% of the battle. I know that growth and fixed mindsets came under the ax. Actually, why don't we do this? Give, give me some of your favorite eviscerated psychological concepts from the replication crisis. What is, what's dying, bleeding out on the cutting room floor?

• 39:05Speaker 00

  Well, I mean, the classic one, I mean, I'm very happy to talk about growth mindsets. We can come to that. But the classic one that kind of was there that kicked off the replication crisis in many ways was priming, like social priming, which is that there are all these things in the environment, words, phrases, ideas, and that when you see them, it changes the whole way that you act. So the classic study, which I remember reading as an undergrad and I was being, I was taught it in lectures and I remember read the study. I was like, wow, this is amazing. It was in the textbook. Incredible. People, you sit people down in a lab with a computer and they're doing like some task where they have to like tell whether a sentence is a real sentence or a word is a real word. Like there's words and non-words, you know, and some of the words that come up in one

• 39:47Speaker 00

  of the conditions, you know, for half the participants are to do with old age. So they say things like Zimmer frame or. Fragile or frail or something. Yeah, exactly. Fragile, frail. One of the words that I remember that was used in study was Florida, because apparently people in the US associate that with old people. Fantastic. It seems about 10 years, but that's what they, that's what they said. And then what they found was that the, you know, for the other half of the participants, it was just random words that didn't have any particular theme. And for, you know, the half that saw the words that are to do with old people, they walked more slowly out of the lab, right? So they, they, when they were leaving the lab, there were like people measuring with stopwatches and they walk more slowly. That was apparently the finding. I remember reading that in social psychology literature and going, wow, that's incredible. That's amazing

• 40:31Speaker 00

  to believe that. And there was a whole range of studies like this, showing people the American flag makes them much more likely to vote Republican at the next election, even years later. You show them the American flag once and they'll vote Republican much later. People, you know, serious scientists published that in scientific journals. Serious science journalists wrote about that in, you know, wherever, whatever magazines that they were writing about science in and didn't think, that sounds a bit, that sounds a bit impossible. Another classic example, which I don't think has ever been, I don't think anyone's ever attempted to replicate it, but it was, you go into a room and there's a box in the middle of the room, like a big cardboard box, and half the participants

• 41:13Speaker 00

  sit in the box and they do like a creativity task, like how many uses of a, for a brick can you think of, like that sort of creativity task. And half the participants sit just next to the box, right? And what they found was that the participants who were sitting next to the box had a higher creativity score and they thought it was because they had the idea in their head activated of thinking outside the box, right? I mean, literally, this is how absurd, you know, some of the social psychology was. Come on. That was published in Psychological Science, one of the top journals of the field in 2012, I think. And so that idea of priming, or social priming, behavioral priming, has really fallen by the wayside. Linguistic priming.

• 41:54Speaker 00

  I think it's pretty solid. So like, if I say an active sentence, then you're more likely to say an active sentence back rather than a passive sentence, you know, that sort of, that sort of thing, like that slightly more boring stuff is totally, is totally real. But the kind of Darren Brown thing where you like, just show someone one phrase and it totally changes their, their behavior. And I think Darren Brown actually like, picked up on a lot of this stuff and put it into his magic act, you know. I don't think he was doing priming either. I think he was doing magic tricks, which is totally fine. But like, he claimed that he was doing, you know, this kind of putting an idea in your head sort of thing. That stuff is completely fallen by the wayside. I think that's, that's, you know, you don't see new studies on that kind of thing anymore, very much anymore. Because there were some very prominent attempts to replicate that, including the one with

• 42:37Speaker 00

  the people walking more slowly out the lab. They actually got, instead of people with stopwatches, who by the way, knew what the hypothesis of the experiment was, they put laser like, like infrared things on the corridor.

• 42:50Chris Williamson?

  Like a hundred meters.

• 42:51Speaker 00

  Yeah, exactly. So that they would break them as they walked through and there was no difference. When you, when you do it objectively like that, there's no, there's no difference in the people who'd seen the old words versus not.

• 43:00Chris Williamson?

  Growth and fixed mindset.

• 43:02Speaker 00

  Yes. Well, there's another one, which I think the story of growth and fixed mindset is a bit different in that it's not complete bollocks in the sense that the, the priming stuff was, but it was massively overplayed. And I think the over, the, the, the real, you know, if you look at the original discussions of growth mindset, early 2000s, maybe late 90s, early 2000s, this was, this is going to change your life. Change the way you talk to your partner, change the way you, your kids do at school, change the way you think about your life. It's going to be massive. It's going to have an enormous effect. They published a paper in Science, so like, what's meant to be the second best journal in the world, saying that growth mindset could solve the Israel Palestine peace process. Like,

• 43:47Speaker 00

  I'm not kidding. That's a paper that exists and it published in Science. And it's all like, very borderline statistic results, statistical results, that sort of stuff. I actually, having now remembered that, I feel like I should go back and do a little, do a little debunking of that, because I've not seen anyone talk about that. But

• 44:04Speaker 00

  And over time, what you now find is that the people who are doing research on growth mindset claim things like growth mindset is a cheap intervention that can have a small to modest effect on kids' educational attainment. That is, if you go into classrooms, you teach kids, working hard is good, you can change the way that you, you know, you can change your level of ability, you can change your skills, you know, then that that does seem to have a small effect on on kids' behaviour. As I think I would predict it would, right? Like, that makes sense. If you tell kids, you're completely stuck, there's nothing you can do, then, you know, I can totally imagine that many of them will, you know, will will take that to heart and and and it'll at least have a small effect on them. There seem to

• 44:47Speaker 00

  be slightly bigger effects on kids who are from very low income backgrounds, for instance. So that's that makes that makes sense, too. So like, now the now the claims, they used to be these absolutely dramatic claims about how it would change your life. Now it's like, this should be part of the toolkit in education. And that's good. And that's because people have come along and done meta-analyses of all the research and found that the effects are really, really small on average. Like, the effects are not earth shattering effects. They're like, just about there, sort of thing. And and that's much better. But I haven't seen anyone say, oh, sorry, we misled the entire world for decades about about, you know, how much of an effect these things will have. I feel like this kind of thing might be being used in a few schools now, if they had talked about it on the level

• 45:30Speaker 00

  all the way through. But it's now it's in every school in in the country, because the claims being made for it were were revolutionary, when in fact, the data just didn't back that up at all.

• 45:39Chris Williamson?

  I had David Robson on to talk about the expectation effect. Have you seen it?

• 45:43Speaker 00

  I mean, I'm aware of the the sort of effect. I don't know. Is there is that book?

• 45:48Chris Williamson?

  Yes. Yeah, yeah. So he wrote a book, science writer, very good, really, really good. And I would highly recommend you go and check it out. And so he's talking about basically the placebo effect, but across multiple different sorts of domains, right? The expectation that you have about your outcomes can very heavily influence the outcomes. My two favourite studies from that, one of them was talking about.

• 46:08Chris Williamson?

  Gluten intolerance has 10xed from 3% to 30% over the last 20 years. So they brought people into a food hall and they put them under study settings. Some people did and did not have self-reported gluten intolerances. They told everyone that they were going to give them a meal which had gluten in, had no gluten in. People were breaking out in hives, running to the toilet with diarrhea, they had inflammation, all of this stuff. The other one was a study about VO2 max tests. It turns out that a particular type of genetic variation allows you to blow off CO2 and upregulate oxygen more efficiently. People were brought in, they were split randomly into two groups that were mixed with do and do not have the genetic variant. One group was told you do, one group was told you don't, you should do really well, you should do really badly.

• 46:51Chris Williamson?

  Surprise, surprise, the group that was told that they should do badly did worse overall. However, what they found was that people in the group that were told they would do worse, but did have the correct genetic variant to blow off CO2, didn't do as well as the people that didn't have the genetic variation but were told that they should. And David's synopsis for this was, your expectations are more powerful than your genes. Now he's doing it as a little bit of a tongue in cheek sort of play. My point being, you have the opportunity with the expectation effect to create a placebo sense of what may be the outcome. Is that justified in jazzing up some sort of effect?

• 47:36Chris Williamson?

  And is there maybe an argument to be made that the more sexy that you make it, the more outlandish that you make it sounding, it does end up becoming a self fulfilling prophecy? You do end up getting people who believe in ego depletion, I'm sure that you've seen this. People that believe that willpower is a limited resource have limited willpower. People that don't believe that that's the case seem not to. I'm not sure how replicable that is.

• 47:57Speaker 00

  Well, I think, yeah, I think there's been some controversy over that replication. But yeah, no, I get the, I get the, the, the idea.

• 48:02Chris Williamson?

  Is there a justification for people jazzing up any kind of psychological tool that you may be able to use? Because by doing that, you increase the expectation effect, which does genuinely increase the effect. The only issue is that it's not happening on the mechanism that it's claimed on.

• 48:18Speaker 00

  Yeah, I mean, well, it's not just psychological stuff. I mean, if you look at like the whole term of bedside manner that doctors use, you know, doctors can have a big effect on how their patients do by just. Being nice to them or acting in a way that acting in a way that that's not like, you know, really harsh or accusatory or whatever. And I think any doctor will tell you that that's the case. However, I think we really like the most important thing as scientists that we that we have. When you're doing an intervention in a school and you're a teacher, I think jazzing it up, whatever, you know, carry on. If it helps, that's completely fine. But as scientists, our number one thing is to try and get to the truth.

• 48:59Speaker 00

  And and, you know, we need to completely like control as rigorously as possible for these kind of effects. And that's really hard in a behavioral context. It's really hard to make the control group feel like they're doing something similar to the the kind of active treatment group. It's really, really difficult. They're maybe not having exactly the same kind of intervention. They're if it's one of those video game studies that I mentioned before, there may be, you know, there were loads of studies published where the control group was just people who didn't do anything versus people who were playing the video game. And what you want is someone to be playing like a different video game, which you don't think has the active ingredient of like the brain training or whatever it is. But that's hard. And it's so hard that when you look across studies, you find that tons and tons and

• 49:43Speaker 00

  tons of studies, even, you know, randomized medical trials, don't really have very good controls, whether it's like a placebo pill or whether it's just, you know, treatment as usual or whatever. People don't put as much effort as they should into these controls. And I think I think you're absolutely right that there are these there are these big expectation effects. You've got it. You've got to try your best to rule them out because it's a big source of bias in these in these kind of studies. Just reading a meta analysis on homeopathy right now that claims homeopathy helps people's ADHD. I'm just writing a blog about it, which is a very interesting case of a meta analysis, by the way, which does everything right.

• 50:22Speaker 00

  Pre-registered, it had a bias check, it had a publication bias check, it did, like it was all, everything was all done, you know, by the book. And yet, obviously, it's bollocks enemy because it's homeopathy. So, so it, so it, like, you can follow all the rules and still, and still have, and still not have a realistic result. But, one of those rules that people need to follow a lot more. And when people look at the quality of trials, medical trials over the years, they do find that the quality has been increasing somewhat, you know, since the, you know, latter half of the 20th century. People are getting better at doing controls. People are getting better at blinding. People are getting better at doing randomized studies now. So like, if you look at a randomized controlled trial now, on average,

• 51:07Speaker 00

  it will be better than a randomized controlled trial from like, you know, the 1980s or even 1990s, a little bit, on average. That's, that's what, that's what the best data shows, as far as I'm aware. But, but yeah, these expectation effects still come up in, you know, in loads of different, in loads of different trials.

• 51:23Chris Williamson?

  So I've just realized, my question was, should we not be telling the public something which can benefit them by using the expectation effect to fill in gaps in the magnitude of something that's maybe quite small? What you're saying is that science needs to protect itself from this by using unbelievably quarantined,

• 51:44Chris Williamson?

  controls, making sure that this doesn't impact because the expectation effect can be such a big deal. Yeah, I mean, this, in that case, it works in both directions. You can influence publicly what people think. All right, what the other one that I saw, which I haven't spoken to anybody about yet, and you're the guy. What happened with the recent SSRI and depression stuff? Because that demonstrates, I think, quite nicely how important it is to get this right because you're potentially going to commit.

• 52:14Chris Williamson?

  Tens of millions, hundreds of millions of people perhaps, over several decades to a particular course of treatment that may be based on something that isn't legit. Have you looked at this?

• 52:24Speaker 00

  Yeah, it's funny because it relates back to the homeopathy thing I just mentioned because in that homeopathy meta-analysis, they say, well, look, we know that there can't be any actual, you know, molecules of the substance left in the homeopathic remedy because it's been diluted so many times that there aren't any molecules of whatever, you know, active ingredient left. But the randomized controlled trials show that there is overall an effect and therefore we don't need to know the mechanism. We just know it has an effect, right? And I think that they are wrong about that. It was interesting that they said that because under most circumstances, I would actually, that argument sounds quite good to me, just not in the case of homeopathy, which obviously

• 53:06Speaker 00

  is literally impossible for it to be having an effect, given the laws of physics as we know them. But the SSRI thing is interesting because they're making a similar argument there. They're saying, so this paper that came out recently from, I think it was University College London, said when we look at, you know, so the theory underlying SSRI antidepressants is that they allow, you know, for differences in the amount of serotonin that people have in their brains and that might affect their, their, their depression levels. They mean that there's more serotonin just going around in there. And it was always a bit of a vague theory. And I think most researchers would say that the kind of chemical imbalance theory is

• 53:51Speaker 00

  a very, very high level, like, it's a thing that maybe doctors will say to their patients, but it's not actually that justified by the science. And indeed, this study kind of confirmed that, which is that there's no obvious differences in the amount of serotonin in the brain of people who are depressed versus people who aren't. Now, if you think back to the homeopathy thing, then it's like, well, okay, the mechanism isn't there, maybe the mechanism that we think, you know, has an effect, isn't there.

• 54:22Speaker 00

  But if you look at SSRIs, they're not like homeopathy in that they have big side effects. People who have SSRIs have a huge range of different side effects. So there is something active going on in there. And it could be that they're having an effect even if not via the serotonin. So if you look at the studies on antidepressants, my friend Soloni Datani did a great series on, you know, the website Our World in Data. Oh, yes, I do. Yeah, you've probably, like, everyone's seen the graphs from that website and stuff. But they publish a whole lot more interesting stuff. She did a whole series on antidepressants and the research. And her overall conclusion was, and this is broadly my conclusion as well, when I looked at literature, is, yes, the effects are exaggerated in the literature.

• 55:06Speaker 00

  So if you look at the way that antidepressant studies are published, there was actually a really terrifying study done on antidepressants. And I think this applies to almost all areas of science, which is, they took registered trials. So when you do a medical trial, you've got to register with the government, right? And that's just a kind of, legally, you have to do it. Otherwise, there's no way you'll ever get it published. So there's registries, and they have like, all the trials that have ever been done on them. And in this particular selection of trials, they found that it was about 50/50. So about half of the trials, the antidepressant didn't work, and half the trials it did. But by the time it got to the studies that actually got published in papers, almost all of the negative studies,

• 55:50Speaker 00

  just no one ever sent them for publication. But almost all of the positive studies, people did send for publication. Then, if you look at those, the few negative studies that actually got published, they kind of shifted the outcome a little bit. They said, well, we were going to measure things on this depression measure, but actually, we didn't find a result on that. So we'll just, we'll talk about a different depression measure, which we did find a result on. So this is like, you know, dredging through the data to find any old thing. Then, if you look even closer, you find that even some of the negative ones were kind of written up in a slightly spin sort of way, like a positive sort of way, saying like, well, this is very promising. And all this, when actually, it was just a null trial. And so the literature that we see at the end of this kind of process, and it's almost like

• 56:34Speaker 00

  a laundering process of the literature, doesn't bear much relation to the studies that actually were done, which is really scary. And I think that happens across all areas. They actually found in that same paper that it applies to therapy trials as well, not just antidepressant trials. So, I mean, that... Is like a really, really terrifying discovery. Having said that, I still think that if you adjust for that kind of thing, the publication bias and so on. It is the case that antidepressants do seem to have a very, like a small effect on on people's levels of depression. I don't think the mechanism matters in that case. I think even if it's not to do with serotonin, they seem to be doing something and the randomized controlled trials do seem to show that.

• 57:17Speaker 00

  But then there's like massive interpretational differences in in how you look at the numbers. So Ervin Kirsch, the famous critic of antidepressants, published a book called The Emperor's New Drugs, I think it's called, about antidepressants. And he said, an effect size of point two is the average that you get out of studies of antidepressants. So like point two standard deviation difference in the depression score. What a tiny pathetic effect. You've got to give up these drugs, they don't have an effect, that's not going to work at all. You look at the, you know, most recent meta-analyses that are published that account for things like I was talking about, the publication bias. And they say, effect size of point two, effective medical treatment.

• 58:00Speaker 00

  This is really good, you know, even though it's not a massive effect, this is still going to have a bigger impact across the population and all sorts of stuff. So people can look at exactly the same data and draw massively different interpretations. My interpretation is, we've got to do a bit better on this. We've like, we've got to do proper research and really understand this and get better at treating depression, both from the therapy angle and the drug angle. But at the moment, it does look like on average, for each person who uses them, like one type of antidepressant will have some kind of mild effect, at least on their depression. Now, I think one of the big things that hampers depression research is that we're not very good at defining what depression actually is.

• 58:44Speaker 00

  And so there's loads of research on whether. You know, we've already talked about there's this thing called intelligence, right? And intelligence doesn't exist in the sense that there's not like a thing we can measure with a ruler in the brain or in like that, that is intelligence. We infer the existence of intelligence from the fact that there are all these different tests that you give people and they correlate positively together. And there's this thing that comes out, this latent variable that comes out and it's called intelligence. That explains, you know, half the variation in the tests and all this sort of stuff. And so the question is, you know, is there something called depression? That's the same, that's the same thing that we can infer from all the symptoms? Insomnia, low mood, just crying sometimes for no reason.

• 59:27Speaker 00

  You know, all, all the different things that come with, with depression. Is there this thing called depression? And the argument from quite a lot of people, Aiko Fried is one, who's a, a, a, a, someone that I've known for a long time, a researcher who's done really good research on this. His idea, his, his idea is that there isn't necessarily this thing called depression. And what we should be focusing on is the symptoms. We should be focusing on this kind of network of symptoms, which sometimes bump into each other and cause each other. So like the insomnia causes low mood and the low mood causes you to be angry at your spouse and that causes you to, and, and so there's all this kind of like network and you can try, there's very statistical approaches you can try to understand this kind of network

• 1:00:12Speaker 00

  of effects. And that's a different way of looking at things than saying that there is this thing called depression that causes all the symptoms. This latent variable, which we'll never be able to measure that's, that causes all the symptoms. I'm kind of undecided on this debate. I think, um, it, it seems quite a, quite a strong thing to say that there's no such thing as depression because like a lot of people seem to have very similar symptoms over, over time. And, um, the general medical approach is to try and treat the underlying cause rather than the symptoms. And, and so this would be quite a, uh, this would be quite a departure from that. But I think the, the, um, there's a lot of mileage in this kind of research of like examining the symptoms rather than saying, um, you know, there's this one thing which we, we have to try and treat. So, um, you know,

• 1:00:58Speaker 00

  That might be one of the reasons why the trials on antidepressants are are kind of all over the place is that they all measure depression slightly differently. People experience depression slightly differently. People are at different phases of depression and the symptoms might be different. And if one symptom causes another, then in different ways and different people. So it becomes like a moving target rather than, you know, I don't know, if you were doing a drug of, if you're doing a test of like, do you have the COVID virus in your system, which is, you know, yes, there's difficulties in some ways of measuring that. But like, it's a much more objective thing than, you know, are you depressed? I mean, there's many different ways you can you can answer that question.

• 1:01:34Chris Williamson?

  That seems very much like a behavioral geneticist's answer. I spoke to Plomin about a year ago and he was telling me, he says he has a predisposition for being fat. And he said, it was such a great example and it really helped to identify how this collection of genes contributes to the outcomes that we get in life. So he said that whenever he walks past a bakery, if he smells fresh bread, he will be going in and he will be buying it and he will be eating most of it. But there are many ways to get fat. You may be fat because you have an aversion to exercise. You may be fat because you have downregulated ghrelin release so that the hormone that tells your stomach that it's empty, that signals to you to eat, maybe that's upregulated or

• 1:02:19Chris Williamson?

  some, you know, there's a million different ways that you could do that. Or for you to be fit, maybe you don't sleep so much. So you're always up early and because you're up early, it means that you go and exercise because you've got nothing else to do, right? There are tons and tons and tons of different things. And the same thing goes for what you're talking about here. What you talk about when you're referring to depression is a particular milieu of a bunch of different things that people seem to link together. Where do they come from? Is everyone's depression the same? You're never going to actually get to experience somebody else's brain. So you use these words to describe, but we all know how culturally influenced a lot of the things that we consider are, right? We use the language that other people have done. It's precisely why juries aren't allowed to or told not to watch the news while they're

• 1:03:02Chris Williamson?

  doing, for precisely that reason, to not try and influence their opinion. So it does make a good bit of sense. I also think, is it, what's it called? The Richmond scale of depression and it's naught to 61 or something like that, or the Richard scale?

• 1:03:15Speaker 00

  There's the Beck depression inventory and there's a whole, there's a whole like.

• 1:03:19Chris Williamson?

  Litany.

• 1:03:20Speaker 00

  Tests you can use. Yeah.

• 1:03:21Chris Williamson?

  Yes, and I'm pretty sure that no matter what the mechanism is that SSRIs could be working on, it seems like they get to move someone about a point on one of the scales that I seem to remember reading about.

• 1:03:33Speaker 00

  Yeah.

• 1:03:33Chris Williamson?

  And you go, well, a point isn't nothing, but it's a big difference if it's the one between taking your life and staying alive.

• 1:03:39Speaker 00

  Sure.

• 1:03:39Chris Williamson?

  So saying that there is no use for them, that seems like baby and bathwater being thrown out, but saying that it's this effect, but then also going, well, serotonin does seem to impact people's subjective sense of wellbeing and how they actually, so if it is like.

• 1:03:53Speaker 00

  It's a, it's a mess. It's a mess. And a huge amount of that mess comes from low quality research, like low quality studies, low quality antidepressant trials, low quality research on depression more generally, people not thinking about alternate explanations of the, the, the data and kind of getting stuck in, in particular paths. And I think this is a broad problem across all research. But yeah, you're completely right that this is, this, this, this becomes like a real like thicket of, of, of, of complication. However, I do, I feel uncomfortable going too far down that particular line because like for many years, anti psychiatry, the anti psychiatry movement has made like these similar arguments.

• 1:04:36Speaker 00

  And I feel, I feel like an aversion to saying, well, maybe they were kind of right in some, in some, in some respects because, you know, they are, they are, I mean, I was about to say they are insane. I mean, the, the, that maybe that's not an insult to them because they don't think there's such a thing. But, um, you know, I, I live, um, I, so I, I work at the, the Maudslay Hospital in, in, in, in London, which is a psychiatric hospital. Our, our campus is like behind the, the psychiatric hospital. And, you know, every so often there's Scientologists out there protesting because they have this big anti psychiatry thing. So they're really worried about.

• 1:05:11Chris Williamson?

  What's the problem with psychiatry?

• 1:05:14Speaker 00

  Well, L. Ron Hubbard didn't like it. I think possibly because he saw a psychiatrist and the psychiatrist said, "You need help." And so he was, and so he was.

• 1:05:24Chris Williamson?

  And now downstream from that, the echoes of L. Ron Hubbard's bad interaction with a psychiatrist are still.

• 1:05:29Speaker 00

  That's my understanding. That's my understanding. And I think, and I think, you know, they are trained from day one that like psychiatry is the evil thing and psychiatrists are out to, to manipulate us, control us and shock our brains.

• 1:05:43Chris Williamson?

  Imagine that. Imagine that. But trying to manipulate and control us.

• 1:05:47Speaker 00

  But, but, yeah, particularly psychiatrists. It's very, very strange. But they were particularly, they were, they were campaigning about electroshock treatment, which is another like interesting thing, which like, my understanding is, if you talk psychiatrists, that for very, very, very severe cases of depression, where people are like literally catatonic and can't move from, from depression, electroshock, electroconvulsive therapy, I think they call it, does, does help. It can kind of jolt people out of these kind of catatonic states.

• 1:06:19Speaker 00

  However, we don't really know that much about it. And certainly the long-term effects, it's, it's unclear. I saw a meta-analysis that looked a bit, a bit shaky in terms of its statistics. I'm going to write something on this soon. Those Scientologists have like inspired me to like, look into this. But what I don't think is, is like a terrifying cabal of psychiatrists who are trying to control the world. I think it's, I think it's like, it's been observed anecdotally, at least in many cases, that this helps. And so, and so people, people use it. But yeah, you know, there was a whole, you know, through the 1970s,

• 1:06:53Speaker 00

  80s, this whole movement of, you know, there's no such thing as schizophrenia. It's just a bad adaptation to the societal conditions that people get, get, get in. No, no. There's this thing called schizophrenia. It's really bad, like psychosis at least. There's this thing and it's really, really bad and people lose a complete control of their grip on reality. And, and, and, and that's a really serious thing and it's not to do with society. It's something, you know, very, that's gone very, very wrong in their brains. And, and that's another thing that we don't know how to treat that well. I mean, we've got drugs, antipsychotics that can, can, can kind of hold it, hold it back, but it's difficult to know how to sort of prevent or predict it.

• 1:07:33Chris Williamson?

  Kind of like the broken clock is right twice a day scenario. It seems like if you have a replication crisis and a lot of studies and previously held models are up for the chopping block, there will be some people in the past that decided to point at that. And that, that ends up making them seem like Cassandra's, right? Very effective ones. So given, given the fact that you've got this thicket, as you put it, which is a complete mess, even for people who, like you, can read scientific papers, who, like you, understand how effect sizes and P hacking and all of this stuff can be done. And coming out the

• 1:08:12Chris Williamson?

  back of two years where everybody had to be a closet epidemiologist, virologist and understand you've got the media perverse incentives, clickbait, you've got academic issues with regards to the only stuff that gets published is the stuff that is set, all of this stuff, all put together. First off, it's not a surprise that people are losing faith in the powers that be, in the ones that they should have done. The bigger question, I think, is how people can be effectively skeptical without losing faith in everything and becoming confused and nihilistic and easy to manipulate.

• 1:08:51Speaker 00

  I've just written the start of a book proposal on exactly that question of like, how do you question the consensus without like losing your mind and becoming a conspiracy theorist? And

• 1:09:05Speaker 00

  I think this is a really difficult task because the incentives are so strong. Once you see that the scientific establishment is really screwed up on something like the replication crisis, or you know, you mentioned during the pandemic, there were all those cases of studies that came out that were fraudulent, papers having to be retracted from the Lancet because they were based on entirely fraudulent data, like there was a study on hydroxychloroquine that had to be retracted, and the Lancet were like, well, from now on, we're gonna, literally, this is what happened, they said, from now on, we're gonna make sure that at least one of the reviewers of the paper, one of the peer reviewers, has expertise in the topic that they're reviewing. It's like, whoa, you didn't do that before. So, you know,

• 1:09:47Speaker 00

  I can, like, I can, I understand why some people are like, holy shit. But I think part of it is that we need to raise our standards in general, like, we need to, first of all, we need to have some standards, rather than just accepting stuff that someone on our side says, which is a massive temptation for all of us. There's someone we trust, they said something, so we're like, oh, yeah, I'm sure that's, I'm sure that's true. And that's kind of my argument in the previous book, in Science Fictions, is that, like, if you raise your standards, then you won't become a homiopath, or a conspiracy theorist, or whatever, because, because the standards that you have are, you know, will, will just kind of, you know, all that stuff will get chucked out as well. It's just that you'll also chuck out lots

• 1:10:27Speaker 00

  of really crap, you know, mainstream science, too. So I think there's probably like, a good checklist that can be written for like, how you should read papers, and I kind of sketch one of them out in the previous book, but I think I'm gonna do a book-length treatment of that, like, what to look at when you're reading the scientific evidence. But even then, like, you can't be an expert in every single area, and it takes, it takes expertise in each area. So I think what we can do is encourage, like, a free and open and, and to be honest, as aggressive as possible, debate on scientific research. So there's like, there's organizations out there, like the Science Media Center, for instance, and I would encourage anyone, if they see like, a controversial paper in the news, take a

• 1:11:09Speaker 00

  look at the Science Media Center website, because what they do is, they ask a whole bunch of scientists who are unaffiliated with the study itself, what they think about it. And so you'll get, you know, six or seven responses, and sometimes they say, this study seems perfectly good, but regularly, they say... This has been way overhyped. There's a major problem with this study.

• 1:11:29Chris Williamson?

  You know what would be an amazing website would be rotten tomatoes, but for psych science studies.

• 1:11:36Speaker 00

  Yeah. Well, there's already a website called Pub peer. So like publication peer, whatever pub peer, which which tries to do that exact thing. So they've got a little bot that goes through Twitter and finds like threads that people write about papers and post them under a link to that paper on the website. So if you, again, another thing which people might be interested in doing is if you find a paper that looks dodgy in some respect, put it into pub here and see if there's any discussion. And that's that's pretty much what you're talking about. The only problem is there's so many papers out there and nobody has the time to kind of look into them all. And there's probably not discussion on every single paper that you want there to be discussion on. But another thing they do on pub here is look for fraud. So they'll say,

• 1:12:17Speaker 00

  do you think that the microscope picture in figure three looks like it's been photoshopped? And it turns out that in many cases it has. There's this incredible, she calls herself scientific integrity consultant called Elizabeth Bick, who goes through thousands of biological papers and finds that scientists, you know, they don't just manipulate data when they do fraud, but they manipulate the pictures as well, whether it's like blots or microscope images, all that sort of stuff. So many of them have been like duplicated, retouched, recolored, cropped, all that sort of stuff to show the results in a much better light. And that's scientific fraud, but it happens in thousands of papers every year. And peer reviewers look at it and go, nope,

• 1:12:56Speaker 00

  looks fine to me. And it gets passed on. And so this whole community has to come out and re-review the papers. And I think encouraging that sort of thing, exactly as you say, websites that do this kind of thing, that that that collect together reviews of papers, collect together people's opinions. Another thing scientific journals can do, and some of them are doing this now, including some of the top ones like nature, they're publishing the peer reviews alongside the paper. So you can see what people thought of it when they first saw it. And it's and in some cases, including a case I saw recently of a big, you know, this genetics paper that claimed to like revolutionize genetics, when in fact I would say it didn't at all. Like you can see that the reviewers said, no, no, no, there's a big problem here. The control condition doesn't work.

• 1:13:42Speaker 00

  And the editor just was like, well, better publish this anyway and and just like completely ignored what the reviewer said. So so but but we can see that. That's obvious now. We wouldn't have been able to see that back back before they published reviews. Science was just done in a complete black box. So there's this movement towards open science that is publishing the peer reviews online, publishing the data set online. So anyone can can go in and dig into it and have a look and making sure that the paper is open access. So anyone, even if they don't work at a university and they don't want to pay for the paper, can can click it and read it. Making sure that the materials are online. So other scientists can come along and replicate it in exactly the same way. A big problem is that scientists read each other's papers and they don't even know where to begin to to to start replicating the paper

• 1:14:26Speaker 00

  because they they're like, oh, there's there's actually there's no description here of of what they did in this particular condition. So I'm going to have to spend months emailing back and forth with the scientist like, what is the point of scientific papers if they don't actually describe what what you what the, you know, what was done in the experiment? Well, it turns out in in many, many cases, they don't. So, yeah, there's this, there's this movement of open science. I think one sort of like rule people could have in their heads is like, if a paper looks like it's open and transparent, they've published their data online, everything's open and clear, they've registered their plan, like before they touch the data, they've actually like registered the plan of what they're going to do with it. So they don't like fuck about with it when they when they when they get the data and like, oh, well, I'm sure that participant wasn't paying attention or that

• 1:15:11Speaker 00

  person, let's cut out everyone above 50, even though that wasn't the plan. But let's cut out everyone who's above 50, because I think probably older people have a different reaction to this drug. This sort of thing happens all the time. And you can kind of justify it to yourself like, yeah, yeah, it is the case that older people have a different reaction to this drug. So let's just cut them out altogether. Oh, lower the whole, we found a significant result. Let's send that off for publication like, and it wasn't that case. It wasn't that wasn't the case before they did this. So having a clear plan and sticking to it, posting your data online, probably most frauds are not going to post their data online because they're scared that there'll be some like obvious thing in the data, like number seven comes up too often or something like like and they and they like, so it doesn't seem like a real data set. Those are kind of things that you

• 1:15:54Speaker 00

  can look for. But I also think seeking out other scientists' opinions and and something like the Science Media Center is really useful for that. But also just like looking on Twitter and seeing what. People have said about a paper like, okay, you'll get lots of uninformed takes on the paper as well. But, you know, seeing what people said, you don't necessarily have to agree with it or have to take it as gospel. But, you know, seeing that, okay, actually, maybe tweets one to five of this thread are kind of like, are garbage. But tweet seven actually does illustrate that there's a problem in table three of this paper. So I better just, just, you know, adjust my certainty about it somewhat. So like, there's that, that sort of thing. And, you know, that's what I'm doing when I'm like, writing about low quality papers. I'll read the paper and have my own critique of it, but I'll also see what other people have said about it.

• 1:16:37Speaker 00

  Because the whole point of having a scientific community is that we all like criticize each other all the time. And one of the big problems is that we've got to the point where it can be very socially awkward to criticize each other. Some fields, this is much better than others. In psychology, as much as I enjoy going to like psychology lectures and so on, everyone is generally quite nice to each other. And they're like, oh, that's a great paper. Well done. They start off like my friend Chris Chabri pointed this out recently, like often the person who's chairing a seminar will start the question and answer session by saying, that was a brilliant talk. Thank you so much for that. That sets the wrong tone because the tone you actually want is, okay, thanks for giving us the talk. Does anyone want to critique that, you know?

• 1:17:22Speaker 00

  And in economics, I've done, I've only done this once. In economics, I went to economics seminar thing and I was told I was doing a talk, but I wasn't talking about my own stuff. I was to talk, I was given someone else's data set who would be, that was presented earlier that day. And I was told to just like, check it, critique it and do my own like mini talk at the end of his talk that was critiquing it. That's the sort of thing we want now. Economics seminars can get a bit like macho, aggressive, like that sort of thing. And that's, you know, can, can, can distract from the, the actual content, which is what we want. But that sort of level of like constant skepticism, constant argument is what science is all about. And I think we lose that if people are basically too nice and too trusting of each other.

• 1:18:07Speaker 00

  And those are like being nice and being trusting. They're good things that we want to encourage, but maybe not so much in science.

• 1:18:15Chris Williamson?

  Stuart Ritchie, ladies and gentlemen. If people want to find out the stuff that you do online, where should they go?

• 1:18:20Speaker 00

  I'm on Stuart J Ritchie on Twitter. I'm also StuartRitchie.substack.com for all my longer writings. And my book is Science Fictions, came out a couple of years ago.

• 1:18:29Chris Williamson?

  Amazing. Stuart, I appreciate you. Thank you.

• 1:18:31Speaker 00

  Thanks so much.

• 1:18:32Chris Williamson?

  Mr. President, my company foreme of the explosion on our rig on October 2nd.

• 1:18:36Speaker 02

  Causing massive water displacement and triggering a tsunami. They all keep the blame on me. Tom Cruise is. Digger got us into this, that's what. And Digger's gonna dig us up. From the director of Birdman and The Ravenant.

• 1:18:49Speaker 03

  What would happen if I stopped doing it? You stall, you freeze to death, that's what.

• 1:18:53Speaker 02

  With great power comes great deniability.

• 1:18:56Speaker 03

  I'm not the first then.

• 1:18:57Speaker 02

  Digger, see the trailer online, only in theaters October 2nd. Rated R, under 17, not a middle without parent.