Bad Medicine, Part 1: The Story of 98.6 (Rebroadcast)

We tend to think of medicine as a science, but for most of human history it has been scientific-ish at best. In the first episode of a three-part series, we look at the grotesque mistakes produced by centuries of trial-and-error, and ask whether the new era of evidence-based medicine is the solution.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Stephen J. Dubner?

  Freakonomics Radio is sponsored by Southern New Hampshire University. The best investments are the ones you make in yourself, and no one knows that better than Southern New Hampshire University. With more than 200 online degree programs, no set class times, and low online tuition, Southern New Hampshire University is built to fit your life today and set you up for success tomorrow. Education is an investment. Why not make it a smart one? Apply for free at SNHU.edu/freakonomics. Freakonomics Radio is sponsored by Progressive. Whoever says saving money is hard doesn't know that new customers who bundle and save with Progressive save hundreds because switching your home and auto insurance is better than

• 0:43Stephen J. Dubner?

  cutting down on the things you love, like those iced lattes that help you get ready to face the day. So switch your home and auto to Progressive and see if you could save and keep sipping away. Get your quote today at progressive.com. Progressive Casualty Insurance Company affiliates and other insurers. Not available in all states. Freakonomics Radio is sponsored by Mint Mobile. With Mint Mobile's summer sale, all of Mint's plans are $15 per month, even unlimited. Free yourself from big wireless and switch to Mint for high-speed data and coverage on the T-Mobile 5G network. To get your new wireless plan for just $15 a month, go to mintmobile.com/freak. That's mintmobile.com/freak.

• 1:28Stephen J. Dubner?

  Upfront payment, $45 for three months, $90 for six months, or $180 for 12 months. Plan required. $15 per month equivalent. Taxes and fees extra. Initial plan term only. Greater than 50GB may slow when network is busy. Includes up to 20GB hotspot. Capable device required. Availability, speed, and coverage varies. See mintmobile.com.

• 1:56Stephen J. Dubner?

  Freakonomics Radio is sponsored by ChatGPT. ChatGPT Work is a new way to work in ChatGPT. ChatGPT Work can pull info from across your apps and connect to your tools to help you create finished work. You can even automate work in ChatGPT with the Scheduled Tasks feature, and the whole time, you stay in control. Put ChatGPT to work on your most ambitious ideas and projects. Get started at chatgpt.com by selecting Work Mode, available on Plus and Pro plans.

• 2:31Stephen J. Dubner?

  Hey there, Freakonomics Radio listeners. We are taking advantage of August to replay you a special three-part series we did last year called Bad Medicine. Today, Part One, The Story of 98.6. And it starts right now. We begin with the story of 98.6. You know the number, right? It is one of the most famous numbers there is because the body temperature of a healthy human being is 98.6 degrees Fahrenheit, isn't it?

• 3:05Speaker 01

  So now I'm going to take your temperature. If you don't mind, just open your mouth and I'll insert the thermometer. Perfect.

• 3:14Stephen J. Dubner?

  The story of 98.6

• 3:16Speaker 02

  dates back to a physician by the name of Carl Wunderlich.

• 3:21Stephen J. Dubner?

  This was in the mid-1800s. Wunderlich was medical director of the hospital at Leipzig University. In that capacity, he

• 3:29Speaker 02

  oversaw the care and the taking of vital signs on some 25,000 patients.

• 3:36Stephen J. Dubner?

  Pretty big dataset, yes? 25,000 patients. And what did Wunderlich determine?

• 3:42Speaker 02

  He determined that the average temperature of the normal human being was 98.6 degrees Fahrenheit or 37 degrees centigrade.

• 3:55Stephen J. Dubner?

  This is Philip Makowiek, a professor of medicine and a medical historian at the University of Maryland.

• 4:01Speaker 02

  Well, I am an internist by trade and an infectious disease specialist by subspecialty. So my bread and butter is fever.

• 4:11Stephen J. Dubner?

  There's one more thing Makowiek is.

• 4:14Speaker 02

  I am by nature a skeptic, and it occurred to me very early in my career that this idea that 98.6 was normal, and that if you didn't have a temperature of 98.6, you were somehow abnormal, just didn't sit right.

• 4:33Stephen J. Dubner?

  Philip Makowiek, you have to understand, cares a lot about what is called clinical thermometry. And if you care a lot about clinical thermometry, you care a lot about the thermometer that Carl Wunderlich used to establish 98.6.

• 4:49Speaker 02

  His thermometer is an amazing key to this story of 98.6.

• 4:55Stephen J. Dubner?

  So you can imagine how excited Makowiek was when, on a tour of the weird and wonderful Mutter Museum in Philadelphia, the curator told him they had one of Wunderlich's original thermometers.

• 5:09Speaker 02

  I said, "Good heavens, may I see it?" And she said, "Sure, would you like to borrow it?" And I said, "Of course." And so I was able to take this thermometer back to Baltimore and do a number of experiments.

• 5:23Stephen J. Dubner?

  The Wunderlich thermometer, Makowiek realized, was not at all a typical thermometer.

• 5:28Speaker 02

  First of all, it was about a foot long, fairly thick stem, and registered almost 2 degrees centigrade higher than current thermometers or thermometers of that era.

• 5:42Stephen J. Dubner?

  2 degrees higher? Centigrade?

• 5:46Stephen J. Dubner?

  Uh-oh.

• 5:47Speaker 02

  In addition to that, it is a non-registering thermometer, which means that it has to be read while it's in place. So it would have been awkward to use.

• 5:59Stephen J. Dubner?

  Makoviac noticed something else about the original Wunderlich research.

• 6:03Speaker 02

  Investigating further, it became apparent that he was not measuring temperatures either in the mouth or the rectum. He was measuring axillary or armpit temperatures. And so that, in many, many ways, his results are not applicable to temperatures that are taken using current thermometers and current techniques.

• 6:27Stephen J. Dubner?

  As it turns out, the esteemed Dr. Carl Wunderlich

• 6:31Speaker 02

  was not the most careful investigator ever to come on the scene.

• 6:38Stephen J. Dubner?

  The more Makoviac looked into the Wunderlich data and how the story of 98.6 came to be, the more he wondered about its accuracy. So he set up his own body temperature study. He recruited healthy volunteers, male and female, and took their temperature one to four times a day, around the clock, for about two days, using a well-calibrated digital thermometer in the patients' mouths. What did they find?

• 7:03Speaker 02

  Of the total number of temperatures that were taken, only 8% were actually 98.6. And so if you believe that 98.6 is the normal temperature, then 92% of the time, the temperature was abnormal. And obviously, that's not even reasonable.

• 7:23Stephen J. Dubner?

  In his study, Makoviac found the actual normal temperature to be 98.2 degrees, not a huge difference. And yet, the whole notion of a normal body temperature was looking more and more suspect. Why? A lot of reasons. Temperature varies from person to person, sometimes so much that one person's normal would register as nearly feverish for another person.

• 7:48Speaker 02

  It's almost like a fingerprint.

• 7:49Stephen J. Dubner?

  Temperature varies throughout the day. It's roughly one degree higher at night than in the morning, sometimes even more. And an elevated temperature isn't necessarily a sign of illness.

• 8:01Speaker 02

  In women, it goes up with ovulation during the menstrual cycle. The temperature goes up during vigorous exercise, and this is not a fever.

• 8:11Stephen J. Dubner?

  And so, Makhoviat concluded.

• 8:13Speaker 02

  Looking at a rise in temperature as a reliable sign of infection or disease is inappropriately simplistic thinking.

• 8:25Stephen J. Dubner?

  Inappropriately simplistic thinking. It makes you wonder, if the medical establishment believed for so long in an inappropriately simplistic story about something as basic as normal body temperature, what else have they fallen for? What other mistakes have they made? I hope you've got some time. It's a long list.

• 8:53Speaker 03

  You take a sick person, slice open a vein, take a few pints of blood out of them. Drilling holes into people's skulls.

• 9:00Speaker 04

  It was literally taking someone to hell and back.

• 9:03Speaker 05

  And it would cause a whole series of malformations and probably a lot of fetal death.

• 9:08Speaker 03

  Lobotomies.

• 9:09Speaker 06

  The overuse of a mercury compound. The Tuskegee case. Losing your teeth and having your gums bleed. DES and thalidomide.

• 9:16Speaker 04

  We use sort of a cement.

• 9:17Speaker 06

  Hormone replacement therapy. The oxycontin and opioid problem.

• 9:21Speaker 02

  As a medical historian, it is patently obvious to me that future generations will look at what we're doing today and ask themselves, what was grandpa thinking of when he did that and believed that? And they'll have to learn all over again that science is imperfect and to maintain a healthy skepticism about everything we believe and do in life in general, but in the medical profession in particular.

• 9:54Stephen J. Dubner?

  On today's show, part one of a special three-part series of Freakonomics Radio, we'll be talking about the new era of personalized medicine, the growing reliance on evidence-based medicine, and especially, pay attention now, I'm going to use a technical term, we will be talking about bad medicine.

• 10:27Speaker 07

  From WNYC Studios, this is Freakonomics Radio, the podcast that explores the hidden side of everything. Here's your host, Stephen Dubner.

• 10:41Stephen J. Dubner?

  We have a lot of ground to cover in these three episodes: medicine's greatest hits, the biggest failures, where we are now, and where we're headed. In the interest of not turning a three-part series about bad medicine into a 20-part series, we're not even going to touch adjacent fields like nutrition and psychiatry. Maybe another time.

• 11:03Stephen J. Dubner?

  Let's start, very briefly, at the beginning. Nearly 2,500 years ago, you had the Greek physician Hippocrates, who is still called the father of modern medicine. You've heard, of course, of the Hippocratic Oath, the creed recited by new doctors. And you know the oath's famous phrase, "First, do no harm," even though, as it turns out, that phrase isn't actually included in the oath. It came from something else Hippocrates wrote. Nor do many contemporary doctors recite the original Hippocratic Oath. There's a modern version, written in 1964 by the prominent pharmacologist Louis Lasagna. The pledge begins, "I swear to fulfill, to the best of my

• 11:46Stephen J. Dubner?

  ability and judgment, this covenant." It is a fascinating, inspiring document, and I think, before we go too far, it's worth hearing some of it.

• 11:56Speaker 08

  I will respect the hard-won scientific gains of those physicians in whose steps I walk, and gladly share such knowledge as is mine with those who are to follow.

• 12:07Speaker 09

  I will remember that there is art to medicine as well as science, and that warmth, sympathy, and understanding may outweigh the surgeon's knife or the chemist's drug. I will not be ashamed to say, "I know not."

• 12:19Speaker 10

  Nor will I fail to call in my colleagues when the skills of another are needed for a patient's recovery. Above all, I must not play at God. I will remember that I do not treat a fever chart, cancerous growth, but a sick human being, whose illness may affect the person's family and economic stability. My responsibility includes these related problems if I am to care adequately for the sick.

• 12:40Speaker 11

  If I am to care adequately for the sick, I will prevent disease whenever I can, for prevention is preferable to cure. And may I long experience the joy of healing those who seek my help.

• 12:52Stephen J. Dubner?

  It's comforting to think about the thoughtfulness, the nuance, the massive responsibility that doctors pledge before they attempt to diagnose or heal us. How well has that pledge been upheld throughout medical history? We'll talk to a variety of people about that today, starting with this gentleman.

• 13:12Speaker 01

  My name is Anupam Jena. I'm a healthcare economist and physician at Harvard Medical School.

• 13:17Stephen J. Dubner?

  So Jena, as both a practitioner and an analytical researcher, is especially useful for our purposes, because one of the themes we'll hit today several times is that medicine, even though it's scientific, or at least scientific-ish, hasn't always been as empirical as you might think, and sometimes not very empirical at all.

• 13:40Stephen J. Dubner?

  Here's an easy question. Can you tell me, please, the history of medicine, or at least Western medicine, in, I don't know, three or four minutes?

• 13:49Speaker 01

  Let me first answer the meaning of life.

• 13:53Stephen J. Dubner?

  Is that going to be easier?

• 13:54Speaker 01

  That'll take about five to six minutes. You know, I would say, how about three words? Trial and error. So I think if you think about medicine, what's, how it's evolved, let's just say, in the last 100 to 200 years, the sorts of practices that at some point in history, people thought were actually medically legitimate, included drilling holes into people's skulls, lobotomies, even as late as the 1940s to 1950s, lobotomies were thought to actually have a treatment effect in patients with mental illness, be it schizophrenia or depression, the practice of bloodletting, which is basically trying to remove the

• 14:34Speaker 01

  Quote unquote, "bad humors from the body" was thought to be therapeutic in patients. Things like mercury, which we know are downright toxic, were used as treatments in the past. And, you know, that was in a time and place where I think it was very difficult to get evidence, but not only that, there was probably a perception of the field that didn't allow for the ability to question itself. And in the last 50-plus years, probably 50 to 75 years, I think we've seen tremendous strides in the ability of the profession to constantly question itself.

• 15:09Stephen J. Dubner?

  So it's easy to get indignant over the idea of these treatments that turn out to be so wrong, but understanding wellness and illness is hard, obviously. So when you look back at the history of medicine, did those interventions strike you as kind of shameful? You can't believe you're in a profession that tried things like that? Or is that just part of the trial and error process that you accept?

• 15:35Speaker 01

  I certainly wouldn't call it shameful. The only thing that's shameful is when someone doesn't believe that they have the potential for being wrong and they don't have that desire to inquire further about whether something actually works or doesn't work. But the idea of trying things, particularly trying things that have a really strong plausible pathophysiological basis, I think that there's nothing wrong with that. In fact, that's what spurs scientific discovery and many of the treatments that we have now.

• 16:03Stephen J. Dubner?

  So I have a broad question for you. The human body is, I think you and I would agree, an extraordinarily complex organism. And over history, doctors and others have learned a great deal about it. But if we consider the entire human body, from a medical perspective only, let's leave out metaphysics and theology and what have you, from a medical perspective, how would you assess the share of the body and its functions that we truly understand and the share that we don't really yet understand?

• 16:33Speaker 01

  Huh. That's a tough one. We've made a lot of headway, but to put a number on it, I would say maybe 30%, 40% that we don't know.

• 16:45Speaker 03

  Ooh, that's a tough question for me to quantify.

• 16:47Stephen J. Dubner?

  I asked the same question of someone else.

• 16:49Speaker 03

  My name is Jeremy Green. I'm a physician and a historian of medicine at Johns Hopkins.

• 16:54Stephen J. Dubner?

  So what's Green's answer?

• 16:55Speaker 03

  There's a Rumsfeldian answer of known knowns. There are known knowns. Known unknowns. Known unknowns. That is to say, we know there are some things we do not know. There are also unknowns. Unknown unknowns. A different way of answering that question would have to do with what our idea of the relevant science of medicine is.

• 17:13Stephen J. Dubner?

  For example?

• 17:15Speaker 03

  If you take, for example, the moment in the Renaissance, the Vesalian moment, when the opening of cadavers and description and rendering in precise, three-dimensional chiaroscuro engravings of the human body was an exciting area for research, that actually this humanist process of opening up cadavers, showing that the innards were not exactly what the ancient Greeks had described. So as a historian, rather than giving you a fixed percent of where we are, I can give you a Zeno's paradox, that we keep on getting close to that finite moment and then reinvent a new, broader room for us to inhabit.

• 17:58Stephen J. Dubner?

  And that's because there's been a lot of progress in how we're able to explore the human body.

• 18:03Speaker 01

  There's the gross anatomy of the body, which you can see with your own eyes.

• 18:08Stephen J. Dubner?

  Anupam Jena again.

• 18:09Speaker 01

  Then go a layer further, and we're now at the microscopic anatomy of the body. So now, what do the cells of the body look like when they are diseased under a microscope?

• 18:21Stephen J. Dubner?

  And now?

• 18:23Speaker 01

  Now go a layer further, where you are now trying to understand things about the body that you can't even see with a microscope. And that's at, let's say, the level of the proteins in the cell, or even further down, the level of the DNA that encodes the protein.

• 18:38Speaker 03

  By the end of the 20th century, there's a very strong genetic imaginary, which really helps to then fuel the excitement behind the Human Genome Project. It's thought once we know the totality of the human genome, we'll know all that we need to know about bodies and health and disease.

• 18:54Stephen J. Dubner?

  Of course, we already know a great deal. And to be fair, for all the mistakes and oversights in medicine, there's been extraordinary progress. What are some of medicine's greatest hits?

• 19:10Speaker 12

  I'm sure every historian of science and medicine would give you a different set of hits.

• 19:15Stephen J. Dubner?

  That's Evelyn Hammonds. She's a professor of the history of science and African-American studies at Harvard.

• 19:21Speaker 12

  The ones that I typically think about are the introduction of more efficacious therapeutics and medicines.

• 19:30Speaker 06

  I would put something like the discovery of insulin right up there near the top.

• 19:37Stephen J. Dubner?

  That's Keith Weilu. He's a Princeton historian who focuses on health policy.

• 19:42Speaker 06

  It transformed diabetes from an acute disease into a disease that you live with. And to me, that is much more the story of what medicine has been able to do in the 20th century.

• 19:57Speaker 01

  The medicine that comes to my mind is statins. They've been shown to have benefit in preventing heart attacks, in the prolongation of life among people who've had heart attacks, and the same thing for stroke and other forms of cardiovascular disease. But there are many, many drugs that are like that.

• 20:16Stephen J. Dubner?

  These are truly awesome interventions for which we should all be thankful. One of the most remarkable developments over the past century and a half is the unbelievable gain in life expectancy. In the US and elsewhere, it nearly doubled. Now, it might be natural to ascribe that gain primarily to breakthrough medicines, but in fact, a lot of it had to do with something else.

• 20:39Speaker 06

  A lot of the advances in mortality and morbidity have come from really changes in the nature of social life. Infectious disease as the source of high mortality in the early 20th century began to drop long before penicillin and the antibacterials came along in the mid-century because of improvements in housing. Sanitation, diet, and sort of tackling urban problems that really created congestion and produced the circumstances that made things like tuberculosis the leading cause of mortality.

• 21:14Speaker 12

  For example, if you think about the reversal of the Chicago River, it used to flow into Lake Michigan in the 19th century, and people were dumping their waste into it. So every summer, there would be hundreds of deaths of babies and children from infant diarrhea because the water was so contaminated. They reversed the flow of the river, so it flowed downriver toward the Mississippi, and that significantly improved the health of the people who lived there.

• 21:42Stephen J. Dubner?

  So we've got public health improvements to thank, and yes, better therapeutics and medicines, also new and better ways of finding evidence.

• 21:53Speaker 04

  I actually think it's the technology that really revolutionized how we think is the use of controlled experiments.

• 21:59Stephen J. Dubner?

  That's Vinay Prasad. He is an assistant professor of medicine at Oregon Health and Sciences University. Prasad treats cancer patients, but also...

• 22:08Speaker 04

  The rest of my time I devote to research on health policy, on the decisions doctors make, on how doctors adopt new technologies, and when those things are rational and when they're not rational.

• 22:20Stephen J. Dubner?

  Which means that Prasad is part of a relatively new, relatively small movement to make medical science a lot more scientific.

• 22:30Speaker 04

  You know, if you think about medical science, for thousands of years, what was medicine but something that somebody of esteemed authority had done for many years and told others that it worked for me, so you better do it.

• 22:41Stephen J. Dubner?

  Even though medical science seemed to be based on evidence, Prasad says...

• 22:47Speaker 04

  The reality was that what we were practicing was something called eminence-based medicine. It was where the preponderance of medical practice was driven by really charismatic and thoughtful, probably to some degree, leaders in medicine. And, you know, medical practice was based on bits and scraps of evidence, anecdotes, bias, preconceived notions, and probably a lot of psychological traps that we fall into. And largely from the time of Hippocrates and the Romans until, you know, maybe even the late Renaissance, medicine was unchanged. It was the same for a thousand years. Then something remarkable happened, which was the first use of controlled clinical trials in medicine.

• 23:30Stephen J. Dubner?

  Coming up on Freakonomics Radio, how clinical trials began to change the game.

• 23:35Speaker 04

  It really doesn't matter that the smartest people believe something works. The only thing that really counts is what is the evidence you have that it works?

• 23:43Stephen J. Dubner?

  How some people didn't have much of an appetite for actual evidence.

• 23:48Speaker 13

  There was a great deal of hostility to it from, I'd say, the medical establishment.

• 23:53Stephen J. Dubner?

  And, in a strange twist, how better science is pushing medicine not always forward, but sometimes backwards.

• 24:01Speaker 01

  It is quite common to see practices that end up getting reversed. And the best estimates are that happens about 15% of the time.

• 24:19Stephen J. Dubner?

  Freakonomics Radio is sponsored by Southern New Hampshire University. The best investments are the ones you make in yourself, and no one knows that better than Southern New Hampshire University. With more than 200 online degree programs, no set class times, and low online tuition, Southern New Hampshire University is built to fit your life today and set you up for success tomorrow. Education is an investment. Why not make it a smart one? Apply for free at SNHU.edu/freakonomics.

• 24:52Stephen J. Dubner?

  Freakonomics Radio is sponsored by Progressive. Whoever says saving money is hard doesn't know that new customers who bundle and save with Progressive save hundreds because switching your home and auto insurance is better than cutting down on the things you love, like those iced lattes that help you get ready to face the day. So, switch your home and auto to Progressive and see if you could save and keep sipping away. Get your quote today at progressive.com. Progressive Casualty Insurance Company, affiliates, and other insurers. Not available in all states.

• 25:24Stephen J. Dubner?

  Freakonomics Radio is sponsored by PNC Bank. Boring. Great word, bad rep. Yet, time and time again, here at Freakonomics, we find that when something culture-changing happens, boring things like calculations and careful planning are usually behind it. Which begs the question, why not apply it to other aspects of life, like banking, for example? Well, PNC Bank did, since its inception, proving time and again that the best banking strategy isn't the exciting type, but the steady, reliable, definitely not YOLO type. PNC Bank: brilliantly boring since 1865. Brilliantly boring since 1865 is a registered mark of the PNC Financial Services Group, Inc. PNC Bank, National Association, Member FDIC.

• 26:17Speaker 01

  All right, take a deep breath through your mouth, in and out.

• 26:21Speaker 01

  Out, good, okay? One more.

• 26:24Speaker 01

  One more.

• 26:27Speaker 01

  One more.

• 26:28Stephen J. Dubner?

  Anupam Jena is an MD and a healthcare economist.

• 26:32Speaker 01

  All right, I'm going to lift up your shirt and listen to your heart.

• 26:35Stephen J. Dubner?

  In most developed countries, we tend to think of medicine as a rigorous science. And we think of our doctors as, if not infallible, at least reliable.

• 26:47Speaker 01

  I think that the typical patient probably does look to their doctor for answers, and they value very highly what that opinion is.

• 26:58Stephen J. Dubner?

  But as we've been hearing, the history of medical science was often eminence-based rather than evidence-based. When did evidence really start to take over?

• 27:08Speaker 01

  Evidence-based medicine has become hugely important in the last 25 to 30 years.

• 27:14Stephen J. Dubner?

  The movement is a result, Jena says, of at least two factors. Number one...

• 27:19Speaker 01

  We're doing more randomized controlled trials, and that tells us more information about what works and doesn't work.

• 27:24Stephen J. Dubner?

  And number two?

• 27:25Speaker 01

  Improvements in computer technology have now allowed us to study data in a way that we couldn't have done 30 years ago.

• 27:35Stephen J. Dubner?

  There's also been a movement to collect and synthesize all that research and all those data.

• 27:40Speaker 14

  So, our vision is to produce systematic reviews that summarize the best available research evidence to inform decisions about health.

• 27:49Stephen J. Dubner?

  That's Lisa Barrow, a pharmacologist by training who studies the integrity of clinical and research evidence.

• 27:56Speaker 14

  And I'm also co-chair of the Cochrane Collaboration.

• 27:58Stephen J. Dubner?

  The Cochrane Collaboration was founded in Britain but is now a global network. The systematic reviews they produce-

• 28:05Speaker 14

  Are really the evidence base for evidence-based medicine. And we've been a leader in so many ways in developing systematic reviews. We were the first to regularly update these reviews. We were one of the first to have post-publication peer review and a very strong conflict of interest policy. And actually, we were one of the first journals that was published only online.

• 28:28Stephen J. Dubner?

  Which means that whatever realm of medical science you're working on, you can access nearly all the evidence on all the research ever conducted in that realm, constantly updated, available on the spot. Compare that to how things used to work. Looking up some five or 10-year-old medical journal to find one relevant article that may well have been funded by the pharmaceutical company whose drug it happened to celebrate.

• 28:55Stephen J. Dubner?

  How is Cochrane funded?

• 28:56Speaker 14

  We're primarily funded by governments and nonprofits.

• 28:59Stephen J. Dubner?

  What about industry money?

• 29:01Speaker 14

  We don't take any money from industry to support any official Cochrane groups.

• 29:05Stephen J. Dubner?

  Which means, in theory at least, that the evidence assembled by the Cochrane Collaboration is pretty reliable evidence. As opposed to-

• 29:14Speaker 13

  A whole variety of things.

• 29:18Speaker 13

  Opinion? What the doctor had been taught 30 years previously in medical school? Tradition, what they had been told to do by or advised to do by a drug company representative that had visited them a week previously.

• 29:33Stephen J. Dubner?

  That is Sir Ian Chalmers, who co-founded the Cochrane Collaboration. He's a former clinician who specialized in pregnancy, childbirth, and early infancy. He was a medical student in the early 1960s. When Chalmers observed his elders in practice, he was struck by how much variance there was from doctor to doctor.

• 29:54Speaker 13

  Okay, so, some doctors would, if a woman had a baby presenting by the breach, would do a cesarean section, without any questions asked, as it were. Or they may take different views about the way that the baby should be monitored during labor, or the extent to which drugs should be used during pregnancy for one thing or another. So lots and lots of differences in practices. It's as long as your arm. It's madness, isn't it?

• 30:25Stephen J. Dubner?

  When he became a doctor himself, Chalmers worked at a refugee camp in Gaza. And, as he discovered-

• 30:32Speaker 13

  Some of the things that I had learned at medical school were lethally wrong.

• 30:35Stephen J. Dubner?

  Like, how you were supposed to treat a child with measles.

• 30:40Speaker 13

  I'd been taught at medical school, never to give antibiotics to a child with a viral infection, which measles is, because you might induce resistance to the antibiotic resistance. But these children died really quite fast after getting pneumonia from bacterial infection, which comes on top of the viral infection of the measles. And what was most frustrating was that it wasn't until some years later, that I found that there had been six controlled trials comparing antibiotic prophylaxis, given preventatively, with nothing.

• 31:24Speaker 13

  Done by the time I arrived in Gaza.

• 31:26Stephen J. Dubner?

  And those studies suggested that children with measles should be given antibiotics. But Chalmers had never seen those studies.

• 31:34Speaker 13

  So I feel very sad that in retrospect, I let my patients down.

• 31:39Stephen J. Dubner?

  This led Chalmers to embark on a years-long effort to systematically create a centralized body of research to help attack the incomplete, random, subjective way that too much medicine had been practiced for too long. He was joined by a number of people from around the world, many of whom, by the way, were more versed in statistics than in medicine.

• 32:00Speaker 13

  So we embarked on these systematic reviews, about 100 of us. And that resulted at the end of the 1980s in a massive, two-volume, 1,500-page book. At the same time, started publishing electronically.

• 32:17Stephen J. Dubner?

  And so, the Cochrane Collaboration became the first organization to really systematize, compile, and evaluate the best evidence for given medical questions. You'd think this would have been met with universal praise. But as with any guild whose inveterate wisdom is challenged, as unwise as that wisdom may be, the medical community wasn't thrilled.

• 32:44Speaker 13

  There was a great deal of hostility to it from, I'd say, the medical establishment. In fact, I remember a colleague of mine was going off to speak to a local meeting of the British Medical Association, who had basically summoned him to give an account of evidence-based medicine and what the hell did people who were statisticians and other non-doctors think they were doing, messing around in territory which they shouldn't be messing around in. And he asked me before he drove off, what should I tell them? I said, when patients start complaining about the objectives of evidence-based medicine, then one should take the criticism seriously.

• 33:29Speaker 13

  Up until then, assume that it's basically vested interests playing their way out.

• 33:34Stephen J. Dubner?

  It took a long while, but the Cochrane model of evidence-based medicine did become the new standard.

• 33:41Speaker 13

  I would say it wasn't actually until this century. So one way you can look at it is where there is death, there is hope. As a cohort of doctors who rubbished it, moved into retirement and then death, the opposition disappeared.

• 33:56Speaker 04

  Yeah, so that's been the slower evolution.

• 33:59Stephen J. Dubner?

  That again is Vinay Prasad from Oregon Health and Science University.

• 34:05Speaker 04

  The very first studies with randomization concerned tuberculosis.

• 34:10Stephen J. Dubner?

  This was in the late 1940s.

• 34:11Speaker 04

  And then from then until really the 1980s, the end of the 1980s, we did use randomized trials, but they weren't mandatory. They were sort of optional.

• 34:21Stephen J. Dubner?

  One big benefit of a randomized trial is that you can plainly measure in the data the cause and effect of whatever treatment you're looking at. This may sound obvious, but it is remarkable how many medical treatments of the past were conducted without that evidence. Anupam Jena again.

• 34:38Speaker 01

  I think some of the biggest mistakes in the last century, let's say the first from 1900 to 1950, things like lobotomies to treat mental illness, either depression or schizophrenia, those strike me as being some of the most horrific things that could be done to man without any really solid evidence base at all.

• 34:58Stephen J. Dubner?

  This is one of the trickiest things about practicing medicine day to day. Let's say you're a doctor and a patient comes to see you with a persistent headache. You make a diagnosis and you write a prescription. What happens next?

• 35:13Stephen J. Dubner?

  In many cases, you have no idea. The feedback loop in medicine is often very, very sloppy. Did the patient get better? Maybe they never came back, but maybe they went to a different doctor. Maybe they died. If they did get better, was it because of the medicine you prescribed? Maybe. Or maybe they didn't even fill the script. Or maybe they did fill the script, but stopped taking it because they got an upset stomach. Or maybe they did take the medicine and they did get better, but maybe they would have gotten better without the medicine. Like I said, you have no idea.

• 35:54Stephen J. Dubner?

  But with a well-constructed randomized controlled trial, you can get an idea. Vinay Prasad again.

• 36:01Speaker 04

  The moment, I think, in my mind, that kind of set us on a different course was a study called CAST.

• 36:06Stephen J. Dubner?

  CAST stands for Cardiac Arrhythmia Suppression Trial. It was conducted in the late 1980s.

• 36:12Speaker 04

  CAST was a study that, one of the things doctors were doing a lot for people after they had a heart attack, was prescribing them an anti-arrhythmic drug that was supposed to keep those aberrant rhythms, those bad heart rhythms, at bay. That drug, actually, you know, carefully done randomized trial, turned out not to improve survival, as we all had thought, but to worsen survival. And that was a watershed moment, I think, where people realized that randomized trials can contradict even the best of what you believe. It really doesn't matter in medicine that the smartest people believe something works. The only thing that really counts is what is the evidence you have that it works?

• 36:48Stephen J. Dubner?

  The rise of randomized controlled trials led to a rise in what are called medical reversals. Vinay Prasad wrote the book on medical reversals, literally. It's called Ending Medical Reversal.

• 37:00Speaker 04

  And you know, what is a medical reversal? Doctors do something for decades, it's widely believed to be beneficial, and then one day, a very seminal study, often better designed, better powered, better controlled than the entirety of the pre-existing body of evidence, it contradicts that practice. It isn't just that it had side effects we didn't think about, it was that the benefits that we had postulated. Turned out to be not true or not present.

• 37:24Stephen J. Dubner?

  For instance.

• 37:25Speaker 04

  In the 1990s, we would recommend to postmenopausal women to start taking estrogen supplements because we knew that women before they had menopause had lower rates of heart disease, and we thought that was because of a favorable effect of estrogen. And then in 2002, a carefully done randomized controlled trial found that actually it doesn't decrease heart attacks and strokes. In fact, if anything, it increases them.

• 37:45Stephen J. Dubner?

  I asked Prasad what first got him interested in studying medical reversal.

• 37:49Speaker 04

  So I think I started to get interested in this even when I was a student, and I saw that there were some practices that had been contradicted just in the recent past, but were still being done day in, day out in the hospital. I mean, the example that comes to mind is the stenting for stable coronary angina. A stent is a little foldable metal tube that goes in a blocked coronary artery, and the doctor springs it open, and it opens up the blockage. And stents are incredibly valuable for certain things. If you have a heart attack and there's a blockage that just happened a few minutes ago, and the doctor goes in and opens that blockage up, we're talking about a tremendous improvement in mortality, one of the best things we do in medicine. But stenting, like every other medical procedure, has something called indication drift, where, yeah, it works great for a severe condition, but does it work just as good for a very

• 38:34Speaker 04

  mild condition? And so, over the years, doctors had used stenting for something called stable angina. And stable angina is just that slow incremental narrowing of the arteries that happens to, sadly, all of us as we get older. But the bulk of stenting was this indication drift, and we thought it worked and made perfect sense. And then in 2007, a well-done study showed that it actually didn't improve survival and didn't decrease heart attacks, which were, even to this day, studies show that most patients who undergo this procedure believe it will do those things. And in fact, it's been disproven for eight years.

• 39:06Stephen J. Dubner?

  And yet, while stenting for stable angina did decline, it didn't disappear. The rate of inappropriate stenting, Prasad says, is still way too high. Now, this obviously starts getting into doctors' incentives, financial and otherwise. And we'll get more into that in parts two and three of this series. As Prasad makes clear, there is a long, long list of medical treatments that simply don't stand up to empirical scrutiny. Some common knee surgeries, for instance, where orthopedic surgeons take a tiny camera-

• 39:41Speaker 04

  Make a tiny incision and go in there and actually sort of debrid and remove those sort of scuffed and scraped knees. And in fact, people sort of felt a lot better. They had improved range of motion. There's no argument there. But you've studied it against maybe just taking ibuprofen, or maybe just doing some physical therapy. What if you studied it against making the patient believe that you were doing the surgery, but you don't actually do it? And in fact, they've done those studies. Those are called sham studies. We give the appearance that we're going to do this procedure, and the only thing we omit is actually the debridement of the meniscus and the cartilage. And in fact, when you do it that way, you find that the entire procedure is a placebo effect. There's another example where we use sort of a cement. We inject into a broken vertebral

• 40:22Speaker 04

  bone, and that, again, was found to be no better than injecting a saline solution in a sham procedure. And the cement itself costs $6,000. And I said, you know, at a minimum, you can save yourself the $6,000, and you don't need to use the cement.

• 40:35Stephen J. Dubner?

  What would be the incentives for me to do the study that might result in a reversal? Because we know how publishing works, whether it's in your field, in any academic field, or in the media as well. It's the juicy, sexy, new findings that get a lot of heat, and it's the maintenance articles, or the reversal articles, that nobody wants to hear about. So I would gather that there are fairly weak incentives to doing the studies that would result in reversals, which also makes me wonder if there is a woeful undersupply of such studies, which means that there probably would be even more reversals than there are.

• 41:11Speaker 04

  Yeah, so I think that's a fantastic question. One of the things we did in the course of our research was we took a decade worth of articles in probably one of the most prestigious medical journals, the New England Journal of Medicine. And there's about maybe 1,300 articles that concern something doctors do. About 1,000 of those articles were something new, something that came off, that, you know, is coming down the pipeline, the newest anticoagulant, the newest mechanical heart valve. And if you tested something new, exactly as you'd expect, 77% of those published manuscripts concluded that what's newer is better. But we also discovered about 360 articles-

• 41:46Speaker 04

  Tested something doctors were already doing. But if you tested something doctors were already doing, 40% of the time, we found that it was contradicted or a reversal.

• 41:54Stephen J. Dubner?

  I'd love you to talk about the various consequences of reversals, including perhaps a loss of faith in the medical system generally.

• 42:02Speaker 04

  So if you find out something you were doing for decades is wrong, you harmed a lot of people. You subjected many people to something ineffective, potentially harmful, certainly costly, and it didn't work. The second harm, we say, is this lagtime harm. Doctors, you know, we're like a battleship. We don't turn on a dime. We continue to do it for a few years after the reversal. And the third harm is a loss of trust in the medical system. And that's the deepest harm. And I think we've seen it in the last decade, particularly with our shifting recommendations for mammography and for prostate cancer screening, where people come to the doctor and they say, "You guys can't get your story straight. What's going on?" It's a tremendous problem. And I'm afraid that probably what we are doing is we are making people feel like there is nothing

• 42:46Speaker 04

  that the doctor does that's really trustworthy. And I'm afraid that that's sort of the deepest problem that we're faced, this loss of trust.

• 42:53Stephen J. Dubner?

  OK, so how do you not throw out the baby with the bathwater? What are some solutions to a practice of medicine and medical research that results in fewer reversals?

• 43:04Speaker 04

  So that is a million-dollar question. One is medical education. You know, we have a medical education where for two years, students are trained in the basic science of the body. Only in the latter years, the third and fourth year of medical school, are students trained in the epidemiology of medical science and evidence-based medicine in thinking, not just how does something work, but what's the data that it does work? And, you know, I've argued that that needs to be flipped on its head, that the root, the basic science of medical school, is evidence-based medicine. It's approaching a clinical question, knowing what data to seek and how to answer that in a very honest way. So that's one. The next category is regulation. And this is where you get into, you know, what is the FDA's role and what does the FDA do? And I think many people in the community

• 43:49Speaker 04

  hope that the products that are approved by the FDA are both safe and efficacious for what they do. But, you know, we were ...with a problem in the 80s and 90s that we had never faced before, which was the HIV/AIDS epidemic. And advocates rightly said that we need a way to get drugs to patients faster, maybe even accepting a little bit more uncertainty, and I think that was right. And I think that's still right for many conditions that are very dire, for which few other treatment options exist, and which sometimes have very low incidence, so it's very hard to do those studies because very few people have it. But what's happened is that mechanism has been extrapolated to conditions that are not dire, that have very good survival, that don't have few options, that actually have many options, and that many people do have.

• 44:32Speaker 04

  So we've had, again, sort of a slippery slope for, you know, what qualifies for this accelerated approval. So I think there's ways in which regulation can be adjusted. And then I think the last thing is the ethic of practicing physicians. You know, we have to have an ethic where, when we offer something to someone and there's uncertainty, we should be very clear about communicating uncertainty. I think it's a tragedy today that, you know, no matter what you think of stenting for stable coronary artery disease, it's a tragedy that so many people who are having it done believe something that is clearly not true, that it lowers the rate of heart attacks and death. That's just factually not true, and the fact that many people believe that, I think, speaks to the fact that as doctors, we allow them to believe it.

• 45:12Stephen J. Dubner?

  And let me ask you one last question. I have a pretty good sense of, having spoken to you now for a bit, of what has prevented in the past medicine from being more scientific or more evidence-based. But what do you believe are the major barriers still that are still preventing it from becoming as evidence-based as you'd want it to be?

• 45:29Speaker 04

  So, we should be honest about what medicine is. And in the United States, medicine is something that now takes nearly or over 20% of GDP. It's a colossus in our economy. We spend more on medicine than any other Western nation. We probably don't get as much from it, from what we're spending, because it's such a large sector of the economy. The entrenched interests, the people who, the companies and the people who really profit from the current system, are tremendously reluctant to change things. I think we see that with, just for one instance, the pharmaceutical drug pricing problem we're having right now. I think it, no one will, no one will doubt that the pharmaceutical Has made some great drugs, they've also made some less than great drugs. But does every drug, great or worthless, have to cost $100,000 per year? And I don't invent that number.

• 46:14Speaker 04

  That's actually the, the cost per annum of the average cancer drug being approved in the United States in, in the last year. Well over $100,000 per year of treatment. I, I think there's got to be a breaking point and people are recognizing that.

• 46:36Stephen J. Dubner?

  Next week on Freakonomics Radio, in part two of Bad Medicine, how do those great drugs, and the less than great ones too, get made? And then, how do they get to market? We'll look into the economics of new drug trials and how carefully the research subjects are chosen.

• 46:54Speaker 15

  Now, that's very useful for a company that are trying to make their treatment look like it's effective. But does the population of people in this randomized trial really reflect the real world people out there?

• 47:06Stephen J. Dubner?

  We'll look at who's been left out of most clinical trials.

• 47:10Speaker 05

  That suggested that women shouldn't be included in clinical trials because of the potential adverse events to the fetus.

• 47:18Stephen J. Dubner?

  And how sometimes the only thing worse than being excluded from a medical trial was being included.

• 47:25Speaker 12

  The use of vulnerable populations of African Americans, people in prison, children in orphanages, vulnerable populations like these have been used for medical experimentation for a fairly long time.

• 47:40Stephen J. Dubner?

  That's next time on Freakonomics Radio. Freakonomics Radio is produced by WNYC Studios and Dubner Productions. This episode was produced by Stephanie Tam. Our staff also includes Alison Hockenberry, Merritt Jacob, Greg Rosalsky, Eliza Lambert, Emma Morgenstern, Harry Huggins, and Brian Gutierrez. You can subscribe to Freakonomics Radio on Apple Podcasts, Stitcher, or wherever you get your podcasts. You should also check out our archive at Freakonomics.com, where you can stream or download all our episodes. You can also read the transcripts and find links to all the research and books we've mentioned in this episode. You can also find us on Twitter, Facebook, or via email at [email protected]. Thanks for listening.

• 48:30Stephen J. Dubner?

  Freakonomics Radio is sponsored by EverPure. Is the value of AI not living up to the promise? The problem is probably your data, not your AI models. When it's scattered across different systems, getting your data AI-ready is a challenge. You keep your AI waiting and everything slows down. EverPure unifies your data so you can see it, understand it, and control it all from one simple place and get it AI-ready fast. Unlock the true value of your AI at everpuredata.com.

• 49:03Stephen J. Dubner?

  Freakonomics Radio is sponsored by Ozempic. Innovation happens through rethinking what's possible. And when it comes to GLP-1s, Ozempic pill does just that. Learn more about Ozempic semaglutide tablets, 4 in 9mg, by calling 1-833-OZEMPIC or visit ozempic.com to view the medication guide and ask your doctor what's possible with FDA-approved Ozempic pill.

• 49:34Stephen J. Dubner?

  Freakonomics Radio is sponsored by Southern Company. The world and its energy needs are always changing. Southern Company's commitment to meeting this demand stays the same. That's because Southern Company believes energy is more than a utility. It's what powers possibilities. So they are looking ahead and investing $80 billion in infrastructure upgrades and are committed to fueling growth in ways that benefit all customers so that reliable and affordable energy is accessible for generations to come. Go to southerncompany.com to learn more. Southern Company: building the future of energy.

• 50:11Speaker 16

  This episode is brought to you by Charles Schwab. Is there a right time to sell a stock? Are you taking the right risks with your portfolio? Financial decisions can be tricky, and often, your own cognitive and emotional biases can lead you astray. Financial Decoder, an original podcast from Charles Schwab, can help. Join host Mark Reapy as he offers practical solutions to help overcome the cognitive and emotional biases that may affect your investing decisions. Listen at schwab.com/financialdecoder.