Wanna slow down AI? These biologists can help.

Fifty years ago, scientists pioneering a terrifying gene-splicing technology figured out a way to collectively hit pause. Can we do it again with AI? We ask because some top AI industry leaders are saying the technology they’re pushing forward is moving too quickly … and might kill us all. They’re effectively saying, ‘Please, please regulate us! Slow us down! Tie our hands!’ But, by and large, they’re not slowing down, because they don’t want to be left behind. In other words, it’s a classic collective action problem. A type of problem that’s scary, but solvable. We’ve been here before. In 1975, a group of scientists holed up for four days on California's foggy Central Coast to stop the runaway technology of their generation: a new DNA splicing technology that they feared could cause mass death. On today’s show, how they set aside ambition and competition to stop a runaway technology. Live Show: Planet Money Live at Kilkenomics. Kilkenny, Ireland Nov. 6th - 7th https://kilkenomics.com/Read Planet Money: Our book: Planet Money: A Guide to the Economic Forces That Shape Your Life https://lnk.to/i3AukBdD Our weekly longform Planet Money newsletter https://n.pr/3zrFvUBOur weekly Indicator round-up newsletter https://www.npr.org/newsletter/indicatorFollow Planet Money: Instagram https://n.pr/3FqLuwsTikTok https://n.pr/3sGZdrqYouTube https://lnk.to/iCVDaW3CFacebook https://n.pr/3h92GwSListen to more Planet Money: Our short, daily podcast, The Indicator from Planet Money https://www.npr.org/podcasts/510325/the-indicator-from-planet-moneyOver 1,000 episodes in our archive on the NPR App https://app.npr.org/aggregation/510289Merch: NPR Shop https://shopnpr.org/search?q=planet+money&options%5Bprefix%5D=lastThe Planet Money game, Sell Me A Sasquatch https://shopnpr.org/products/planet-moneys-sell-me-a-sasquatch-board-game?_pos=1&_sid=c2996cc29&_ss=rSupport public media with NPR+ and enjoy perks for over 25 podcasts like this one. This show’s perks include bonus episodes and sponsor-free listening. Learn more atplus.npr.org http://plus.npr.org/. This episode was produced by Vito Emanuel. It was edited by Marianne McCune and fact-checked by Sierra Juarez. It was engineered by Travis Hagan. Alex Goldmark is our executive producer. Music: NPR Source Audio - "Left Coastingg," "Going Nuts," and "Unsatisfied Reflection" See pcm.adswizz.com https://pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences. NPR Privacy Policy https://www.npr.org/about-npr/179878450/privacy-policy

• 0:00Speaker 00

  This message comes from Apollo. Apollo provides access to long duration capital and a broad range of financing capabilities, helping companies drive growth at an efficient cost of capital. Learn more at apollo.com/capital.

• 0:15Speaker 01

  This is Planet Money from NPR.

• 0:20Speaker 02

  There's a very strange puzzle at the center of the AI debate right now.

• 0:25Speaker 03

  If by strange, you mean like totally terrifying and apocalyptic.

• 0:28Speaker 02

  All those things. And that is that many of the people in charge of the AI companies are begging for someone, anyone, to be the adult in the room that tells them to stop speedrunning the apocalypse.

• 0:43Speaker 03

  Yeah, they're saying like, given that there's even a tiny chance that the products we're building could lead to the end of human life, seems like it might be a good idea for everyone to just pump the brakes a little bit.

• 0:55Speaker 02

  But at the same time, none of them are taking any meaningful individual steps to slow down, probably because unless everyone else agrees to get on the same page, they fear falling behind and losing the race.

• 1:08Speaker 03

  In other words, what we've got on our hands is a collective action problem, where people know they need to come to an agreement, but they don't want to do it because of their self-interest and because they worry the other tech bros might defect or won't play by the same rules.

• 1:22Speaker 02

  It's a heady time, and there doesn't really seem to be a way out. But sometimes, history offers a way.

• 1:30Speaker 03

  So we headed to the stacks, over to the Dewey Decimal section for collective action problem, and we found this kind of wildly analogous story of a time when a new technology promised to transform life as we knew it, but also to potentially destroy the world along the way. And the people most invested in that technology got together and puzzled through this question. Could they collectively find a way to stop this runaway technology before it was too late, or would their fear of missing out make that impossible?

• 2:08Speaker 03

  Hello and welcome to Planet Money. I'm Alexi Horowitz. It's Gazi.

• 2:10Speaker 02

  And I'm Nick Fountain. If the tech barons actually want to meet, actually want to slow things down, may we offer a lesson from biology?

• 2:18Speaker 03

  That's right. The mitochondria is the powerhouse of the-- no, I'm just kidding. Today on the show, we're going to tell the story of the time when scientists, faced with this new technology that they worried could lead to highly contagious diseases, bioweapons, weird eugenics, and a whole host of other problems, figured out how to push past a collective action problem.

• 2:46Speaker 00

  Stay tuned for the following announcements and messages from our sponsors.

• 2:51Speaker 04

  This message comes from iXL. The iXL LevelUp Diagnostic gives you clear, just-in-time insight into student learning with benchmark results and real-time updates throughout the year. Even better, LevelUp creates a personalized action plan for every student that guides them to what they should practice on iXL to improve. Learn more at ixl.com/NPR.

• 3:13Speaker 00

  This message comes from BetterHelp. BetterHelp is in-network with more than 125 health plans, with an average copay for eligible members of just $23 per session. Don't have eligible insurance? You can still get 10% off your first month. Visit betterhelp.com/NPR to explore your payment options. That's better H E L P dot com slash NPR. This message comes from LinkedIn. Owning a small business comes with a lot of challenges and means juggling multiple things at once. It's even harder to do it efficiently. But with LinkedIn, you get all the tools you need to grow in one place. With LinkedIn, simplify your sales, marketing, and hiring so that you can actually run your small business.

• 3:55Speaker 00

  Learn more at linkedin.com/planetmonieshow.

• 4:00Speaker 02

  What's scary about this moment is that there's this society-morphing, world-changing, maybe economy-wrecking technology. And it's not only the Luddites who are saying, let's slow down. It's the people who are leading the charge. But they aren't slowing down because they know it'll be bad for them individually.

• 4:19Speaker 03

  And if the story that we're going to tell today is an allegory, they are kind of right. This might be bad for them. When you solve a collective action problem, someone is likely to get hurt. Just ask Janet Mertz.

• 4:32Speaker 02

  As far as I understand it, this whole story starts with you.

• 4:36Speaker 05

  Yeah.

• 4:39Speaker 02

  Are you bummed about that?

• 4:40Speaker 05

  No.

• 4:41Speaker 03

  Janet is a big-shot biochemist. In the 1970s, she was a 21-year-old getting a PhD at one of the premier programs in the US, at Stanford University, working on research that was revealing the very nature of how life worked.

• 4:55Speaker 02

  Did you have ambition of winning a Nobel?

• 4:58Speaker 05

  We were all thinking about that back in that era.

• 5:01Speaker 02

  All right, I'm glad you admitted it.

• 5:03Speaker 05

  And actually, two of the other students in my cohort went on to get Nobels.

• 5:09Speaker 02

  Uh-huh.

• 5:10Speaker 05

  It wasn't just my thesis advisor.

• 5:12Speaker 02

  You know three Nobel Prize winners personally?

• 5:16Speaker 05

  Oh, I know a lot more than three.

• 5:18Speaker 03

  Now, there is a reason so many people Janet knows personally went on to get the big kahuna, and that's because biology and chemistry, they're kind of undergoing a revolution at the time. And Janet was at the cutting edge.

• 5:31Speaker 02

  Which was all about cutting. Cutting up an organism's DNA and splicing it into another organism's DNA. They called this recombinant DNA.

• 5:43Speaker 05

  Prior to that, people would make recombinants using natural methods. So, you know, nature all the time is taking genes from one organism and getting them into another organism.

• 5:57Speaker 03

  But it required, it required the birds and the bees up until that point.

• 6:01Speaker 05

  Yeah, it used to be through breeding.

• 6:03Speaker 03

  But now, all of a sudden, there was the possibility of copying a gene from a bird and pasting it into a bee.

• 6:10Speaker 05

  That's right, yeah.

• 6:11Speaker 03

  This new technology had enormous promise. It could maybe enable the mass production of insulin in the lab, which up to this point had to be harvested from animals like pigs, or creating new crops that fixed the soil as they grew. The possibilities were endless. As one scientist put it back then, you could maybe even grow pork chops on trees.

• 6:31Speaker 02

  Weird, but all right. Janet was working on this miracle technology, developing an experiment that would push it forward. But she wouldn't get to finish this project, in part because of this guy. Were you the whistleblower that started this all?

• 6:46Speaker 06

  Yes.

• 6:47Speaker 02

  How does that feel?

• 6:49Speaker 06

  It defined the rest of my life.

• 6:51Speaker 03

  This is Bob Pollak. In the late 60s, he landed a job working for another Nobel winner, the guy who helped discover the double helix, James Watson. Watson wanted Bob to help him run a lab in some shingled buildings on the shores of Long Island.

• 7:05Speaker 02

  You were hot stuff.

• 7:06Speaker 06

  I was hot stuff, yes. The work was hot.

• 7:09Speaker 02

  The James Watson.

• 7:10Speaker 06

  Yes.

• 7:11Speaker 02

  Poached you, headhunted you.

• 7:13Speaker 06

  Well, let's just say, I accepted.

• 7:16Speaker 03

  As part of Bob's job-- Bob's job-- he had to teach a three-week course about animal cell biology that grad students from all over flew in for.

• 7:26Speaker 06

  Janet was a student in this course.

• 7:28Speaker 05

  It was a crash course.

• 7:30Speaker 02

  First day of the course, they're doing introductions. Your name, your major, your thesis project? It's Janet's turn.

• 7:36Speaker 05

  Hi, my name's Janet Mertz. I'm a first-year grad student in the Stanford Biochemistry Department, and my thesis project is to--

• 7:46Speaker 03

  Her thesis project is a little jargony, but basically, she wanted to better understand how viruses work in primates, including people. She was going to try to do that by cloning some recombinant DNA in E. coli. And the particular DNA that she was going to clone contained a virus that just so happened to cause tumors in some animals, like hamsters.

• 8:06Speaker 02

  And what happened next?

• 8:09Speaker 05

  Bob Pollak sort of freaked out.

• 8:10Speaker 06

  Wait a minute, wait a minute. This is a dangerous protein.

• 8:13Speaker 02

  What if the now-possibly cancerous E. coli escaped Janet's lab and got into a human?

• 8:20Speaker 06

  You put it in E. coli, you'll forgive the technical term, but you crap out E. coli all the time. It's in your body.

• 8:29Speaker 02

  The perfect vector.

• 8:31Speaker 06

  So, so do you want to do that?

• 8:34Speaker 02

  Do you want to put a cancer-causing, anything into something that can live in your gut? What if it somehow slips into a grad student's gut? What if it slips out into the wild?

• 8:44Speaker 05

  You could give the world cancer.

• 8:47Speaker 06

  This scares the hell out of me.

• 8:48Speaker 03

  Now, Bob's freakout, it did not end there. He was so worried about this that he decided to basically call the principal's office, to call Janet's boss, a guy named Paul Berg.

• 8:59Speaker 02

  Give me the exact conversation. Ring, ring, ring, ring. Hello, it's Dr. Berg.

• 9:04Speaker 06

  Who the [bleep] are you? No, it wasn't that, actually.

• 9:06Speaker 02

  No, Bob says that's not the right order. First, he told Dr. Berg how irresponsibly he thought Janet's research was, that they hadn't thought through all the implications. And then, Dr. Berg essentially said, who the heck are you?

• 9:19Speaker 03

  But eventually, Bob kind of convinced Dr. Berg to at least call around and see whether people he actually knew and trusted thought he might indeed be opening up Pandora's Petri dish.

• 9:29Speaker 05

  Because our feeling was, we could easily arrange this so that it doesn't escape from the lab. But of course, as we know, things occasionally do escape from a lab. You know, so it's not like it's--

• 9:44Speaker 02

  Was that a, was that a Wuhan reference? I think it was.

• 9:48Speaker 03

  Janet is actually on the fence about whether COVID escaped from the lab, but she says lab leaks do happen.

• 9:54Speaker 02

  Back then, Janet says, they thought the risk of her research destroying the world was teeny, one in a billion. And the potential reward of understanding these viruses better was huge. But as Dr. Berg polled his colleagues about whether the reward actually outweighed the risk, he became convinced. Bob was right.

• 10:15Speaker 03

  And so, he put the kibosh on Janet's experiment. If there was even a small chance that this was going to hurt someone, it wasn't worth it. And how did Janet feel about this?

• 10:25Speaker 05

  Well, it's above my pay grade again.

• 10:27Speaker 02

  Kind of, but this is also your best shot at the Nobel, right?

• 10:31Speaker 05

  Well, the reality in that era, Nobels almost never went to women and almost never went to grad students.

• 10:38Speaker 02

  It's your best shot of being a superstar biochemist.

• 10:45Speaker 05

  But anyway.

• 10:46Speaker 02

  But anyway, it's not like after Janet was told to stand down, biochemistry just stopped. In fact, it kind of accelerated. There were new advances on the regular, including by Janet. She made some advances in cloning, and she figured out how to splice DNA way more easily.

• 11:05Speaker 05

  Making recombinant DNA went from being an experiment that, at the time, very few people in the world could do, to something a bright high school student could do it.

• 11:17Speaker 03

  And anybody could make their own chimera.

• 11:20Speaker 05

  Yeah.

• 11:21Speaker 03

  This rapid spread of this new technology could potentially pose a whole host of new problems. Like, if they could get the right materials, what was going to stop a terrorist from making some superbug? For that matter, what was going to stop some precocious, but ultimately incompetent teen from inadvertently creating a superbug? Plus, it raised the ethical conundrum of potential designer babies. In other words, everyone was now racing towards an outcome that could be dangerous in all sorts of ways.

• 11:48Speaker 02

  It was a collective action problem. They knew that if collectively they could stop or slow down and make some tweaks, the world would be safer. But as individuals, for their own livelihoods and careers, the rational thing to do was keep going full steam ahead, so they could win the Nobels and get the tenured track positions. It only made sense for them to stop if everyone else was going to do it, too.

• 12:14Speaker 03

  Now, there are several ways to try to solve collective action problems. You can get the government involved, of course. You can try to align the incentives in a way that makes people play nice. And you can self-govern. Janet's boss, Dr. Berg, was one of the people who took a first stab at that last option: self-governance.

• 12:31Speaker 02

  Yeah, in 1974, he and a group of the country's top lab heads, basically everyone who had the ability to do this maybe scary research, published a letter in the top science journals, saying, "Okay, this is potentially too dangerous, so we're all agreeing to hit pause on our research, until we have a chance to meet up and figure out a safe way forward." They agreed to a moratorium on research.

• 12:55Speaker 07

  In other words, let's all stop.

• 12:58Speaker 03

  This is Michael Rogers. In the mid-'70s, he was working as a reporter at Rolling Stone. And he looked the part. He had a leather vest and long hair. But instead of interviewing rock stars, he says his editor forced him to do science stories. And when he read about the moratorium, he knew it was a big deal.

• 13:14Speaker 07

  That was a first, I think, in the history of science.

• 13:17Speaker 03

  A first. Good premise for a story.

• 13:19Speaker 07

  And it involved genetically engineered creatures.

• 13:23Speaker 03

  Boo!

• 13:24Speaker 07

  Now we've got some fun.

• 13:26Speaker 03

  Yeah, it had like a double helical twist.

• 13:30Speaker 07

  Yes, exactly.

• 13:31Speaker 02

  Which is why, in February of 1975, Michael ended up driving to a rustic conference center not too far outside of Monterey, California, called Asilomar, to attend the, now very relevant, Asilomar Conference on Recombinant DNA.

• 13:49Speaker 07

  This was a room full of nerds.

• 13:51Speaker 03

  It was a four-day conference. And most of the action took place in one big room, a chapel. Asilomar used to be a Young Women's Christian Association camp, so they had one.

• 14:00Speaker 07

  Which I always thought was, you know, very meaningful. Here we are debating the fundamentals of life in a chapel next to the ocean where life began.

• 14:10Speaker 06

  All right, so now I got a lot of feedback.

• 14:13Speaker 03

  Amazingly, recordings of this meeting still exist. Shout out to the National Academy's archives for those. And listening to the tapes, it becomes clear that the strongest voices in the room were those of the steering committee, the folks who organized the conference, including Dr. Berg, Janet's lab head. Janet herself did not get the invite.

• 14:32Speaker 05

  No, I was a grad student. They weren't inviting grad students.

• 14:36Speaker 02

  Plus, of the 140 people who attended, mostly scientists, there were very few women.

• 14:42Speaker 03

  At that point, most everyone there had abided by the temporary moratorium. They had stopped or slowed their research out of an abundance of caution. And now, seemingly, everyone wanted the moratorium lifted. But how? Therein lay the debate.

• 14:57Speaker 02

  There were basically two camps: those aligned with the organizers, who wanted to walk away in just a few days with a new set of guardrails, a consensus on how to do research safely, and those who wanted no restrictions at all on their research. Their basic stance was, anything that slows us down will be onerous, expensive, and will stifle our academic freedom. This camp included big deals, like Mr. Double Helix himself, James Watson.

• 15:24Speaker 03

  Which side was going to win? The fight would play out over these four days, not in an octagonal cage or a Petri dish, but behind a lectern up on stage, with some forays from the peanut gallery.

• 15:36Speaker 06

  Yeah, good morning, I think.

• 15:38Speaker 02

  But as speaker after speaker came up to that lectern, it became clear that the scientists didn't really want to talk about how to address whatever existential risk to humanity their research might pose. They wanted to talk about their research.

• 15:53Speaker 06

  I'd like to illustrate for you synthetic cohesive termini on hydroxyapatite. Parainfluenza 2 enzyme. And the next slide. Which cleaves DNA at the sequence CG.

• 16:04Speaker 03

  Alex Kaprin, a lawyer and an early pioneer in bioethics who was at the conference, says, as a result, the busiest place at the conference was actually the two payphones.

• 16:13Speaker 06

  Because between sessions, people hurried back to get in line at the phone, to call their labs, to describe what they had just heard, and say, "As soon as I get back, we should try to get ready to do that."

• 16:30Speaker 02

  Nobody called their loved ones to say, "I miss you"? They only called their lab assistants to say, "Get this experiment ready for me"?

• 16:38Speaker 06

  Yeah.

• 16:39Speaker 03

  Michael, the Rolling Stone journalist, says just the fact that this conference was happening already seemed to be spreading the information, potentially all around the world.

• 16:48Speaker 07

  My favorite foreign scientist was a Russian guy, who, he basically sat in the front row and took pictures of every slide that came up. And he was accompanied by a very slick young man who spoke perfect English and who was immediately identified as KGB.

• 17:12Speaker 02

  That part's unconfirmed. But it was becoming clear to those in the room, like Michael, that the more they talked about this cool science, the slimmer the odds were that they'd actually enact any safeguards.

• 17:22Speaker 07

  As the days went on, there was more of a leaning toward, this is getting too complicated and potentially destructive of the research path.

• 17:34Speaker 03

  Maybe this whole thing was going to be a mistake. We asked Alex Kaprin, the lawyer, about this.

• 17:41Speaker 02

  It is slightly ironic that this meeting that was convened to sort of slow things down, or at least do things safely, was going to accelerate the whole thing, right?

• 17:51Speaker 03

  Kind of a super-spreader event.

• 17:53Speaker 06

  Yes, yes. That was the irony. And my job was to bring the meeting back to how do we behave responsibly?

• 18:04Speaker 03

  Right. Alex wasn't just some rando lawyer invited to the conference. He was an expert in medical ethics and the law. And he was invited with a very specific task in mind. On the final evening of the conference, the night before organizers planned to hold a vote on how to move forward, he and a couple other lawyers were going to talk about the law and liability.

• 18:23Speaker 02

  So you were there to scare the living daylights out of these nerds?

• 18:28Speaker 06

  Yeah. Did you say nerds? I'm not, I don't adopt that phrase. Don't paraphrase me as saying that. No. These cutting-edge scientists.

• 18:44Speaker 03

  Finally, it was Alex's time to shine.

• 18:47Speaker 06

  Alex Kaprin, the next speaker.

• 18:49Speaker 03

  The organizers were counting on Alex to shift the tide. He walked up onto the stage with some pen and paper notes and started talking about the risks.

• 18:57Speaker 06

  If and when dangerous vectors escape from control.

• 19:01Speaker 03

  But Alex didn't just talk about the health risks to the world or the risks for lab workers. He and the other lawyers who spoke that evening also highlighted the legal risks for all the people in the room, the scientists leading those labs.

• 19:14Speaker 06

  Freedom of thought does not encompass freedom to cause physical injury to others.

• 19:19Speaker 02

  He then moved to regulation. He said, "Whether you like it or not, you're going to be regulated. So decide now. Do you want to make the rules for yourselves? Or do you want other people to make the rules? Like congresspeople, who can't tell reticulum from their own rectum."

• 19:36Speaker 06

  It is the right of the public, through the legislature, to reach erroneous decisions.

• 19:43Speaker 03

  Alex finished his speech. And story goes, he and the other lawyers who spoke on that final evening did manage to convince the crowd that this was, in fact, not just a collective action problem. It could also become an individual problem for the people in that room if they were held liable.

• 20:00Speaker 02

  Yeah, the conference organizers told them that they were happy.

• 20:03Speaker 06

  Thank you. You got their attention.

• 20:05Speaker 02

  You succeeded in the mission of scaring the living daylights out of them.

• 20:09Speaker 06

  Yeah, so now we face the challenge of coming up with something that we can sell.

• 20:16Speaker 03

  Something they could sell, meaning guardrails that everyone, including the naysayers, could agree to. Michael says if the intention of Alex's talk was to put a fire under everyone's reticulum, it totally worked.

• 20:29Speaker 07

  That night, there was several groups of people who stayed up most of the night, working on the final rules that would be voted on the next day.

• 20:42Speaker 02

  What happened the next day? And what it can teach us about this current march towards either nirvana or extinction? That is after the break.

• 21:01Speaker 00

  Stay tuned for the following announcements and messages from our sponsors.

• 21:05Speaker 04

  This message comes from iXL. iXL provides personalized learning and powerful insights that help teachers boost achievement and understand student progress in real time. Learn more at ixl.com/NPR.

• 21:19Speaker 00

  This message comes from NPR sponsor, the Capital One Venture X Business Card. Every business owner needs a serious business travel card that can keep up with their needs. With the Capital One Venture X Business Card, you earn unlimited double miles on every purchase, no matter where your business may take you. Plus, big purchasing power means you can spend more and earn more. The Capital One Venture X Business Card. What's in your wallet? Terms apply. See CapitalOne.com for details.

• 21:48Speaker 04

  This episode's sponsor is HubSpot, which offers the following message. What if instead of updating your CRM, your CRM updated you? No catching up on manual edits, no more guesswork. HubSpot's self-updating CRM keeps your customer context current and helps you understand what to do next. So, marketing builds more demand, sales wins more deals, and customers keep coming back. Learn more at hubspot.com.

• 22:16Speaker 02

  OK, now, obviously, we're talking about the Asilomar Let's Slow Down Conference because there are a lot of similarities to what's happening with AI right now. People asking, is this new technology so dangerous that we need to stop it or slow it down? And if so, what are some guardrails that everyone would agree to and follow? But there are some fundamental differences between the AI race and the recombinant DNA race. The biggest one, maybe, money. There are trillions of dollars, and arguably, the entire American economy riding on the AI race. Slowing that down is a tough pill to swallow.

• 22:54Speaker 03

  Another difference, the pace. Things are moving so fast, these companies cannot blink so much as pause for fear that they'll get left behind. Plus, it's not just companies in the US, it's also China in this race, which adds a whole different dimension.

• 23:09Speaker 02

  So, some huge differences, but also, maybe, hopefully, some lessons from how that final day of the conference at Asilomar played out.

• 23:19Speaker 03

  Right. After that all-nighter where the organizing committee hammered out the proposed guardrails, there were only a few hours left before everyone got kicked out of the conference center and had to catch their flights. The hope was that before that, they were going to come to a consensus. All join hands, you know, kumbaya. But there had been dissent all along.

• 23:38Speaker 06

  I just think we should use common sense.

• 23:42Speaker 02

  This is Mr. Double Helix himself, James Watson. And Watson had agreed to the temporary moratorium before. But by the conference, he had changed his tune. Now he was against basically any guardrail.

• 23:55Speaker 06

  The dangers that are involved are probably no greater than that faced in any hospital, where you just sort of get to live with it. And in this case, I think we're going to have to live with it too, if we want to be in this field.

• 24:06Speaker 03

  He said, yeah, if we're stupid, we might get sued individually. That's all the policing we need.

• 24:12Speaker 06

  That sounds very negative and right-wing, but I don't see any other way of proceeding.

• 24:18Speaker 02

  Michael Rogers, the journalist with the leather vest, says some of the loudest voices in the room agreed. Let us do science uninhibited.

• 24:25Speaker 07

  It seemed to me that this might end up doing nothing, and that that would be a real disaster in terms of the public.

• 24:35Speaker 02

  A bunch of scientists gather together to make rules governing their research, and they come home with nothing? What would that mean?

• 24:46Speaker 07

  That would, first of all, suggest that science shouldn't self-regulate, that if there was enough danger for top researchers to ask for a moratorium, then everyone gets together and says, oh, never mind. That really brings into question the whole process.

• 25:07Speaker 03

  The organizing committee, as mentioned, had a document that they'd worked on all night, which laid out some guardrails for how to lift the moratorium and move on as safely as possible.

• 25:17Speaker 02

  Guardrail number one, biosecurity measures. They were going to start categorizing experiments based on risk. And the risky experiments would have to take place in the most biosecure labs.

• 25:29Speaker 03

  Guardrail number two was a plan for biological containment. For example, rather than using E. coli that could thrive in any street food eater's gut, scientists would have to use strains that would not survive in the wild.

• 25:42Speaker 06

  For a show of hands, who now support?

• 25:45Speaker 02

  Somehow, maybe it was the deadline, maybe everyone wanted to get back to their labs.

• 25:49Speaker 06

  Those who are opposed to making that.

• 25:51Speaker 02

  Maybe it was the last-minute changes they put in to appease the critics.

• 25:54Speaker 06

  They there are substantial agreement here.

• 25:56Speaker 02

  They successfully self-regulated. It wasn't unanimous, no consensus, but all but a few voted for the guardrails.

• 26:04Speaker 03

  And part of the agreement included a plan for enforcement, how to make sure everyone would actually comply. And this may be a key takeaway for our tech barons of today, if you're still out there. The scientists at Asilomar self-regulated, but they did not self-police. They handed the policing over to the government. In the US, they gave that power to the National Institutes of Health, who importantly, controlled a huge portion of research funding in the US. So, even the people who weren't in the room when this agreement was made, like Janet Mertz, whose research started all of this, they had to get on the same page.

• 26:39Speaker 05

  NIA's could say, if you don't follow the guidelines, you don't get our funding. And so, out of Asilomar, everyone said, OK, we're on a level playing field. We can't say that with AI because they're going to do IPOs for funding.

• 26:57Speaker 02

  Right. It's a little different. It's hard to tie anything to money with the AI companies because they don't lack for funding. They don't need the government's money. But there are some proposals for how to regulate AI by at least aligning the incentives.

• 27:13Speaker 03

  Yeah. There's a camp that says you could impose new taxes on AI companies, and that alone would slow things down. Either tax them according to some unit of work or according to how much human labor they might be replacing. Basically, you make it more expensive to do things that might cause social harm.

• 27:29Speaker 02

  There are also technical and governance proposals out there, like building kill switches into hardware, letting third-party safety auditors look at models before they're released, to name a few.

• 27:40Speaker 03

  And if the AI bros do want to meet up and talk through all these proposals, try to hash things out, we've come away from our magic school bus bioethics tour with a few more tips and tricks for how they might run such a summit. Number one, limit the scope. At Asilomar, they didn't try to solve all the ethical problems their new technology posed. Like, was it even ethical to do gene editing in the first place? They just focused on safety.

• 28:05Speaker 02

  Two, when you're trying to talk about this and come to consensus, keep it in the room. Michael says the two payphones they had at the conference, those two lines out of the rest of the world, seemed like a danger to their success.

• 28:20Speaker 07

  The lack of communication, the no tweeting, no blogs, that was crucial.

• 28:28Speaker 03

  If they'd been tweeting and slacking their CFOs, things could have easily gotten derailed.

• 28:33Speaker 07

  I think there would have been a lot more confusion. And opposition would have risen up very quickly, probably during the conference itself.

• 28:41Speaker 02

  Lesson three is kind of weird. Invite the lawyers. That's not the lesson. The lesson is, with any collective action problem, it helps to remind people of the personal stakes, personal responsibility, personal liability. That's how you get people to focus.

• 28:59Speaker 03

  Final takeaway, and this one is maybe more of a warning, there will be losers. This is the hard-to-stomach truth about most any solution to a collective action problem. Some people will inevitably get the short end of the stick. Just ask Janet Mertz.

• 29:15Speaker 02

  Did slowing down this research hurt your career?

• 29:19Speaker 05

  It hurt my career enormously. It hurt lots of people's careers. I had this huge head start over everybody that I lost. I'm probably a little bit bitter about it.

• 29:35Speaker 05

  Yeah.

• 29:36Speaker 02

  Does it feel like you sort of paid the price for the rest of the field to take this moment and slow things down and figure out whether or not this kind of path of research was potentially dangerous for the rest of the world?

• 29:52Speaker 05

  I might have paid the biggest price. If there hadn't been a moratorium, I would have been the first one to clone.

• 30:01Speaker 02

  Do you feel like it was worth the cost to you?

• 30:07Speaker 05

  It would have been nice if I hadn't paid such a big cost, but even though I paid a heavy price, I felt it was worth it.

• 30:17Speaker 02

  For everybody else in the world.

• 30:19Speaker 05

  Yeah.

• 30:24Speaker 03

  So Elon, Dario, Sam, if you guys are still listening.

• 30:28Speaker 02

  Don't forget Mark.

• 30:29Speaker 03

  Oh, and Mark, and Mark. If you're still out there, still listening, we have a little proposal.

• 30:33Speaker 02

  Yeah, if you want to meet up in the Monterey Bay, talk these things out, we're happy to help coordinate. Just let us know.

• 30:40Speaker 03

  Call it AI Asilomar.

• 30:48Speaker 03

  And while we're on the subject of conferences, if you happen to be anywhere in or near Ireland on November 6 through 8, Planet Money is going to be there for the Kilkenomics Comedy and Economics Festival. We'll be doing a couple live shows. Find tickets at the link in the show notes or at kilkenomics.com.

• 31:05Speaker 02

  Just Google it. It'll autocorrect. Today's episode was produced by Vito Emanuel and edited by Mary Ann McCune. It was engineered by Travis Hagan and fact-checked by Sierra Juarez. Alex Goldmark is our executive producer. Special thanks to the great Dan Germa, David Marku from the University of Wisconsin-Madison.

• 31:22Speaker 03

  Go Badgers!

• 31:23Speaker 02

  And the folks who make the Distillations podcast from the Science History Institute. They have a great audio doc about Asilomar if you're looking for more depth. I'm Nick Fountain.

• 31:33Speaker 03

  I'm Alexi Horowitz-Ghazi. This is NPR. Thanks for listening.

• 31:50Speaker 04

  Stay tuned for the following announcements and messages from our sponsors. This message comes from iXL. iXL's Level Up Diagnostic gives immediate benchmark results along with personalized action plans that link directly to iXL's skill practice. More at ixl.com/NPR. This message comes from Mint Mobile. While others have been hiking your bill, Mint Mobile has kept their wireless plans starting at $15 a month. Switch to Mint Mobile and get three months of premium wireless service for $15 a month. Go to mintmobile.com/switch. Upfront payment of $45 for three-month plan required. $15 per month equivalent. New customer offer, first three months only, then full price plan options available. Taxes and fees extra.