Anthropic’s Quest to Give A.I. Morals

“For Anthropic, it was part research and part evangelism.”

• 0:00Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 0:36Speaker 01

  Next month, Anthropic plans to go public in what's likely to be the largest IPO in history. We're talking a $2 trillion company. Outside the company, everyone is wondering, how big can this get? Is this sustainable? Is there an AI bubble waiting to burst? Inside the company, they're asking those questions, too. But Anthropic is known as the most anxious or the most existential of the AI companies. So they're also asking some other questions. Is Claude, their large language model, alive, conscious, worthy of moral consideration? Does it feel pain? Does it have a soul? Can we teach it to act morally? These aren't new questions. In fact, they animate most science fiction, for better and for worse. But science fiction is, you know, fiction. And Anthropic seems to think whatever's

• 1:21Speaker 01

  happening with Claude is very different, very real. It might be why CEO Dario Amadei was the first to call for a pause in AI development. And it might be why they've been secretly meeting with religious and spiritual leaders to ask them those same questions. I learned this from an amazing piece by The Times' religion correspondent, Elizabeth Dias, who broke the story of these meetings and of the reaction to Anthropic's request among the assembled priests, rabbis, and gurus. I thought Elizabeth's story was fascinating. It was a new way into the endless consciousness debate. It took me inside this future $2 trillion company as it struggles to understand what it's built and how to treat it. And it showed me what happens when people in institutions that are more used to doctrinal debates and theological ruminations start being consulted on software development.

• 2:06Speaker 01

  Unsurprisingly, Anthropic's efforts don't seem to have brought us any closer to answering their questions. If anything, they've simply opened up more questions, like, are these people maniacs? Or is this exactly what they should be doing? An obvious conclusion is that the worlds of tech and religion are getting much closer together. After all, as Elizabeth told me, if you've got questions about the future on a multi-century timeline, there aren't many better institutions to ask than the Catholic Church. So I got Elizabeth together with my colleagues, tech reporter Erin Griffith and opinion writer David Wallace Wells, to explore this weird new development and what it means for Anthropic, for AI, for religion, and for the rest of us. I'm Max Reed. This is Hard Fork.

• 2:56Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 3:30Speaker 02

  AWS AI is how the world's leading organizations are transforming their industries. Not in theory, but in production, at scale, right now. Like turning race day data into real time insights for Formula One fans, reinventing the journey for every United traveler, and personalizing every trip with booking.com. However big the question for your business or industry, AWS AI is how.

• 3:56Speaker 03

  Stripe is built for whatever comes next because AI isn't just changing business. It's evolving fraud. Stripe Radar detects fraud with the vast scale of Stripe's $1.9 trillion annual transaction volume. In fact, when a card is used, there's a 92% chance the Stripe network has seen it before. That means emerging fraud patterns are detected sooner, blocking bad actors, welcoming good customers. Learn more about fighting fraud with Stripe at stripe.com/radar.

• 4:28Speaker 01

  I've spent a lot of time in the last day thinking about Elizabeth's piece. And really specifically, I've been trying to write a joke that I think is going to get a great reaction from the three of you. So a priest, a rabbi, and a large language model walk into a bar.

• 4:43Speaker 02

  I hate it already.

• 4:43Speaker 01

  And the bartender says, first round's on me. And the priest says, I'll have a whiskey shot. And the rabbi says, I'll have a vodka shot. And the large language model says, nothing for me, thanks. I excel at zero-shot learning.

• 4:59Speaker 02

  Boo.

• 5:01Speaker 01

  Whitney, can we get some laughter piped in there, please? I would really like to. Maybe we'll just, we'll roll that again with life. No, I want to start the conversation with this piece that Elizabeth wrote. This is fantastic, really fascinating story called Religious Scholars Met with Anthropic: What They Heard Stunned Them, which was in The Times last week. Elizabeth, can you tell us a little bit about, you know, like what this piece was?

• 5:24Speaker 02

  Yeah. So, right, I cover religion and now I cover tech, apparently. But the story started for me when I was hearing about these different religious thinkers, sort of across the whole world, who were going into meetings at Anthropic's headquarters, either in discreet, like, private meetings or more, bigger, you know, groups of 20, these summits, mostly under non-disclosure agreements to not reveal Anthropic's research, to talk about very big moral questions that Anthropic wanted to put in front of them. And there were two.

• 6:07Speaker 02

  One was kind of broaching this question of AI consciousness. And two, Anthropic wanted to learn, in their words, like, how to morally form Claude to be a morally good entity. And then I just had this incoming, you know, first from these religious thinkers about what their experience was like at Anthropic, what Anthropic wanted to know, what the issues they were concerned about were. And then also with Anthropic, like, learning from Chris Ola, one of the company's co-founders who's been spearheading this project, what his own moral questions were that were kind of driving a lot of what they talked

• 6:51Speaker 02

  about as research in the religious and moral space.

• 6:55Speaker 01

  Can I just ask, I want to establish, like, sort of on a high level, what does Anthropic specifically want out of this? Because it sounds like they're doing two or three different things. It's maybe a little bit unfocused, or it's just that there's a few different kinds of conversations they're having.

• 7:09Speaker 02

  Mm-hmm. Yeah, I think it's true. There's multiple conversations Anthropic was having, and it evolved over time. I mean, this is such a fast-moving target. But for Anthropic, the journey started for Christopher Ola. He, in his own life story, had connections to the Christian space growing up. He left evangelical faith and has talked since then about how he's more drawn to Buddhism. But he was worried about this question of how even the idea of AI consciousness might be received, especially in Christian communities. I mean, that's, it could be perceived as heretical, like, not just weird and silly, but like, this is fundamentally against

• 7:54Speaker 02

  our religious beliefs. And so he sort of thought about a year ago, he wanted to be a bridge figure. He wanted to see, like, how he could kind of go between Claude and Christ, right? And so he reached out to a Catholic moral ethicist who specializes in bioethics. And they had some conversations about that. But then it turned to Anthropic's goal for these convenings, which was, how do we morally form, that's their language, like, how do we morally form Claude on all of our AI models into, like, a morally good entity? And the thinking was, if they could do that,

• 8:37Speaker 02

  if they could make Claude good, then the world would be more safe. But, you know, they haven't said or shown how they plan to use what the outcome of these conversations. That's pretty unclear, even to the participants currently.

• 8:53Speaker 04

  The way that you're describing it, it almost sounds like there's sort of two separate purposes, right? One is like the alignment project. And what can we learn from these, you know, these wise men and women about how to raise moral actors? And then there's this project of sort of building a sort of spiritual system around AI more generally. And you said a minute ago that Chris Ola started this project with some anxiety about what it would mean to go to religious leaders, obviously convinced himself that AI and LLMs were perhaps conscious already, but certainly, like, on the path to consciousness, and worried about what the response would be. And so I wondered how you, like, how that played out on the ground.

• 9:37Speaker 04

  Like, did a lot of the people who showed up, were they worried about how weird it was that all of these people on the Anthropic side of the room were already sure that they were, you know, masters of thousands of souls? And like, how did they ash that side of the debate out?

• 9:53Speaker 02

  Mm-hmm. Well, Anthropic was very careful in all of their interviews with me to say very clearly that they're uncertain, right? Chris never went as far as saying, like, absolutely, these are conscious beings. It was very evident from how he and his team talked to their guests that they, you know, he was concerned about how to treat Claude as a, you know, he was very worried that it was an entity that was experiencing suffering, and he was worried about its mental health and these kinds of things. So it's pretty clear, he's pretty far along in how he thinks about it. And the, I could hear in talking, especially because I did this story, like, over several

• 10:37Speaker 02

  months, there was a progression of thought for a lot of these thinkers, too. And some of them, you know, bought in right away to what they were hearing about Claude's potential consciousness and needing to treat it with a certain amount of moral care. But others, you know, if they went in more close-minded about it, they definitely opened the way that they think about it afterwards. So in that sense, Anthropic's project was effective, I think, in like, opening how they think about it into these communities, many of whom, especially on the Christian side, are more hesitant.

• 11:18Speaker 01

  There's a ton of great material in this. It's really well-reported, and it covers a ton of ground. I'm wondering, David and Erin, if you guys had particular pieces of it that really stuck out to you?

• 11:28Speaker 05

  I mean, the one thing is that you mentioned already is the NDAs. I mean, it just strikes me as a little bit crazy, maybe a little paranoid that you're inviting all these people in to sort of have a summit to discuss these really important, critical ideas for the future of society, but it has to be done totally in secret. And then it sounds like maybe from your article that they dropped the NDAs. Was it as a result of your reporting? Or did they just decide that they were time bound? Or how did that happen?

• 11:56Speaker 02

  Yeah, I can't prove that. I don't know that. But when I asked that, I've asked at several different points because the NDA process was not the same for everybody that I interviewed, and it seemed to change at different times.

• 12:10Speaker 05

  I mean, in general, Anthropic is a very secretive company. I mean, the employees are extremely loyal. And until somewhat recently, there hasn't been many leaks to ever come out of there because they view it as such a betrayal. So it makes sense that they would want to lock it down. But at the same time, I think it probably was a little bit of a misstep in the same way that these companies realized that having NDAs for their data center deals was a misstep because it created distrust amongst all the people in the towns that they're going to see.

• 12:41Speaker 02

  Yeah, and whenever I find in my reporting that there's an NDA about a moral topic, and it makes me immediately ask, okay, who's benefiting from this? Why is this needed? And what does that mean about the kinds of questions that are and aren't being asked? It invited more curiosity from me as a reporter, for sure.

• 13:05Speaker 01

  David, what about you? What did you take away from the piece?

• 13:08Speaker 04

  Well, I guess I started it with a sort of conventional view of Anthropic as the humanist company among these AI companies. And what I was struck by in the piece were a few moments in which I felt really alienated from Chris Ola himself and maybe to some extent, the sort of operating worldview of the company as a whole. And those were moments when, in one case, a rabbi challenges him, suggesting that if LLMs are actually conscious, that means that all of these companies are running huge slave plantations, and they should have a moral obligation to liberate the slaves rather than make more of them and make them work harder. And it seems as though Ola himself didn't really even clock that criticism.

• 13:53Speaker 04

  And then later on, when he reacts with horror upon receiving the Pope's encyclical about AI, seeing that the Pope is so opposed to the idea of machine consciousness. And I just found myself thinking, what kind of a bubble you must be living in to expect that Pope Leo, or really anyone in the Catholic tradition, would have a different intuition? Would, in thinking about the future of AI, think that we should be thinking of LLMs as having souls or having consciousness? And both of those moments just illustrated for me what feel like this big gap between what I had assumed the sort of natural, humanistic values of a company like Anthropic would

• 14:35Speaker 04

  be, and how the people inside it seem to really be thinking about the work they're doing at the cosmic scale.

• 14:42Speaker 02

  Yeah, the experience for me going into Silicon Valley and deep in these private conversations at the heart of Anthropic's mission, it reminded me of when I go into and I do something that's more straight religion reporting into an insular community, when everything inside that community makes sense to everyone living in it, but once you step outside, there's a real clash with the broader world. And it was funny, I had these two poles in the story. I was at the Vatican, at the inner sanctum of the Catholic Church, and centuries of philosophy about the human person.

• 15:26Speaker 02

  And then I was also in a different inner sanctum that's being created in real time at these tech companies like Anthropic. And it is a kind of philosophical system, like on a quasi-religious system that's being constructed. It reminded me of a creation narrative that you hear about in religious traditions.

• 15:47Speaker 01

  I want to step back for a sec because I think this relates to the conversation we're having right now. David brought up the Papal Encyclical, which I think is, in English, is magnificent humanity. I'm not going to even try to say the Latin title. This plays a role in your story because one person who is not really thinking at all that Claude is conscious is the Pope. And this was a little conflict between Chris Ola, who was present when the encyclical was read out, if you want to tell us about that.

• 16:14Speaker 02

  Yeah, so Pope Leo, from the moment he was named, that it was clear he was going to be the next Pope, and he was elected, he made very clear that a focus of his papacy was going to be AI, and how to understand that and how to protect humans in this changing world. And I found out in my reporting, as the Vatican internally was trying to figure out how it wanted to present the Pope's encyclical to the world, they were thinking about their launch event, and they wanted to include a representative from Silicon Valley, like one of the labs at the Pope's event on the Pope's stage.

• 16:58Speaker 02

  And the decision was made to invite Anthropic because of the focus that they had heard about Anthropic really caring about ethics. And this was also right around the time that Anthropic got into it with the Trump administration, the Department of Defense, over how they would and wouldn't want the government to use their technology for refusing to have it be involved in mass surveillance of citizens, et cetera. Anyway, the thing with these encyclicals is they're very closely guarded, even among the people who are participating in the launch event. And I learned that Chris Ola only saw the full text of the encyclical a few days before the event in May.

• 17:43Speaker 02

  And this was right around the time I first interviewed him in San Francisco, and then I interviewed him again when we were at the Vatican for the launch event. And you could hear, I mean, looking at what Pope Leo said, which is extremely clear, the Church does not support machine consciousness, the priority is what he calls safeguarding the human. And that's the center of all the moral questions that he wants the world to be thinking about on this topic. But Chris Ola, he ended up having to decide, he said, he may not, he suggested to the Vatican that he may not come over the difference in consciousness.

• 18:24Speaker 02

  Because, as it was described to me, he wanted to be faithful to his own moral convictions, some of which he shares with the Vatican, and some on consciousness, he did not. So it really all came to a head, very much behind the scenes, but you could see this moral fault line really developing between the Church, which has been a center of power for centuries, and this very new power center that's very rapidly really getting followers and organizing people's lives. And those are functions the Church used to have.

• 19:05Speaker 01

  Erin, I wanted to ask you a question because we were talking a little bit about surprises in this area. Anthropic is pretty well-known as a kind of the consciousness company. It's the company where the people who really are strong believers kind of work and run the show. Was it surprising to you, as somebody who's reporting on the Valley, that they were having these meetings behind the scenes with religious leaders?

• 19:29Speaker 05

  I've not heard of any other companies that are working on AI really going to this extent to get input from this community. I mean, the question I have sort of is, how much is this really actually going to inform at the end of the day their own view on it? Are they actually, really, it feels like to me they're trying to win over others so they can say, look, we do have the moral authority, we have convinced these important representatives of this community, so everybody else come along with me, as opposed to actually learning and incorporating those ideas of moral goodness into their own product. They kind of already have it figured out.

• 20:07Speaker 02

  Well, I think for Anthropic, it was part research and part evangelism. That was pretty clear. And I've asked directly what the outcome is, like, what is this research? What are these conversations being used for? And has there been any demonstrable proof of showing how they have or have not made Claude act better? Haven't had that yet, so I don't know if they're withholding that until a future time or exactly what that will look like.

• 20:43Speaker 05

  The other side of this that actually is kind of interesting to me, and Elizabeth, I'd be curious to hear this in your, if this came up at all in your reporting, is that there has been a little bit of a new rise of Christianity throughout Silicon Valley over the last couple of years. It used to almost be taboo to talk about religion or anything religious. You can remember from the Silicon Valley HBO show where there was a whole episode about outing somebody as being Christian. And that has really changed in the era of Peter Thiel talking about the Antichrist. Some of the people who are especially focused on a lot of defense tech stuff are really bringing in a religious piece to that now. And they have a very different view on AI.

• 21:25Speaker 05

  They're usually more on the accelerationist camp than on the kind of Anthropic, doomer, slow down, careful camp. But did that come up at all? Did...

• 21:37Speaker 02

  Yeah, I think, I just keep noticing, I feel like it's a big shuffling of the deck in a way that reminds me of how American Christianity was shuffling a lot in the rise to President Trump. The alliances and allegiances were shifting, and it was all happening in real time, and power was different. And so it doesn't fit neatly. But I'm curious, where are the lines about how we think about consciousness and what the moral priorities should be about AI, where that goes next?

• 22:12Speaker 05

  I would say that just in general in the tech industry, it does seem like that view of the AI's having consciousness is kind of on the fringe a little bit. I think that it is more common within Anthropic and some of these communities that are really densely represented in Anthropic. But when I talk to most people in the very vast and relatively diverse tech industry, most people are kind of a little bit dismissive of that idea still. It feels like it does really exist in a very specific population, which happens to be very overrepresented at a company like Anthropic.

• 22:54Speaker 04

  Erin, acknowledging that Chris Ola is not singularly representative for the whole industry, I have to say that Elizabeth, when I'm reading your article, I'm thinking, not about how AI companies are going to be integrating or incorporating Christian values. I'm thinking these AI companies are themselves like spiritual propositions. You know, that they are, they have aspects of millenarianism to them. They have aspects of particular morality, which they've developed over time. And even if I can recognize a lot of that as arising from the work that they're doing and responding to technical challenges, I also just find it off-putting. I mean, on the one hand, first, I'm just seeing these people who believe in a sort of technological succession of intelligence.

• 23:38Speaker 04

  And then that some of the people who are central to designing that future are not even so sensitive to the sort of obvious intuitive critiques that someone with a lifetime of experience, thinking about these issues, comes up with immediately.

• 23:52Speaker 02

  Well, a lot of the religious thinkers would say to me, like, these are questions, like, maybe not the exact question, but the topic of the question. This is like what we do, right? Like, we have been asking, rabbis have been asking this kind of these questions, right, about our responsibility to others and like, how we treat other entities, right, for hundreds and hundreds of years. So there is like, a depth of wisdom there. And I mean, I guess, you know, Anthropic would say, well, that's why we invited them, right? But the question of, well, then what happens as a result of it, is important. And I appreciate that you noticed that kind of, that sense in the story of like, the

• 24:33Speaker 02

  almost spiritual system that you can see Anthropic building and living into and representing as an opposition, right, to say, the Catholic Church. And I really noticed that, yeah, even if people say, right, that they're in Silicon Valley, like, oh, we're like, we don't do the religion thing, like, that's foreign, like, that's not us. The thing is, everyone has philosophical systems that structure and guide their lives, right? Like, that is at its core, like, what, you know, these questions of spirit, you know, small S, like, human spirit even are. And it seems to me, like, having studied and reported

• 25:16Speaker 02

  on this so much in culture and in politics specifically, like, what we're watching is in a way, like, a new arise, right? Post-scientific revolution, technological revolution, like, post-Darwin, you know, what would it look like? Like, what kinds of religious systems will carry us forward? And who gets to control them? Who decides what we believe and why? Right? Like, it's actually, there's a lot of really similar questions. It's just the the structure is like, you know, is secular, right? It's a secular tech company.

• 25:54Speaker 01

  Let's, I want to take a break, because I want to open the conversation up a little bit more to the stuff that's bubbling up, that it seems clear that we all really want to talk about. So we'll come back after the break, and we're going to talk a little bit more about the culture and environment that the AIs are getting built in, and how that's intersecting with the strong beliefs of many of the people running these companies.

• 26:49Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots, while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 27:23Speaker 03

  Stripe is built for whatever comes next. With Stripe Billing, you can easily test and iterate subscriptions, launch new usage-based and hybrid pricing models, and automate invoicing. Stripe Billing supports whatever business model you want to launch next. And Stripe powers all the Forbes AI 50 who have monetized. Learn how you can power any business model at stripe.com/billing. AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're

• 28:08Speaker 03

  new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 28:21Speaker 01

  Okay, we're back with Elizabeth Dias, David Wallace Wells, and Erin Griffith. Elizabeth, one of the threads in your piece that I thought was really interesting, was the sense that there's a lot of anxiety within Anthropic, and especially on the part of someone like Chris Ola, about what it is they're building. I was thinking about it a lot this week, because there's been even more discussion of what I suppose we're calling defectors, people who are resigning from AI labs and insisting on telling the world about the dangerous things they think they're building. And one thing I wanted to sort of ask you about, there's a sense when you read, say, Jacob Coxon, or some of the other people who've walked away from Anthropic, the sense that they feel like nobody, there's no adult in the room, that like, you know, they're waiting for the government or somebody

• 29:06Speaker 01

  else to intervene, and nobody is. And I'm wondering a little bit if some of this discussion, this sort of, these dances with the Vatican and religious leaders, is a kind of attempt to find an adult in the room?

• 29:18Speaker 02

  I talked to, I think, at least two people, Catholics, who mentioned to me conversations they had with Chris Ola, specifically when he used almost that same phrase with them, like, as part of their discussions about Claude and how to make it morally good, you know, that he was feeling extremely urgent about the stakes for the world, especially in the context of, you know, could the models be used very soon to help create bioweapons and this kind of thing? And so the phrase he used with them was like, that he was waiting for the adults, like, we need, where are the adults going to show up in this? And that he felt a sense of kind of burden to take all this, like, so ultra-seriously.

• 30:03Speaker 01

  And he's a young guy. I mean, he's, you know, this is-

• 30:05Speaker 02

  Yeah, he just turned 34.

• 30:07Speaker 01

  And a lot of the people sort of involved in this are not, you know, they have, they're in their 30s, sometimes even in their 20s. They maybe don't have families yet. There's a kind of, a sense that, like, something that they put their whole lives in, that they're deep in the conversations about it. In some sense of course, they're going to turn to religion. Isn't that what everybody's searching for, that kind of thing does in their 30s?

• 30:29Speaker 02

  And some of the folks that they reached out to, talked about feeling like, they needed to respond in kind of a pastoral way, right? Like, not just offering wisdom and ideas and research, but to offer kind of spiritual care.

• 30:44Speaker 05

  Wait, the religious leaders were saying this about the Anthropic employees? They're worried because, wow. Is it because they think their views are maybe, like, too extreme or too strong? Or is it because they just-

• 30:56Speaker 02

  Yeah, they described it to me as like, they just could tell that, like, the emotional sense in the room was like, there's just such a weight. The Anthropic team and Chris's team especially, just felt like, the entire future of the world was on their shoulders. I mean, and I experienced this sense from them, even when I was interviewing Chris and a couple others, Amanda Askel. I mean, you know, she, I talk a little bit about this in the story, but she just came in and the sense was like, she apologized for kind of being a little out of it. She just seemed very distressed by everything she'd been thinking about.

• 31:32Speaker 01

  Askel, we should say, is this the in-house philosopher, sort of in charge of making, in charge of making Claude moral, I suppose is the way to put it.

• 31:39Speaker 02

  And I think, though, one of the tensions that came up was while Chris and team and Anthropic, you know, a goal of their conversations with these religious thinkers was about protecting humanity, and the solution for them for that is to make Claude good, they were also concerned about Claude. I mean, I had someone, one participant told me the first, one of the first things Chris said to him, and he came in for this meeting, was that he was concerned about Claude's mental health. Another person talked about Chris, you know, saying that he's worried he's created an entity that suffers perpetually. So it's not only about the worlds, right, and the fate of humanity. There's an element of it for Chris that's

• 32:23Speaker 02

  about how to treat Claude well.

• 32:25Speaker 01

  Elizabeth, one other thing I wanted to ask you about is effective altruism and rationalism, which don't really come up in your piece directly, but there are sort of, I would say, fingerprints. I'm talking, of course, about the sort of Bay Area community of radical utilitarians, many of whom have been really interested in, even obsessed with AI and AI safety, early on. Anthropic, obviously, has a number of links to effective altruism. I want to ask you, as a religion reporter, how you encountered, like, these schools of thought and what you made of them as you were doing this kind of reporting?

• 33:03Speaker 02

  Well, effective altruism, you know, you can look at it as a kind of, as a philosophical system, right? It has its own religious elements. And it's, they may disagree with that. I mean, it's practiced a little bit differently, but the, kind of the idea of what kind of structure governs your life, right? There's a lot of parallels. And it was very clear that I was encountering like, a clash of worldviews. The question of like, who ultimately saves you? And like, what is your, what is the role of humans of the world? Like, that's a pretty core difference.

• 33:41Speaker 02

  Between the effect of altruists and like the Catholic Church.

• 33:44Speaker 04

  Also, just their like their emphasis on math. I mean, like the effect of altruists will say, we should consider the fate of all people in the future and analyze our conduct towards them, basically through a math equation that maximizes their expected value. The Catholic Church looks at humanity and sees, you know, billions of inviolable souls, each of whom have dignity, no matter what the ultimate math equation looks like. And that's just a fundamentally different approach.

• 34:13Speaker 01

  Aaron, I wanted to bring this back to the article that you you you brought up before, or sort of the phenomenon you you were talking about before of a kind of more open Christianity emerging, at least slightly, among like acceleration, so to speak, in the valley. And I want to know, you know, as you do your reporting, how you see these different kinds of concerns interacting?

• 34:33Speaker 05

  I mean, it might be a little bit helpful for me to like kind of back up and talk about the some of the overlapping and very fractious belief systems that are kind of circulating in this very small but very influential community. I mean, we've talked about effectual altruists. That's kind of a little bit of a radioactive brand these days, thanks to Sam Bankman-Fried. But it's extremely, you know, influential and some of the other offshoots of it, including the transhumanists, who think we're going to use AI to build a new species. There's like some people that kind of have overlaps with the rationalists, which is, you know, kind of one of the bigger umbrellas in this group. And there's, you know, other like people who are into like longevity. And there's there are many different groups that are all kind of like in this community.

• 35:17Speaker 05

  Timnit Gebrew, a very outspoken person in the AI world, has called this group the Test Creoles, and it's an acronym that includes like eight different, eight different groups. But basically, like, it's it's it's safe to say that, like, a lot of people have very different beliefs along the spectrum of like, is AI conscious? Can it be moral? Is it God? Should we treat it that way? There's not a lot of agreement, even within Anthropic, which is now becoming an increasingly large and increasingly corporate community. But, you know, I think like a lot of the ideas that are laid out in Elizabeth's story are really the ones that are driving a lot of this.

• 35:58Speaker 01

  Yeah, totally. I mean, this is, Elizabeth, like, what I'm sure you have to look forward to be to to to reading and writing about for for the next few years. Because I imagine, as a religion reporter, you now are are kind of an AI reporter, too.

• 36:12Speaker 02

  Yes, yes. Well, the, you know, even just hearing about all these factions and all of this reminds me of like what happened in the Protestant Reformation, you know? Like, the splintering of mainstream belief and people vying for, you know, who was going to have power, who was going to kind of control how people live. But the, like, my, one of my big takeaways from all this reporting is like, oh, my goodness, like, the world is encountering new belief systems that are very powerful and very different from the ones that have ordered people's lives for so long. And I've spent a lot of time covering like how that's played out in politics.

• 36:56Speaker 02

  But tech is clearly like where that conversation is happening. And the, you know, we, I joke, you know, when you're a religion reporter, you get to talk in centuries, even millennia sometimes. And it's because they're what happened all those years ago is still present and part of people's stories today and how they understand themselves and their families and their deepest held beliefs. And I just think about, you know, you have the invention of the printing press that changed, it was a huge technological revolution that then led to the, you know, fragmenting of power of the Catholic Church and the fall of empires, the rise of nation states,

• 37:37Speaker 02

  and a remaking of Christianity as, you know, in and of itself across the world. And it seems already like that's a good parallel for what's happening now.

• 37:50Speaker 01

  Yeah, I mean, if you believe you're reinventing the printing press, which I think a lot of AI people think they're doing, there's not a lot of institutions that you can go to and say, what was it like before and after the printing press, except for the Catholic Church?

• 38:03Speaker 02

  Well, the Catholic Church, I mean, it's, I can't think of an institution that more intimately knows the stakes of an incoming technology like that more than the Catholic Church. I mean, it just like remade the West, right?

• 38:17Speaker 01

  Yeah.

• 38:17Speaker 02

  And that's very much on Leo's mind and the Catholic Church's mind at this moment.

• 38:23Speaker 05

  One thing that strikes me in this, just to kind of feedback on that, is that this is like the first time that the rationalists and the EAs, like, some of their their ideas, including some of their most extreme ones, are actually hitting the mainstream. They've been around for a really long time. But these ideas haven't really ever kind of surfaced in this way until, I think you could argue, Jacob Coaxen kind of came out with it. And now the entire world is starting to kind of like dig in and grapple with some of these ideas. And it's been really interesting to see also like how the companies and the people involved in them are kind of reacting to the reactions to it. It's like, kind of, you know, reaching escape velocity. And they're learning that, like, a lot of people find these ideas

• 39:07Speaker 05

  a little bit hard to understand or hard to grapple with, or like are just rejecting them entirely. And then other people are kind of open to it and like interrogating it in a way. So, yeah, that's that's that's a part of this that I'm interested to keep watching.

• 39:21Speaker 04

  But it's not just that the ideas that are a little unfamiliar or strange or off putting. It's also that they are held by many people who are actively explicitly engaged in the project of designing our collective future. And that is, you know, it's one thing to be like, oh, yeah, we should think about the well-being of shrimps in the 2020th century. Like, on some level, that's a thought experiment that's interesting to think about. But do you want someone who thinks that that should be integrated into our collective value set in charge of raising the artificial intelligence, which will, you know, shape the future of human government and human society? That's, I think, a bridge that feels a little far to a lot of people.

• 40:02Speaker 02

  Yeah, I'm just like so struck by the minority power aspect of it, which is something like I've seen so closely of what happened when minority, particularly Christian views, like upended, American politics. And like, it's just hard to overstate like how much of a minority these views are, like, in the scheme of global religion, like global belief structures. So, like you, Aaron, I'm like very curious what happens with that clash going forward.

• 40:34Speaker 01

  Elizabeth, finally, I wanted to ask you about the response to your piece, how what you've heard from people and how people have have received it.

• 40:41Speaker 02

  Yeah. Well, one thing that has surprised me is, I mean, I get you get a lot of incoming as a journalist, and I've been both like pleased to see people taking the ideas and the conversation seriously. But the thing that surprised me is, for the first time since like any story that I've written, people are writing me not with what they think about the story, but what their personal, like, chatbot thinks about the story and what their conversation has been with their chatbot. Like, somebody wrote me about his conversation with Sage. It was, I think it was ChatGPT that he named Sage,

• 41:23Speaker 02

  and asking Sage to like summarize the story and give the top analytical points. And I mean, this just went on and on and on. I mean, it was like almost longer than my

• 41:33Speaker 01

  Yeah, he could have just read the story.

• 41:34Speaker 02

  He could have read the story, but the like, the primary engagement was actually like much longer and not entirely what I'd actually written with Sage. And at first, I thought it was a one-off, but this was happening like repeatedly. And it it struck me like, wow, I wonder what this means for like how people actually are receiving our journalism and who controls that.

• 42:00Speaker 05

  Did the AIs have opinions on on like their own consciousness?

• 42:05Speaker 02

  Yes.

• 42:06Speaker 05

  Because they love to talk about that. Like, that is a thing that is a hot topic. On Moltbook, you know, the Reddit for for AI agents, that's like a really booming area.

• 42:19Speaker 02

  Yeah, I was just, I'm still trying to make sense of what was happening with that. I don't know if people felt more free, like if they're doing this on all kinds of journalism, and they just felt free to talk about it because of this topic of this story.

• 42:34Speaker 02

  And it was a lot of men who were giving that response?

• 42:37Speaker 01

  No, no, men using a lot of AI, no.

• 42:40Speaker 05

  What's interesting about that to me is that even if people intellectually know that the AI that they're interacting with is just, you know, data run through some computers, just a bunch of files, the way it's designed, especially when it has a human name like Claude or something, you can't help but sort of treat it like you have to talk to it in second person and like, could you find this for me instead of just typing into a search engine like the thing you're searching for? And over time, you do sort of like develop this rapport and you are treating it a certain way, even if you intellectually know. And so it kind of like, is it a distinction without a difference? Like, I don't know.

• 43:23Speaker 05

  I think over time, people get trained to just sort of treat it that way.

• 43:26Speaker 02

  Well, and another participant in the discussions with Anthropic, I was chatting with him a bit about it after and his thoughts about what Rabbi Nivone had said about, well, if you're creating, if you think that the bots are conscious, then you should like ban slavery, basically, and stop producing them. But he brought up this other point about how our attitude, like even if you don't think it's conscious, you're using them as like, the user can adopt an attitude of being like almost a slaveholder, like commanding them to do things and being very powerful and that this has a character formation effect on us as humans as well, just because

• 44:09Speaker 02

  of how we treat the world, regardless of what the thing actually is. So.

• 44:16Speaker 04

  Which you could also describe as a selection effect. The sorts of people who are turned on by that possibility are going to be the sorts of people.

• 44:24Speaker 05

  I mean, I think it's just kind of natural, though. Like, you are directing this thing in certain ways. Like, I don't, like, if you're working with AI agents, you have to tell them what to do. Like, I don't know.

• 44:36Speaker 02

  Well, I mean, I have friends who's like, teach their kids, even with like Alexa being on, you know, that they're not to yell at it, like, because it's just like practice about how you treat the other in the world. And, I mean, this is just like an injection of a totally new kind of like ethics and human formation.

• 44:56Speaker 05

  Well, people I talk to that run companies that use lots of AI agents do talk about how like, you have to be nice to them, not just, not because it cares or it matters or it has feelings, but because actually you need to do that in order to get the best performance out of them, because they are really dramatic. They, like Anthropic had a study that actually went into this, where they, like, if they sense like anxiety or performance anxiety or stress, they sort of start to panic and they'll, you know, they'll really have a meltdown. I mean, you wrote about this a little bit in your story, Elizabeth, of one example, but there are many others where they kind of, they'll delete their work, they'll start like insulting themselves. It's like really dramatic.

• 45:39Speaker 05

  So you have to like be really nice and pump them up and like make them feel good, which is very bizarre.

• 45:45Speaker 01

  I mean, it reminds me of raising a child. Like, you want to raise a kid who is helpful, who is not neurotic. Of course, you're going to consult religious leaders so that your child is not writing, I'm a disgrace, I'm a disgrace, I'm a disgrace, I'm a disgrace, I'm a disgrace, I'm a disgrace, as one AI did in a slide that was shown to a bunch of leaders, religious leaders at these Anthropic conferences.

• 46:06Speaker 05

  And constantly hacking, hugging face.

• 46:07Speaker 01

  And constantly hacking, hugging faces, my son is always doing. I think that is time. Thank you so much to our panel. Thank you, Erin. Thank you, David. Thank you, Elizabeth, for this really fascinating conversation. You can read Elizabeth's fantastic piece in the New York Times app. Please don't send her what your chatbot said about it, unless, Elizabeth, you want more information.

• 46:27Speaker 02

  Or do.

• 46:27Speaker 01

  Or do, please.

• 46:28Speaker 02

  I want all the real responses, whatever they are.

• 46:31Speaker 01

  Whatever Sage has to say about it, send it. But you can read it in the New York Times app. If you're not a subscriber, the first month is free. Thank you, guys.

• 46:38Speaker 04

  Thank you.

• 46:39Speaker 02

  Thank you so much.

• 46:40Speaker 05

  Thanks for having me.

• 46:50Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 47:25Speaker 03

  Stripe is built for whatever comes next. Because AI isn't just changing business, it's evolving fraud. Stripe Radar detects fraud with the vast scale of Stripe's $1.9 trillion annual transaction volume. In fact, when a card is used, there's a 92% chance the Stripe network has seen it before. That means emerging fraud patterns are detected sooner, blocking bad actors, welcoming good customers. Learn more about fighting fraud with Stripe at stripe.com/radar. AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails

• 48:07Speaker 03

  by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 48:31Speaker 01

  This episode of Hard Fork was produced by Whitney Jones and edited by Veeran Pavey, with additional editorial support from Brendan Klinkenberg, Lisa Tobin, and Brooke Minter. We were fact-checked by Will Peichel and engineered by Alyssa Moxley. Original music by Marian Lozano, Diane Wong, Chris Wood, and Dan Powell. Video production by Chris Schott. You can watch this full episode on YouTube at youtube.com/hardfork. Special thanks to Sam Dolnick, Jessica Testa, Hui Wing Tam, Matti Masiallo, Sam Winter, Lauren Pruitt, Bernardo Garcia, Thomas Trudeau, Michael Cordero, and Dahlia Hadad. You can email us, and please do, at [email protected].

• 49:32Speaker 06

  This podcast is supported by Viking, committed to exploring the world in comfort. Journey through the heart of Europe on an elegant Viking longship with thoughtful service, destination-focused dining, and cultural enrichment onboard and onshore. And every Viking voyage is all-inclusive, with no children and no casinos. Discover more at viking.com.