Silicon Valley's Effective Altruist vs. Accelerationist Religious War

Molly White, Ryan Broderick, and Deepa Seetharaman join Big Technology Podcast to dive deep into the Effective Altruism (EA) vs. Effective Accelerationism (e/acc) debate in Silicon Valley that may have been at the heart of the OpenAI debacle. White is a crypto researcher and critic who writes Citation Needed on Substack, Broderick is an internet culture reporter who writes Garbage Day on Substack, Seetharaman is a reporter at The Wall Street Journal who covers AI. Our three guests join to discuss who these groups are, how they formed, how their influence played into the OpenAI coup and counter-coup, and where they go from here. -- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/ Questions? Feedback? Write to: [email protected] Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  Let's dig into Silicon Valley's religious war between the effective altruists and the accelerationists in a deep discussion with a fantastic group of guests. That's coming up right after this. This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs,

• 0:42Alex Kantrowitz?

  and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay? Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation. Welcome to Big Technology Podcast, a show for cool-headed nuanced conversation of the tech world and beyond. And this week, we're going to get into the EA and accelerationist

• 1:25Alex Kantrowitz?

  war that may or may not have been at the center of the OpenAI debacle, but certainly is starting to play a bigger role in Silicon Valley, and we need to get into it. So we have an unbelievable group of guests joining us today, all of whom have written about this. First, returning, actually in short order after her first appearance, and a crowd favorite, Molly White is here. She is the author of Molly White's newsletter, which you can find at newsletter.mollywhite.net. And she's also a crypto researcher and critic. Molly, welcome back to the show.

• 1:58Speaker 01

  Thanks for having me back.

• 2:00Alex Kantrowitz?

  Great to have you here. We also have Ryan Broderick. He's the author of Garbage Day on Substack. It's a great newsletter that looks at internet culture. I would say you're probably the most immersed reporter on internet culture working today. Ryan, welcome.

• 2:13Speaker 02

  Thank you. Thank you for having me. And yeah, I'm excited to talk about my new religion that I worship now. So I'm very excited.

• 2:22Alex Kantrowitz?

  I'm excited to hear which one you picked. We also have Deepa Sita Raman, who's a reporter covering AI at the Wall Street Journal, who just wrote a terrific, long story about the role of EA in the explosion at OpenAI. Deepa, welcome.

• 2:38Speaker 03

  Thank you. Thanks for having me.

• 2:40Alex Kantrowitz?

  The term effective altruism and the term accelerationism have been thrown around in a wild amount, without most people knowing exactly what they mean. And I think that like, on this show, in particular, like getting the definition and talking about who these organizations are and what they believe in is pretty crucial. So Molly, in your story, you did a great job defining exactly, you know, which, what these two groups believe in, who they are. Can you take us quickly through like what the divide is between EA and the effective accelerationists? And I know that your perspective here, and we'll get into it, is that, you know, we shouldn't just be focusing on these two groups. But like, just for definition purpose, let's talk about who they are.

• 3:22Speaker 01

  Sure. So the two groups are, you could probably spend an hour talking about each one, but at least, especially when it comes to AI, the two groups have become very prominent in terms of their philosophies around AI. So briefly, effective altruism is a group of people who believe in sort of doing as much good as possible with this sort of like data-driven analysis approach. And they have, some of them have very recently started thinking about whether it's really the best idea to be focusing on people who are alive today versus the possible thousands, millions, trillions of people who could be alive in the long-term future.

• 4:05Speaker 01

  And when they start thinking about that, they start thinking about existential risk. So things that could threaten those groups of people. And one thing that they have been focusing on a lot lately is this idea of artificial intelligence as an existential risk and the idea that if you were to create a super intelligent artificial general intelligence, it could pose, you know, it could, it could kill all humans, it could pose this enormous risk to people. And so that's part of the effective altruist concern and why they have largely adopted this idea that although they think we should be developing artificial intelligence, we should be doing so in a slower, more methodical way that tries to account for these risks

• 4:50Speaker 01

  and tries to create an AI that is aligned with humanity for whatever that means. The effective accelerationists, on the other hand, believe that we should just go all out, no breaks, develop as quickly as we can without any real regard for the risks and that it will all just sort of work itself out because of some philosophy that they sort of root in these thermodynamic bias of the universe that, you know, it just, the universe is not biased towards destroying itself and so therefore everything will be fine. Those are sort of the general competing ideologies that have been recently at the forefront of the AI debate, although they are hardly the only groups involved.

• 5:33Alex Kantrowitz?

  Ryan, can you talk a little bit about the formation of these groups or like how they, they manifest online? I, I understand there's like big message board communities that sort of put forth these ideologies and debate them and it's like pretty interesting, the way that they organize and, and build online.

• 5:50Speaker 02

  Yeah, I've tried very hard not to have to explain any of this for many years now because it's just so tedious and embarrassing for everyone involved.

• 5:58Alex Kantrowitz?

  Sorry, here we go.

• 5:59Speaker 02

  Yeah, but basically at the, the, the middle 2000s, there was the rise of a very kind of niche but influential message board and blog called Less Wrong. And Less Wrong is kind of like the thinking man's 4chan, I guess, with like a real preoccupation with the emergence of AI. The possibly apocryphal story about Less Wrong that I think is very funny and I try to spread as much as possible is that they became obsessed with the idea of an AI arising in the future that would kill anyone that didn't help it to be born and sort of created this like version of cyber hell that they became obsessed with and had to ban discussion of it from the board for a while. Good like message board drama stuff.

• 6:41Speaker 02

  I highly recommend going back and checking out some of the conversations, but it was the birthplace for several sort of digital philosophies, online philosophies. So the the big one is obviously effective altruism, but it was also the birthplace of the very beginnings of neo-reactionism, which was sort of the the the initial philosophy that Steve Bannon subscribed to. A lot of kind of like techno feudal thought was born on Less Wrong, this idea of using automation to replace democracy, replace countries, turn them into machine operated city-states. And you'll see kind of references, sly references to this idea still kicking around Silicon Valley because it's like a very seductive idea.

• 7:26Speaker 02

  Effective accelerationism didn't so much start on Less Wrong. It's sort of like this, I I sort of think compared to the the idea that you had, the the alt-right, who then became embarrassing to Gen Z, who then created the Groyper's. And effective accelerationism is sort of the same idea, which is that the effective altruists, the Less Wrong guys, they're like doomer boomer cringe dudes. So we got to create like this new idea to galvanize younger people around. And so in in that's sort of how I kind of view the breakdown. But if you wanted a really good example-

• 8:01Alex Kantrowitz?

  But it's more than just a different iteration of the same ideology, right? Like you have one, especially regarding AI, you have one that's very much into slowing down AI progress, worried about that risk. And then you have another group that's like, no, put the gas pedal on as hard as you can. Or or am I not seeing it right?

• 8:22Speaker 02

  I I have not found much difference in what these two groups want, only the timeframe in which they want them to happen. It is, in my opinion, I have not seen enough difference between the two. In fact-

• 8:37Speaker 02

  Effective accelerationism was more or less a meme until last week. Like, it wasn't really even like a thing. It was sort of just like a way to make older Silicon Valley guys look cringe or more cringe. So it's now becoming a little more nuanced and a little more coherent. But it it it mirrors almost exactly kind of like the splintering of the far right post Trump. Like, but but for AI, essentially.

• 9:01Alex Kantrowitz?

  Yeah, but you had, I mean, it is like you had Mark Andreessen, right? You know, effectively, whether whether someone labeled him as this effective accelerationist or not, sort of carrying the banner of this group. And actually, what he's saying is holding weight within Silicon Valley. Don't stop and basically continue to build as fast as you can. And don't worry about regulation or safety or anything like that. That is a divide, though.

• 9:28Speaker 02

  It is. It is definitely a divide. I am I'm curious if Mark Anderson is sort of thinking this through in a philosophical sense? Or if he's just sort of like latching on to it because it's exciting?

• 9:45Speaker 03

  Well, I mean, I was just gonna say, I mean, isn't Mark, when he did his tech optimist manifesto? When was that? A couple of months ago? Didn't he say that any type of deceleration of AI would cost lives? And so if you're slowing down AI development, you're basically a murderer?

• 10:05Speaker 02

  Yeah.

• 10:06Alex Kantrowitz?

  Right.

• 10:07Speaker 03

  Hang on.

• 10:07Alex Kantrowitz?

  He did say that.

• 10:08Speaker 02

  He did.

• 10:09Speaker 03

  Yeah.

• 10:09Speaker 02

  Yeah.

• 10:10Speaker 03

  Well, I mean, like, you know, there's also this, it's a real belief in, like, effective altruists definitely think that, you know, AGI or AI is going to usher in this new potentially golden era. And so do the effective accelerationists. So I do agree that it's a it's a difference in time scale. But where like they are like, well, we need to get everything perfect and right. However, we define right and perfect first, the accelerationists are like now, now, now. And there's a moral imperative to go now, now, now.

• 10:45Alex Kantrowitz?

  So Deepa, can you talk about how this like practically split? You know, within the boardrooms and the executive offices within some of like the bigger tech companies? You know, especially OpenAI, but also, you know, this has played out with Anthropic as well, which is effectively a split. Away from OpenAI. So talk about how this we actually manifested these ideologies in different companies and yeah, paces of innovation.

• 11:12Speaker 03

  I think, I think it's, it's, it's, it's really interesting. It basically is a part of, it's a force that shapes everything. Either you are pro EA, or you're anti EA, or you're mixed on EA, but no one is, I've met, has been neutral on EA, or like, they don't think about it that much, right? Like, it's definite, it's a, it's a force, whether you like it or not. And I think the way that it, it sort of has coursed through these companies is, is pretty interesting. I mean, OpenAI had a lot of EA-aligned researchers and founders at its very beginning, and a lot of the early employees. So Dario Amade, who's now the

• 11:54Speaker 03

  CEO of Anthropic, was, you know, running research at OpenAI for a long time, and he is very EA-aligned. For this story, we asked all the companies, by the way, if they agreed with characterization of them as EA or not, and all of the companies told us they are not EA-aligned.

• 12:14Alex Kantrowitz?

  Well, sure, they're not gonna say they are now. I mean, goodness gracious.

• 12:18Speaker 03

  Just a little data point, but, you know, the way they're perceived is definitely EA, right? And so you have at OpenAI kind of two different cultures forming, one that is more EA-adjacent. So this is like worrying about existential risk, worrying about, you know, an era where humans are treated by machines the way humans currently treat animals, which is something that Ilya Sutskever has said internally to various employees. Then you also have this like very practical side. And these are a lot of people that came from Facebook, actually, that are trust and safety type alums that think about, oh, technology at scale can do really crazy things. Let's try to figure out how those principles might

• 13:01Speaker 03

  apply to generative AI. And those two cultures.

• 13:08Speaker 03

  I theoretically don't need to be in conflict, but are at times, especially with the resource disparity. So if you look at OpenAI, they had and have this team that is dedicated to super alignment, which is how do we align super intelligent AI systems that, as far as I know, don't really exist yet? And so, and what should the alignment look like? And one of the projects that, you know, we reported on was they were trying to create an AI scientist. So this is like not a science, like a human scientist, like an AI scientist that would study alignment. And they have all that really interesting work going. Ilya Satskover is like part of that team, so it has a lot of internal cachet.

• 13:53Speaker 03

  It's, you know, interesting. And then they have these more, I don't know, like hand-to-hand combat kind of trust and safety folks that are working on things like the next election, right? And so OpenAI, a couple of months ago, just hired a somebody on their policy team focused on the 2024 election. So it's like on two tracks, right? Like we have the super alignment team that is building these systems, safety systems for systems that don't exist yet, based on values that don't, they haven't defined. But then we're also planning for, you know, we just hired one person so far for the election, which is an election a lot of us have been calling the AI election for a while,

• 14:38Speaker 03

  because we're, you know, the impact that AI could have on that, on that time. So a lot of the people and sources we talked to saw that resource disparity as, A, not necessary, right? Like, why can't we do both was a question that I heard a lot. But also reflective of EA-style principles, that even if people at the company don't think they're EA, they're influenced by EA, such that the long-term stuff gets more, more resources than the short-term stuff. So that's just one example of how it cuts across.

• 15:12Alex Kantrowitz?

  Molly, I'm gonna turn it to you. I'm kind of curious, like, some of these things that EA is thinking about, whether that's the risk from AI or how to like be maximally effective? Like, it seems like they're an interesting way to try to solve problems. I'm kind of curious whether you think there's merit to their ideas? And what do you think the, uh, the impact is, you know, as it gets applied within tech companies?

• 15:34Speaker 01

  Yeah, so the thing about effective altruism is on the surface, it seems very reasonable. You know, the idea is that you should, you know, most people are altruistic to some degree. And you want your altruism to be effective, like that is hard to argue with. Um, but you end up sort of in this rabbit hole around trying to define exactly what effective means. And there's all of this philosophizing behind it. It's very heavily tied to the sort of rationalist utilitarianist, uh, schools of thought, where you're sort of trying to develop these impartial equations around what the most effective use of your money and your time is. Um, and so, and so people take that in very different directions.

• 16:18Speaker 01

  Um, there are people within the EA movement who, you know, donate massive portions of their income to, you know, pandemic prevention or malaria bed nets in Africa, or, you know, things that are probably doing a lot of good. In fact, the EA community has done a lot of good in some, you know, charitable movements. Um, so it's, it's really hard to sort of criticize that. But then you also have this side of it that has spun out into this almost extremist. Um, you know, we must focus on existential risk over any other problem, including very serious problems that are affecting humans today. Um, we must value the lives of future humans above the lives of humans today.

• 17:01Speaker 01

  Um, and so it's one of those things that like, it's, it's really hard to, to criticize the movement because you'll often get people who will say, you know, oh, but look at all the good things. Yeah. And it's, it's one of those things that like, the problem is that when you, when you take effective altruism and you put it into practice, you end up with a vast sort of milieu of, of in implementations, some of which are enormously problematic. And so, um, it's, it's sort of hard to criticize EA on the face of it because you could be talking about 10 different things. But in the same vein, you know, you should not necessarily be looking at EA and going, oh, it's just effective, you know, philanthropy. That's great, um, because it is much, much more than that.

• 17:44Alex Kantrowitz?

  And Molly, you've been studying these organizations, some organizations where people that are, you know, Inherently tied to EA, rise very high or sometimes run them. I mean, Sam Bankman-Fried, who we've talked about on this show, was tied to EA. We also have, you know, folks extremely high with an OpenAI and Anthropic tied to EA. How has it been so effective at churning out leaders in the tech world who believe in this stuff?

• 18:11Speaker 01

  Well, I think, I think it goes in both directions to some extent. I think some people were, to some extent, shaped by EA. You know, Sam Bankman-Fried encountered it very early in his life and, at least to his telling, made decisions about his life based on the philosophy. Whereas there are other people who have sort of adopted it after much of their careers. So, you know, Elon Musk, for example, has somewhat embraced effective altruism, but it doesn't seem like it was really a defining feature, more as it was something he learned about and said, oh yeah, that sounds about right. And I think the same is true of Marc Andreessen and effective accelerationism, for that matter. You know, he was not thinking about effective accelerationism, which was not even a thing back in, you know, when he was developing web browsers.

• 18:56Speaker 01

  This is something he is now adopting, sort of post hoc. And I, you know, I argued this in an essay that I wrote, but I think, you know, effective altruism and to an extent, effective acceleration, or sorry, the other way around, effective accelerationism and to an extent, effective altruism are really just sort of a rebranding of a lot of the same philosophies that have existed in Silicon Valley for a long time, where people really want to feel good about themselves, like they're, you know, pursuing a higher cause, not just the pursuit of massive wealth. And these philosophies are very convenient because they allow people to define themselves as these hero figures who are, you know, accumulating wealth, but not because they're

• 19:38Speaker 01

  greedy, not for their own purposes, it's for a higher cause. You know, in the case of the altruists, it's because they're going to donate it all at some point in the future, maybe. In the case of accelerationists, you know, they're creating all of this wealth, they're accumulating all this wealth in the pursuit of this technological goal that is, you know, completely beyond criticism, because anything other than that is akin to murder. And so, you know, I think it's a, I think it's a philosophy of convenience in a lot of cases.

• 20:09Speaker 02

  Also, I just want to step in before we get too far away, which is that like, they're not opposites in the sense that like EA are communists. Like, they're both pretty aligned on the idea of making as much money as humanly possible. They're both supported by a lot of anti-democratic thinkers. They're both sort of center to not maybe just full-on right-wing. Like, they're not total opposites. They're just sort of aesthetic. Most of it is aesthetic, I would argue, but then also, yeah, resources internally, sure. But like, they're not, they're not opposed to each other. One is not like a workers' party.

• 20:48Speaker 01

  Right. And I would add something that I wrote about, which is that, you know, I think recently, we've been seeing this framing of the AI debate as being between the effective altruists and the effective accelerationists, as though those are the two sides of the debate, when in reality, that is a very small portion, and it's a very extreme and very loud portion of the debate. But there is sort of this vast history of people who are, have been studying artificial intelligence, machine learning, you know, ethics, all these different things for decades, who are like, hello, you know, like, we have a lot to say here. We are not the shiny meme-y effective accelerationists, but you know, we have been thinking about this.

• 21:28Speaker 02

  And that's actually a really good point where it's like, Molly makes a really good point here, where like, those are two buckets of the people who love AI. And it's not dissimilar from the AI is going to destroy the world versus AI is going to fix the world debate, which is not really a debate. That's just like a marketing exercise we're watching being played out like in front of Congress sometimes. So, these are not really opposites in any way.

• 21:53Alex Kantrowitz?

  Okay, I want to bring in deeper here, because you're, the headline that you have on your story seems to sort of paint a different picture.

• 22:00Speaker 03

  I, I think it's not necessary. I think effective altruism is so polarizing that it does split the industry. I think one group that has also served as a foil for the EA people, or what's called the AI ethics group. So, these are the people that think we should probably focus on current problems, things like bias and misinformation and all kinds of things. Those are the people that are actually getting squeezed at OpenAI, right? Like, that's a lot of the trust and safety folks that I spoke to.

• 22:37Speaker 03

  Were felt like they were more in that bucket. And the that at least that they thought the OpenAI team that did trusted safety was more in that bucket. And they were pushing really hard against this movement, but it just didn't have the same clout. Right. And so, and then you also see this others, this other group, the effective accelerationists that just think it's, you know, again, like murder to slow down AI development, but the AI ethics people are like, but we don't. This is an art. These are not our people either, right? So they're just kind of stuck in the middle. Trying to find ways to minimize harm and not a lot of people are listening to them. At all the companies.

• 23:20Alex Kantrowitz?

  But but how did this factor in in the Altman? Episode, right? Because the narrative is that he did get pushed out by the EA group and he was more aligned with the acceleration folks. And in fact, I think this is from your story. He called effective altruism an incredibly flawed movement with very weird emergent behavior. Good take.

• 23:42Speaker 03

  I I think I think it it played out in a couple of ways. I think first, it's probably important to note that the board in their statement as sort of lacking in detail as it was, said that they didn't fire Sam for anything but lack of candor, right? Like they didn't fire him because he wasn't AI safety enough. But our understanding is that there were a lot of AI safety debates. This there was constant tension about, you know, the pull to commercialize versus the pull to versus the pull to to slow down and like, think about the long term risks, you know, while the trust and safety people are like, look at me like they're

• 24:24Speaker 03

  trying to get everyone's attention and not really succeeding either. But Sam. I think got into some disputes with board members a little bit about these issues.

• 24:35Alex Kantrowitz?

  Because the board was EA heavy.

• 24:37Speaker 03

  Yeah, well, it two of the board members were associated with EA organizations. So that's Helen Toner, who used to work at Open Philanthropy, which is a very EA organization. And

• 24:51Alex Kantrowitz?

  Tasha McCauley.

• 24:51Speaker 03

  Tasha, right.

• 24:52Alex Kantrowitz?

  And

• 24:53Speaker 03

  And she was on.

• 24:53Alex Kantrowitz?

  And Ilya also, who's.

• 24:55Speaker 03

  If you talk to people who know Ilya, he's like, I'm not EA. I'm AI safety, which is like a subgroup, I guess, of this, right? So I mean, partly, this is a little, a hard conversation I have because there's, it's not like you're a Democrat or a Republican and you've registered. You have like, it's a big.

• 25:12Alex Kantrowitz?

  It's more religious, right?

• 25:13Speaker 03

  Right, so there's a.

• 25:14Speaker 01

  And the names are all very similar and somewhat overlapping. Like AI safety is different from AI ethics, apparently, you know? So it's like.

• 25:23Speaker 02

  It's like, it's like types of milk. It's like 2% whole soy, you know? They're all kind of different.

• 25:30Speaker 03

  Yeah, yeah, well, they're all milk, right?

• 25:32Speaker 02

  They're all pro-milk, they're all very pro-milk, yeah.

• 25:36Alex Kantrowitz?

  If you think everybody's so similar, what do you, do you think there was an ideological dispute there? Because it seems like in your piece that you thought there was.

• 25:43Speaker 02

  Yeah, no, I mean, like, I have no trouble believing that a bunch of nerds read too many blog posts and destroyed a company over it, like, and then that it's happening, having ripple effects across Silicon Valley, like, yeah, no, I, I, I am extremely dismissive of this stuff. And I think it's very silly, actually. But I do think that the people who believe it take it very seriously. I, I sort of view AI as a bottleneck for these groups because like effect, especially the effect of altruists, who, you know, they could be focusing on COVID-24 or like vaccines for the swine flu that just broke out or whatever. But they are obsessed with AI because they've determined that it is the ultimate risk to humanity. Because it's exciting and interesting and it like makes for a good post. And, and that is sort of my take across the board with these groups is that like.

• 26:28Speaker 02

  They all require the buy-in that AI is important and not just autocorrect.

• 26:33Alex Kantrowitz?

  Okay, so Deepa, sorry, you were trying to take us through exactly what happened there between these two groups.

• 26:39Speaker 03

  Right, so, you know, so Sam would have like conversations with different people at the organization. And there were a lot of debates inside OpenAI about AI safety and where is the line and how do we navigate the line and whatever. But it wasn't so much his stance that bothered the board. It was the response, like how he treated the board and how he engaged in these conversations. Which, again, the board hasn't provided a ton of detail here, so it's very easy to wildly, wildly speculate. But I think it's safe to say that they felt like he was lying, right? Or that he was misleading them in the way he responded to two different things. A lot of this is still very up in the air.

• 27:24Speaker 03

  I'm working hard to try to get examples. I'm trying, you know, but it is, but it has been a little bit frustrating. Because if the board is going to say, hey, Sam lacked candor in these conversations, like what happened, right? And how, give me a specific example of how he responded. And what did AI safety specifically, like what conversations around AI safety were so

• 27:50Speaker 03

  made him respond in such a, in a way that made the board really take an extraordinary step to fire him, right? There's still a lot of open questions there.

• 27:59Alex Kantrowitz?

  Yep, and right now, what we're kind of seeing is that, you know, people using the framing that

• 28:07Alex Kantrowitz?

  this was this battle between EA and accelerationists, like people are saying the accelerationists won and EA has lost. And EA has, you know, had taken a couple of setbacks here with the OpenAI situation and this Sam Bankman-Fried situation. So Molly, how would you say that that they are responding and what do you think this means for that movement?

• 28:28Speaker 01

  Well, I mean, I think it's kind of the same way that crypto responded to the SBF movement, where they can just be like, "Oh, that wasn't us." You know, like, there's a lot of reframing and redefining of terms when things seem to go poorly for a movement. And so, I, you know, I wrote this piece about effective altruism and effective accelerationism. And the vast response that I've received from people who define themselves to be effective altruists is like, "Oh, that's a totally different group of people. You know, that's not what EA is all about," trying to sort of downplay what has happened recently, or sort of do the, you know, "No true Scotsmen" around, like, "Those aren't real effective altruists"

• 29:10Speaker 01

  type of things. But, you know, like I said earlier, I think it's really hard to sort of cast such a broad net around a movement that has, you know, a very broad range of people in it. But I do think you're right, that the takeaway that a lot of people got was, "Effective accelerationists have won, effective altruists have not. You know, we should be, clearly because of this, you know, the winning philosophy is just to develop AI with no brakes and, you know, go completely pedal to the metal on it," which I think is concerning, you know, especially again, because I feel like it removes a whole important part of that conversation. You know, not only are there more people than effective altruists

• 29:55Speaker 01

  and effective accelerationists at OpenAI having these conversations, there's a lot more in AI than OpenAI. And so, you know, the idea that effective accelerationists now will be dictating the future of this entire field, I think, is, it's something that risks becoming sort of a self-fulfilling prophecy, where we now give these people far more attention and sort of, you know, weight to their opinions than probably ought to happen.

• 30:25Alex Kantrowitz?

  Yeah, Molly, you've written about this, but like, one of the things that we've also seen is that like, both groups believe that AGI are like really scary doom type of AI applications are right around the corner. Which like, if you're looking at the technology, you're like, what? Like even this big Q star, you know, revelation that came over the weekend about OpenAI like, you know, it seems to be not a complete nothing burger, but not like a, you know, 10 alarm fire like people are making it out to be. So why are they also convinced?

• 30:54Speaker 01

  Speaks a little bit to Ryan's point that like, there isn't actually that much of a difference between these two groups. They do, they do diverge on sort of a very important point, but they both have this sort of mythology in their heads about this, you know, godlike AI being that is just around the corner. And, you know, I think it's important to look at the historical record when it comes to that. We have seen predictions of gods being created since, I mean, about as long as we have a historical record.

• 31:25Alex Kantrowitz?

  Molly, Bitcoin could go to a million. So hyper Bitcoinization could happen.

• 31:33Alex Kantrowitz?

  AGI and Bitcoin are going to the moon.

• 31:37Speaker 01

  Yep. Yeah. But like, I mean, you know, we heard this with the sort of sparks of artificial general intelligence paper. We've seen this, yeah, we've seen this, you know, for years and years in the computing sector and far longer than that. Just in the sort of religious side of things. And so I think, you know, it, it's one of those things where people get very tied up in the mythology and the religion behind it. And they don't necessarily pay all that much attention to the technology. And people begin to really change their own definitions of what artificial general intelligence means, what sentience means, what humanity means,

• 32:20Speaker 01

  in order to make these things seem more plausible or more true than they really are.

• 32:26Speaker 02

  I also don't think it's an accident that we're hearing about all of this stuff, right? As every major company has rolled out an AI widget for their service and it's sort of been a flop. Like, we're in like an extremely, uh, like, kind of boring moment for AI in between releases. The hype is sort of dying down as it does every time we get a new version of these tools. And now all of a sudden, there's like two competing like doomsday cults around this idea. Like, that doesn't seem like totally an accident to me.

• 32:58Alex Kantrowitz?

  Yeah, Deepa, like from a practical level, like, how is this playing out? You know, in corporate boards and within AI companies, is it gonna continue to spiral?

• 33:07Speaker 03

  I was talking to somebody about this last week who said that they felt that the OpenAI saga and all the speculation about EA was more damaging for EA than Sam Bankman-Fried. Because with SBF, you can look at him and be like, that's one guy. And this is like a movement. It's like a lot of people acting in unison, apparently, irrationally, right? Like, that is the, the, the attitude and that's the fee, that's the vibe right now. And I think there's a lot of truths to that. I think that, you know, EA people already are viewed as a really insular, really kind of clubby organization that they only ever cite EA papers written by other EA people

• 33:48Speaker 03

  and they only ever want to work with EA people. This isn't obviously everybody. This is a stereotype. But that, I wonder if the double hit to their integrity and their image is going to force like a further contraction or where they just sort of like go deeper. But the reality is also that over the last year, this movement has had a lot of influence, not just on a practical level, like on the ground at AI companies, even though that's significant. Like, we haven't talked about hiring yet and a lot of EA organizations start on student campuses. And that's the pipeline. If you want to hire an AI and like researcher at this point, you're gonna hire from a serious university like Berkeley or Stanford or Oxford.

• 34:33Speaker 03

  And a lot of them are in this movement. So it's a gigantic overlap. So you're probably gonna hire a bunch of EA people and you're gonna have to make them happy because they could leave at any time, right? There, there's huge competition for people to actually build these systems. So there's that, but then, you know, so they'll still have a lot of pull just from the hiring perspective. They're also have a lot of pull on a policy side. I mean, if you look at some of the comments from the EU about AI risks, they talk about existential risk. Like Grishi Sunak's entire AI safety summit, a lot of that were, a lot of the speakers were EA aligned or EA. The White House even, like definitely, you know, Dario Amadai and like other people who are broadly viewed as EA people.

• 35:18Speaker 03

  They went to the White House and talked about their views, like they have a seat in the room. So I don't think it's just gonna like go away at all because they've already been there and, and they're very serious and they keep, you know, they're in the room. They can, they continue to hold some kind of influence.

• 35:35Alex Kantrowitz?

  Right. I gotta ask you what's gonna happen to Anthropic. I mean, Anthropic was seen as this like counterweight to OpenAI, but run by, you know, a lot of, I mean, I know they're like quote unquote, not associated with EA, but they have big EA influence there. They left OpenAI because of safety. You have Amazon that just invested 1.25 billion in that company. Google also invested billion, like more than a billion.

• 36:01Alex Kantrowitz?

  Are we gonna see the same stuff happen in Anthropic as we did within OpenAI? I mean, what's the future of that company now? Cause it's even more closely tied to this movement.

• 36:11Speaker 03

  Yeah, well, it's also, as, as I understand it, a little bit more homogenous, like there are a lot more EA people there, right? And there's a culture fit test, like when you get hired, where you're asked

• 36:28Speaker 03

  that, that as far as I understand, I don't have like the questions in front of me, but a lot of the questions are sound, they're basically trying to test whether or not you'd be EA enough to be at Anthropic. That's how it's been described to me.

• 36:45Speaker 03

  And they, you know, they have a philosopher on staff who

• 36:50Speaker 02

  Well, who, yeah, everybody does that.

• 36:52Speaker 03

  Right. But it's Amanda Askel. So she's the former wife, I think, or of Will McCaskill, who is the, right, the founder of EA. So there's like a big, tight connection there. They, but you know, they're not, they're, will they be split up by it? Probably not, just because they are all coming from the starting point that EA is good. It's just maybe the execution is a problem, or we are misunderstood, and it's a public issue. It, this, some of this stuff kind of reminds me a little of like what it was like covering Facebook, if you remember, Alex, like, there are a lot of people, there are a lot of people internally at Facebook that were like, well,

• 37:32Speaker 03

  no, Facebook's really good. We just need to get better at managing the bad stuff and like, making sure people can see how good we are. There's definitely that population inside the company. And it, it gives me the same echo.

• 37:46Alex Kantrowitz?

  Right. So it seems like we're, we're largely EA skeptical in this conversation. I, first of all, if someone has a counterpoint they want to make, my email is [email protected]. So hit me up there. I'm happy to listen to it. Maybe we can bring you on. But also, like, I'm curious just from the group here, like, what would someone who's steeped in EA or effective accelerationism say is like, you know, is there, is there any like rebuttal to some of these points that we're making that we should be, you know, that we should be considering?

• 38:20Speaker 02

  Yeah. I mean, they, they, I mean, Molly's point was correct, which is that like, if you, if you really talk to a person who believes in EA, like, they're, they sound very reasonable, and most of the things they're going to point to are, are not Sam Bankman-Fried committing financial crimes, right? Like, the, it's, it's mainly just that when you start to take that long-term view, things start to become normalized that possibly wouldn't be if you weren't looking so far in the future. That's why a lot of EA guys end up showing their ass on Twitter, like, posting weird stuff about race science and like demographic shifts and stuff. And with the accelerationists, like, I, I mean, I, I, I hesitate even calling it a philosophy. I think

• 39:05Speaker 02

  it's still very much just a meme. Like, it's the Dogecoin to EA's Bitcoin, right? Like, it's not really anything. Yet, it could be. And in fact, like, I think that if there's really any legacy from the last week of drama inside of OpenAI, it's that the accelerationists all found each other and like now know how to talk about each other. And now we'll start to see sort of like those memes becoming more serious and becoming like, you know, possibly a real counterpoint as opposed to a bunch of different factions wanting different things.

• 39:33Alex Kantrowitz?

  You do have people like with their Twitter bios, like Gary Tan, the head of Y Combinator is like, right in his bio, he calls himself an effective accelerationist.

• 39:42Speaker 02

  Yeah.

• 39:43Speaker 01

  Yeah, I think, you know, I think a victory of the past weeks, week or two weeks or so, is just the attention that has been given to the effective accelerationism idea. Again, I agree, it's not really a philosophy. I wouldn't even really call it a movement. I think it is mostly a rebrand of, you know, move fast break things, which has been Silicon Valley's, you know, mantra since early Facebook. But with the added sort of religious, almost philosophy behind it, and the, the very effective altruist style of speaking and writing in this very sort of esoteric and long-winded way.

• 40:29Speaker 01

  It's, I think it's, uh, it's, it's probably, it's something that I don't give a lot of weight in terms of its own sort of

• 40:40Speaker 01

  foundation, the, the quality, I guess, of the ideas behind it. But I do think that the effectiveness of its proponents in terms of spreading it, sort of memifying it, making it appealing to especially younger people who are just getting into tech and trying to find a way to think about their place within this sort of huge capitalist structure, is something that probably should be taken seriously.

• 41:11Speaker 01

  As well as the tendency of some of these so-called movements, you know, except effective altruism being one of them, to almost radicalize the people within them as they normalize these sort of thought exercises that can lend, like, lead down a very concerning path.

• 41:30Alex Kantrowitz?

  Ryan, I'm kind of curious, like you read the message boards. I mean, don't message boards and online forums have a tendency to radicalize and also like make the most intense ideas rise to the top? Is this just kind of like a classic case of that?

• 41:45Speaker 02

  I've never heard of that ever happening before. What could you possibly be referring to? Yeah, I mean, no, it does. I mean, that's definitely the case. I mean, we didn't even really go further into this because it's so dense. But the reason I was even writing about effective accelerationism this week is because I got sent a tip about a group that was causing trouble on TikTok from my readers. And when I started poking around, it turned out that they were a group I'd written about previously of crypto accelerationists that had rebranded into AI accelerationists and were part of the New York downtown scene trying to get Peter Thiel's money. Like, these are the same people that have been kicking around for five years trying to bring about some kind of white nationalist apocalypse via automation. And

• 42:28Speaker 02

  they went from cryptocurrency to AI. And it's the same people doing the same stuff, sharing the same memes that they've been sharing since 2017. And I think when you talk about the radicalization of these message boards of these online communities that are talking about these philosophies, what it is is that you don't know who you're talking to and you don't know what their goals are, where their funding's coming from, or what they're influenced by. So, you know, it's very common to be reading like a Twitter thread and all of a sudden, like if you don't know, I mean, Molly or Deepa might be able to do this, but I don't think a normal person would be like, oh, yeah, there's the EA person, there's the rationalist, there's the neo-reactionist, there's like the techno-feudalist Muldas Mogberg fan or whatever. Like, normal people don't have the time to care about this shit.

• 43:11Speaker 02

  And unfortunately, like, people in Silicon Valley really care about it. And so, it's very thorny. And we don't really know where this stuff is traveling or how it's mutating until it's in front of our faces.

• 43:23Alex Kantrowitz?

  And one of the things that I wrote down before we started is why do we need these movements? Like, why can't we just have folks that are building cool things and, you know, saying they're trying to build cool stuff like ChatGPT is a cool product? Like, you don't really need a like a religious philosophy.

• 43:40Speaker 02

  I do think there was like a schism around Clippy in the 90s. There were the pro Clippiists and the anti Clippiists. And this was very similar, I think.

• 43:48Alex Kantrowitz?

  Exactly.

• 43:50Speaker 03

  But there have always been, there have been movements around technology. Like, who should technology be for? What should it look like? And this is kind of what we had asked of Facebook, right? Like, why didn't you guys think about the harms beforehand?

• 44:01Alex Kantrowitz?

  Connecting people is de facto good.

• 44:03Speaker 03

  Right, right. But maybe it's not. And we asked that of the tech companies. And so I think the defense that you'd hear from EA groups is like, look, like, these are real problems and we're going to focus on the problems. It's just going to be a debate about like what the problems, what problems are most, you know, valuable to focus on.

• 44:23Alex Kantrowitz?

  Yeah.

• 44:23Speaker 03

  Yes.

• 44:24Alex Kantrowitz?

  I mean, in retrospect, if AI does wipe us all out, like next week, this podcast is going to look dumb. So.

• 44:30Speaker 02

  I can't wait. Yeah. Put me out of work. Give me some UBI. But I do think that like, you're right that this stuff goes all the way back. I mean, the Protestant Reformation, one version of it is to blame it on the accessibility of literature and the printing press and the ability to read the Bible and go, wait a minute, I don't agree with this. Right? So like every technological revolution, if we're really in one right now, does typically involve some sort of political, social, religious upheaval. I just think the tech guys are so excited about the idea that they're in a revolution that they may have like invented a religion before they've been able to prove if we're in one or not.

• 45:05Speaker 01

  And I think there's also this, this sort of urge among some of the wealthier people in Silicon Valley to try to ascribe a greater meaning to the work that they do. And these types of philosophies are very appealing to those people. You know, Marc Andreessen being one of the most prominent who wrote this literal, he called it a manifesto. Um, you know, I think, I think that that just sort of exposes that, you know, he is sort of an embarrassed billionaire to some extent, who wants to define his legacy as more than just amassing wealth off of venture capital. And so by adopting this religion and becoming one of its preachers, you know, I think that's a way to do that for some of these people.

• 45:50Speaker 01

  And I think that's why you see people in those positions become drawn to this. And I also think they realize that it is a very effective way to spread their own influence by becoming the, you know, leaders of these sort of movements. They realize that, you know, they can preach to a very willing audience who might not otherwise listen if they're just saying, you know, here's this new tech thing that I'm interested in. Don't you think it's cool?

• 46:17Alex Kantrowitz?

  So just stitching everything together, my takeaway here is that we've really just seen kind of round one, that like the story doesn't end with OpenAI. It almost begins here with the accelerationists starting to find each other and EA, you know, maybe reforming, but not going away.

• 46:34Speaker 02

  Yeah.

• 46:34Alex Kantrowitz?

  And we'll see. I mean, hopefully there's room for others, right? That's, that's the real question.

• 46:39Speaker 02

  I, I actually, to your point, I will say, I think that the philosophies that we've seen sort of emerge and the rebranding of these philosophies that we've seen happen over the last month or so, have a good chance of outliving the generative AI fad if, if we're in one. Like, we may lose interest with mid-journey next week, but I think these people are going to be around and sort of thinking about this stuff in this way for a lot longer.

• 47:02Speaker 01

  I think that's very true. I mean, if you look at crypto, a lot of those people have moved into effective accelerationism without even blinking. And, you know, the, the crypto was the, the last old interesting thing. And now it's artificial intelligence or AGI or whatever they want to focus on. But, you know, they, they're continuing the same beliefs and, and behaviors with just whatever is in front of them.

• 47:26Alex Kantrowitz?

  Great. All right. So let's, uh, let's round this one out. I just want to give everybody, um, who's listening a chance to, um, find our great guests online. So Molly White's, uh, oh, it's called Citation Needed Now. Okay. Remember you rebranded it. Newsletter dot Molly White dot net. And then Ryan Broderick's, uh, Garbage Day, garbage day dot email. And you can find Deepa's work on WSJ.com. Okay. Well, thank you, Molly, Deepa and Ryan. This has been a great conversation. Really appreciate you coming on and helping us break this down. And I also like really love that, you know, you didn't take it face value that this is a one versus one, but kind of broadened it out and talked about what we're really looking at here, which is great.

• 48:07Speaker 02

  Wait, were we supposed to fight each other? Oh, no, no. I see. The philosophy.

• 48:10Alex Kantrowitz?

  Yeah, yeah, yeah, exactly. So thank you so much.

• 48:13Speaker 01

  Ryan and I will one versus one each other at a different point.

• 48:16Speaker 02

  For sure.

• 48:17Alex Kantrowitz?

  We will host it. We'll host it. Um, yeah. So thank you so much. Thank you to our guests. Thank you to our listeners. Um, and thank you to everyone who helps make this podcast possible. We'll be back on Friday with another show, breaking down the week's news and continue with our coverage of these religions of OpenAI, of the AI field every week, Wednesdays and Fridays. All right. We'll see you on Friday on Big Technology Podcast.

• 48:44Speaker 04

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid and risked 350,000 Virginians healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com, not authorized by any candidate or candidate's committee.

• 49:15Speaker 05

  This week, it pays extra to be a Milo's Rewards member. Through October 14, members get $10 Milo's money when you spend $200 or more on an eligible purchase during Milo's Money Days. Also, using your Milo's Rewards credit card, even better, that's an additional $20 Milo's money. Use it to save on your next Lowe's haul. Exclusions, restrictions and more terms apply. See Lowe's.com slash Milo's Money Days for details. Milo's money expires 30 days after issuance. Subject to change. Credit offer subject to credit approval.