Mayhem at OpenAI + Our Interview With Sam Altman

Last week, we interviewed Sam Altman. Since then, well, everything has changed. The board of OpenAI, maker of ChatGPT, fired Altman as chief executive on Friday. Over the weekend, it looked as if he might return. On Sunday night, Microsoft hired Altman to lead a new A. I. venture. Who knows what will happen next. Today, an update on a crazy weekend in tech, and our interview with Sam Altman.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 0:35Kevin Roose?

  Casey, how was your weekend?

• 0:37Casey Newton?

  What? There was no weekend. There was only work.

• 0:40Kevin Roose?

  That was a trick question.

• 0:41Casey Newton?

  And there will only ever be work. Yeah.

• 0:44Casey Newton?

  What is happening, Kevin?

• 0:46Kevin Roose?

  I don't know, man. I am on two hours of sleep. I've been working all weekend and I'm increasingly certain that we are, in fact, living in a simulation.

• 0:56Casey Newton?

  I mean, it would be nice if we were living in a simulation because that would suggest that there is at least some sort of plan for what might be about to happen next. But I think recent events would suggest that actually there is not.

• 1:06Kevin Roose?

  Yeah, I had a moment this morning where I woke up and I looked at my phone from my two-hour nap and I was like, I'm huffing fumes. This can't be real.

• 1:15Casey Newton?

  I mean, let's just say, like, over the course of a weekend, OpenAI as we know it ceased to exist. And by the time this podcast gets put in the air, I would believe anything it told me about the future of OpenAI, up to and including it had been purchased by Etsy and it was becoming a maker of handcrafted coffee mugs.

• 1:38Kevin Roose?

  Honestly, would not be the strangest thing that's happened this weekend.

• 1:42Casey Newton?

  Not remotely. Wouldn't be in the top five.

• 1:49Kevin Roose?

  I'm Kevin Russo, tech columnist for The New York Times.

• 1:52Casey Newton?

  I'm Casey New from Platformer.

• 1:53Kevin Roose?

  And this is Hard Fork.

• 1:54Casey Newton?

  This week on the show, one of the wildest weekends in recent memory will tell you everything that happened at OpenAI and what's going on with Sam Altman. And then later in the show, we will present to you our interview with Sam Altman from last week. So before he was fired, we asked him about the future of AI and we're going to share that conversation with you.

• 2:26Kevin Roose?

  So this episode is going to have two parts. The first part, we're going to talk about the news that happened at OpenAI over the weekend and run down all of the latest drama and talk about where we think things are headed from here. And then we are going to play that Sam Altman interview that we discussed on our last emergency podcast, the one that we conducted last week and planned to run this week, but that has since become fascinating for very different reasons. So let's just run down what has happened so far because there's been so much. It's like enough to fill one of those epic Russian novels or something. So on Friday-

• 3:03Casey Newton?

  And the good news, by the way, is that the events are all very easy to understand. There's no way you'll mess up while trying to describe what happened over the past three days.

• 3:10Kevin Roose?

  Yeah, let's try this on a couple hours of sleep. Okay. So Friday, when we recorded our last emergency podcast episode, Sam Altman had just been fired by the board of OpenAI. He was fired for what were essentially vague and unspecified reasons. The board put out a statement sort of saying that he had not been candid with them, but they didn't say more about what exactly had led them to decide that he was no longer fit to run the company. So he's fired. It's this huge deal, huge shock to all of the people at OpenAI and in the tech industry. And then it just keeps getting weirder. So Greg Brockman, the president and co-founder of OpenAI, announces that he too is quitting.

• 3:53Kevin Roose?

  Some other senior researchers resign as well. And then Saturday rolls around and we still don't really know what happened. Brad Lightcap, who is OpenAI's COO, sent out a memo to employees explaining that they know that Sam was not fired for any kind of malfeasance, right? This wasn't like a financial crime or anything related to like a big data leak or anything. He says, quote, "This was a breakdown in communication between Sam and the board."

• 4:22Casey Newton?

  And let's say that by the time that Brad put that letter out, there had been reporting that at an all hands, Ilya Sutskever, the chief scientist at the company and a member of the board, had told employees that getting rid of Sam was the only way to ensure that OpenAI could safely build AI, which led to a lot of speculation and commentary that this was an AI safety issue driven by effective altruism on the board. So it was very significant when we then get a letter from Brad saying explicitly, this was not an AI safety issue. And of course, that only served to make us even more confused. But lucky for us, further confusion with them.

• 5:05Kevin Roose?

  This was actually the clearest that things would be for the rest of the next 48 hours. So OpenAI, its executives are saying this isn't about safety or anything related to our practices. But what we know from reporting that I and my colleagues did over the weekend is that this actually was at least partially about AI safety and that one of the big fault lines between Sam Altman and the board was over the safety issue, was over whether he was moving too aggressively without taking the proper precautions.

• 5:38Casey Newton?

  Yeah.

• 5:39Kevin Roose?

  After this memo from the COO went out, there were reports that investors in OpenAI, including Sequoia Capital, Thrive Capital, and also Microsoft, which is the biggest investor in OpenAI, were exerting pressure on the board to reverse their decision and to reinstate Sam as CEO and then for the entire board to resign. They had sort of a deadline for figuring some of this stuff out, which is 5:00 PM on Saturday. That came and went with no resolution. And then Sunday, a bunch of senior OpenAI people, including Sam Altman, who is, by the way, now no longer the CEO of this company officially,

• 6:23Kevin Roose?

  gather at the offices of OpenAI in San Francisco to try to work through this all.

• 6:29Casey Newton?

  That's right. There is some reporting that all All of a sudden, at least some people on the board are open to the idea of Sam returning, which was one of those moments that was both shocking and not at all surprising. Shocking because they had just gotten rid of him, not at all surprising because I think by that point, it had started to dawn on the world and on OpenAI in particular, on what it would mean for Altman to no longer be associated with this company, where he had recruited most of the star talent.

• 6:58Kevin Roose?

  Totally, and the employees of OpenAI were sort of making their feelings known as well. They did this sort of campaign on X on Saturday where they were posting, you know, heart emojis in sort of quote posts of Sam, sort of indicating that they stood by him and that they would follow him if he decided to leave and, you know, start another company or something like that.

• 7:23Casey Newton?

  Yeah, it was something to behold. It was essentially a labor action aimed at the board. And what I will say was, in this moment, you realize the degree to which the odds were weirdly stacked against the board. Because on one hand, the board has all of the power when it came to firing Sam. But beyond that, there is still a company to run, there is still technology to build. And so now you had many employees of the company being very public in saying, hey, we do not have your back, we did not sign up for this, and you're in trouble.

• 7:57Kevin Roose?

  Yeah, and so on Sunday, there was a moment where it sort of looked like Sam Altman was going to return and sort of take his spot back as the CEO of this company. He posted a photo of himself in the OpenAI office, wearing a guest badge, like one that you would give to a visitor to your office.

• 8:14Casey Newton?

  I will say, I have worn that exact badge at OpenAI headquarters before.

• 8:18Kevin Roose?

  Yeah, and so the caption on the photo was something like, this is the first and last time I'll ever wear one of these. So it kind of sounded like he was setting the scene for a return as well. And I would say there was just a feeling among especially the company's investors, but also a lot of employees and just people who, you know, work in the industry, that like this wasn't going to stand, that there were too many mad employees, that the stakes of kind of blowing this company up over this disagreement were too high. And if there really wasn't a smoking gun, right? If there was really nothing concrete that the board was going to hold up and say, this is why we fired Sam Altman, there was this sense that that just wasn't going to work, that there was no way that the board could actually go through with this firing.

• 9:02Casey Newton?

  Yeah, and I think one way that the employees and the the former executives were very effective was in using social media to create this picture of the overwhelming support that was behind them, right? So if you were an observer to this situation, you're only seeing one side of the story, right? Because the board is not out there posting, they haven't issued a statement that lays out any details about what Sam allegedly did. And so instead, you just have a bunch of people saying like, hey, Sam was a great CEO. I love working for the guy. OpenAI is nothing without him. All these posts are getting massively reshared. It's easy to look at that and think, oh, yeah, he's probably going to be back in power by the end of the day.

• 9:40Kevin Roose?

  Totally. So that was the scene as of Sunday afternoon. But then, Sunday evening, Pacific time, this new deadline, 5:00 PM Pacific time, has been given for some kind of resolution. That also comes and goes. And there is no word from OpenAI's headquarters about what the heck is going on. It sort of feels like there's like a papal conclave, and everyone is waiting for the the white smoke to emerge from the chimney. And then, we get word that the Board of Directors of OpenAI has sent a note to employees announcing that Sam Altman will not return as CEO after all, and sort of standing by its decision.

• 10:21Kevin Roose?

  They still didn't give a firm reason or a specific reason why they pushed him out, but they said that, quote, "Put simply, Sam's behavior and lack of transparency in his interactions with the board undermine the board's ability to effectively supervise the company in the manner it was mandated to do." And they announced that they have appointed a new interim CEO. Now, remember, this company already had an interim CEO, Mira Maroti, the former Chief Technology Officer of OpenAI, who had been appointed on Friday. She also signaled her support for Sam and Greg, and reporting suggests that she actually tried to have them brought back. And because of that, the board decided

• 11:05Kevin Roose?

  to replace her as well. So Mira Maroti's reign as the temporary CEO of OpenAI lasted about 48 hours before she was replaced by Emmett Shear, who is the former CEO of Twitch, and who was the board's choice to take over on an interim basis.

• 11:23Casey Newton?

  The board found an alternative man, or Altman, to lead the company.

• 11:30Kevin Roose?

  So that was already mind blowing. This happened, you know, at night on Sunday and I thought, well, clearly things cannot get any crazier than this.

• 11:39Casey Newton?

  That's when I went to bed, by the way. I was like, whatever's happening with these people can wait till the morning. And then, of course, I wake up and an additional four years worth of news has happened.

• 11:50Kevin Roose?

  Yes. So after this announcement about Sam Altman not returning and Emmett Shear being appointed as the interim CEO, there is a full on staff revolt at OpenAI. The employees are outraged. They start sort of threatening to quit. And then just a couple of hours after this note from the board of directors comes out, Microsoft announces that it is hiring Sam Altman and Greg Brockman to lead an advanced research lab at the company.

• 12:20Casey Newton?

  An advanced research lab, I assume means that Satya has just given those two a fiefdom and they will be given an unlimited budget to do whatever the heck they want. But of course, because Microsoft owns 49% of OpenAI, at this advanced research unit, Sam and Greg and all their old friends from OpenAI will have access to all of the APIs, everything that they were doing before. They will just get to pick up where they lift off and build everything that they were going to do. But now firmly under the auspices of a for-profit corporation. And by the way, one of the very biggest giants in the world.

• 12:54Kevin Roose?

  Yeah. So I think it's worth just pausing a beat on this move because it is truly a wild twist in this saga. So just to explain why this is so crazy. So Microsoft is the biggest investor in OpenAI. They've put $13 billion into the company. They're also sort of highly dependent on OpenAI because they've now built OpenAI's models into a bunch of their products that they are kind of betting the future of Microsoft on in some sense. And this was a big bet for them that over the course of a weekend was threatening to fall apart, right? Sam Altman and Greg Brockman were the leaders of OpenAI. They were the people that Microsoft was most interested in having run the company.

• 13:36Kevin Roose?

  Microsoft did not like this new plan to have Emmett Shear take over as CEO.

• 13:41Casey Newton?

  And they said it's sheer madness, Kevin.

• 13:45Kevin Roose?

  And so they did kind of the next best thing, which was to poach the leaders of OpenAI, the deposed leaders, and bring them into Microsoft, along with presumably many of their colleagues, who will be leaving OpenAI in protest if the board sticks to this decision.

• 14:04Casey Newton?

  Yeah, man. So this one threw me for a loop because if you have spoken with Sam or Greg or many of the people who work at OpenAI, you get the strong impression, these people like working at a startup, okay? Working at OpenAI is, in many ways, the opposite of working at a company like Microsoft, which is this massive bureaucracy with, you know, so much process for doing anything. I think they really liked working at this nimble thing, at a new thing, being able to chart their own destiny. Keep in mind, OpenAI was about to become the sort of only big new consumer technology company that we have seen in a long time in Silicon Valley. And so initially, it's like, okay, they're going to work at Microsoft? What the heck?

• 14:49Casey Newton?

  Because, Kevin, one thing you didn't mention, which is fine, because we did have to get through that timeline. But it was like, the instant that Sam was fired, reporting started leaking out. He was starting a new company with Greg, right? So my assumption had been, these guys are going to go off back into startup land. They're going to raise an unlimited amount of money and do whatever they want. At the same time, you think about where they were in their mission when all of this happened on Friday, and they had a very clear roadmap, I think, for the next year. And if they would have to go out, raise money, build a new team, train a large language model, think about how much time it would take them just to get back to where they were before, right? They would probably lose a year,

• 15:34Casey Newton?

  if not more, of development. So, and this is pure speculation, but my guess is that part of their calculus was, look, if we, if we deal with the devil we know and we go to Microsoft, we get to play with all of our old toys, we will have an unlimited budget, and we can skip the fundraising and team building stage and just get back to work. So I have to believe that that was the calculus. But, you know, that said, it still was a very unexpected outcome, at least to me.

• 16:00Kevin Roose?

  It's a crazy outcome, and it means that Microsoft now... Has a hand in two essentially warring AI companies, right? They have what remains of OpenAI and they have this long-term deal with OpenAI. And they also control, by the way, the computing power that OpenAI uses to run its models, which gives them some leverage there. So it is a fascinating position that Microsoft is now in and really makes them look even more dominant in AI than they already did.

• 16:28Casey Newton?

  That's right. But, but listen, all of that said, everything that you just said is true as we record. However, Kevin, by the end of the day, I would believe any of the following scenarios. Greg and Sam have quit Microsoft. Greg and Sam are starting their own company. Greg and Sam have returned to OpenAI. Greg and Sam have retired from public life. Greg and Sam have opened an Etsy store. This is all within the realm of possibilities to me. Okay? So if we're back doing another one of these emergency pods tomorrow, I just want to say that while I accept that everything that Kevin just said is true, I'm only 5% confident that any of it lasts to the end of the week.

• 17:04Kevin Roose?

  Yes, we are still in the zone where anything can happen. In fact, there have been some things that have had, that have happened even since the Microsoft announcement. So, super early on Monday morning, like 1 a.m. Pacific time, when I was still up, but I guess you were asleep, because some of us are built for the grindset. Emmett Shear, the new interim CEO of OpenAI, put out a statement saying that he would basically be digging into what happened over the past weekend, speaking to employees and customers, and then trying to kind of restore stability at the company. And my read of this letter was that he was basically telling OpenAI employees, you know, please don't quit.

• 17:47Kevin Roose?

  I am not the doomer that you think I am, and you can continue to work here. Because one other thing that we should say about Emmett Shear is that while we don't know a ton about his views on AI and AI progress, he has done some interviews where he's indicated that he is something of a, of a AI pessimist, that he doesn't think that AI should be going so quickly ahead, that he wants to actually slow it down, which is a position that is at odds with what we know Sam Altman believes.

• 18:17Casey Newton?

  Yeah, as soon as he was named, people found a recent interview he gave where he said that his P doom, his probability that AI will cause doom was between 5 and 50%. But if you listen to that interview, it sure sounds like the P doom is closer to 50 than this to 5, I would say. The other interesting thing in that statement is that Emmett said, before he took the job, he checked on the reasoning behind firing Sam. And he said, quote, "The board did not remove Sam over any specific disagreement on safety. Their reasoning was completely different from that." So once again, we have someone talking about the firing without telling us anything and making it even more confusing.

• 18:56Kevin Roose?

  Totally. But that is not even the end of the timeline. We are still going because after this 1 a.m. memo from Emmett Shear, OpenAI employees start collecting signatures on what amounts to an ultimatum, saying that they will quit if the board does not resign and replace Sam. And if the board does not resign and bring Sam Altman back as CEO, this letter starts going around OpenAI and eventually collects the signatures of the vast majority of the company's roughly 700 employees, almost all of its senior leadership and the rank and file, saying that if the board does not resign and bring back Sam Altman, they will go work

• 19:40Kevin Roose?

  for Microsoft or just leave OpenAI.

• 19:43Casey Newton?

  And do you know how much you have to hate your job to go work for Microsoft? These people are pissed, Kevin.

• 19:48Kevin Roose?

  And then, as if it couldn't get any crazier, just Monday morning, Ilya Sutskever, the OpenAI co-founder and chief scientist and board member who started all of this, who led the coup against Sam Altman and rallied the board to force him out, posted on X saying that he, quote, "deeply regrets his participation with the board." He said, quote, "I never intended to harm OpenAI. I love everything we've built together, and I will do everything I can to reunite the company." So that is it. That is the entire timeline of the weekend, up to the point where we are recording this episode. Casey, are you okay? Do you need to lie down?

• 20:30Casey Newton?

  Well, I do need to lie down. But you know, sometimes Kevin, when you're watching like a TV show or a movie and like the central antagonist has a sudden change of heart that's completely unexplained, there's no obvious motivation. I always feel like, wow, the writer's really copped out on this one, you know? At least give us some sort of arc. That was the moment Ilya Sutskever had this moment where, as you say, after leading the charge to get rid of Sam for reasons that the board did not specify, but that Ilya strongly hinted had something to do with AI safety, he now spins around and says, hey, time to get the band back together. I mean, just a tremendously undermining moment for the board generally and for Ilya in particular.

• 21:12Kevin Roose?

  Totally. So, right now, as things stand, there are a lot of different factions who have different feelings and emotions about what's going on. There's the people at OpenAI, the vast majority of whom are opposed to the board's actions here and are threatening to walk out if they're not reversed. There are the investors in OpenAI who are furious about how all this is playing out. So, a lot of people with a lot of high emotions and a lot of uncertainty yelling at these, what used to be four and are now three, OpenAI board members who have decided to just stand their ground and stick it out.

• 21:48Casey Newton?

  So, let's pause there because I think that while all of us agree that the board badly mishandled this situation, it is worth taking a beat on what this board's role is. When I listen back to the episode that we did on Friday, this is a place where I wish I had drilled down a little bit deeper. The mission of this board is to safely develop a superintelligence absent any commercial motive. That is the goal of this board. This board was put together with the idea that if you have a big company, like let's say a Microsoft, that is in charge of a superintelligence that, you know, and what is that? Something that is smarter than us, right?

• 22:31Casey Newton?

  Something that will outcompete us in natural selection. They didn't want that to be owned by a for-profit corporation. And something happened where three of the, at one point, at least four of the people on this board, and now it's down to three, but three of those people thought, we are not achieving this mission. Sam did something, or he didn't do something, or he behaved in some way that made us feel like we cannot safely build a superintelligence, and so we need to find somebody else to run that company. And until we know why they felt that way, there is part of me that just feels like we, we, we just can't fully process our feelings on this, right? Like, you know, I, I think it was actually really depressing to see how quickly polarizing

• 23:16Casey Newton?

  this became on social media as it sort of turned into Team Sam versus the, versus Team Safety. That's actually a really bad outcome for society, right? Because I think we do want, if we're going to build a superintelligence, I would like to see it built safely. I'm not sure that it is a for-profit corporation that is going to do the best job without having watched for-profit corporations create a lot of social harm in my lifetime, right? So I, I just want to say that, that, you know, I'm sure before the end of this podcast, we will continue to criticize the board for the way that it handled this. But at the same time, it's important to remember what their mission was

• 23:58Casey Newton?

  and, and to assume that they, that they had at least some reasons for doing what they did.

• 24:03Kevin Roose?

  Yeah, I mean, I was talking to people all day yesterday who thought that, you know, the, the money would win here, basically, that these investors and Microsoft, they were powerful enough and they had, you know, enough stake in the outcome here that they would sort of by, by any means necessary get Sam Altman and Greg Brockman back to OpenAI. And, and I was very surprised when that didn't happen, but maybe I shouldn't have been because the, as someone pointed out to me when I talked to them yesterday, who was sort of involved with the situation, said, you know, the board has the ultimate leverage here. This structure, this convoluted governance structure where there's a nonprofit that controls a for-profit

• 24:44Kevin Roose?

  and the nonprofit can, you know, vote to fire the CEO at any time, like, it was set up. For this purpose, I mean, you can argue with how they executed it. And I would say they executed it very badly, but it was meant to give the board the power to shut this all down if they determined that what was happening at OpenAI was unsafe or was not going to lead to broadly beneficial AGI. And it sounds like that's what happened.

• 25:09Casey Newton?

  That's right. Another piece that I would point to, my friend Eric Newcomer wrote a good column about this, just pointing out that Sam has had abrupt breaks with folks in the past, right? He had an abrupt break with Y Combinator, where he used to lead the startup incubator. He had an abrupt break with Elon Musk, who co-founded OpenAI with him. He had an abrupt break with the folks who left OpenAI to go start Anthropic for what they described as AI safety reasons, right? So there is a history there that suggests that, you know, right now, a lot of people think that the board is crazy, but these are not the first people to say Sam Altman is not building AI safely.

• 25:46Kevin Roose?

  Right. Here's the thing, like, I still think there has to have been some inciting incident, right? This, this does not feel to me like it was kind of a slow accumulation of worry by Ilya Sutskever and the kind of more safety minded board members that just woke up one day and said, you know what, like, it's just gotten a little too aggressive over there. So let's shut this thing down. I still think there had to have been some, some incident, something that Ilya Sutskever and the board saw that made them think that they had to act now. So, um, you know, so much is changing. We have to just keep going back to this caveat of like, we still don't know what is going to happen in the next hour to say nothing of the next day or week or month.

• 26:30Kevin Roose?

  But that is the state of play right now. And I think this is, I mean, Casey, I don't know how you feel about this, but I would say this is the most fascinating and crazy story that I have ever covered in my career as a tech journalist. I cannot remember anything that made my head spin as much as this.

• 26:46Casey Newton?

  Yeah, certainly in terms of the number of unexplained and unexpected twists, it's hard for me to think of another story that comes close. But you know, I think we should look forward a little bit and talk about what this might mean for OpenAI in particular. OpenAI was described to me over the weekend by a former employee as a money incinerator. ChatGPT does not make money. Training these models is incredibly expensive. The whole reason OpenAI became a for-profit company was because it costs so much money to build and maintain and run these models. When Sam was fired, it has been reported that he was out there raising money to put back into the incinerator. So think about the position that that leaves the OpenAI board at.

• 27:29Casey Newton?

  Let's say that they're able to staunch the bleeding and retain a couple hundred people who are closely associated with with the mission. And the board thinks that these are the right people. Who is going to give them the money to continue their work after what has just happened, right? Now, look, Emmett Shear is very well regarded in Silicon Valley. He was texting with sources last night who were sort of very excited that that he was the guy that they they chose. And so no disrespect to him, but this board has shown that it is serious when it says it does not have a profit incentive. So unless it's going to go out there and start raising money from foundations and philanthropists and kindly billionaires, I do not see how they they get the money to keep maintaining the status quo.

• 28:14Casey Newton?

  And so in a very real sense, over the weekend, OpenAI truly may have died.

• 28:20Kevin Roose?

  It it truly may have. I mean, you're right, like, we are gonna take a bunch of money and incinerate it. And by the way, we're also gonna move very slowly and not accelerate AI progress is not a compelling pitch to investors. And so I I don't think that the sort of new OpenAI is gonna have a good time when it goes out to raise its next round of funding. Or, by the way, and this is another factor that we haven't talked about, to to close this tender offer, this fund, this round of secondary investment that was going to give OpenAI employees a chance to cash out some of their shares. That, I would say, is doomed.

• 28:52Casey Newton?

  Yeah, and I'm sure that motivated a lot of the signatures on the letter demanding that Sam and Greg come back, right? Because those people are about to get paid and not anymore.

• 29:01Kevin Roose?

  Totally. So that's some of what lies ahead for Microsoft and OpenAI, although anything could change. And that brings us to the interview that we had with Sam Altman last week. So last week, before any of this happened on Wednesday, which is two days before he was fired.

• 29:20Casey Newton?

  It was a simpler, more innocent time.

• 29:22Kevin Roose?

  That's true. I actually do feel like that was about a year and a half ago. So we sat down with Sam Altman and we asked him all kinds of questions, both about sort of the year since ChatGPT was launched and what had happened since then, and also about the future and his thoughts about where AI was headed and where OpenAI was headed. So then all this news broke and we thought, well, what do we do with this interview now? And we thought about, you know, should we even run it? Should we, you know, chop it up and just play the most relevant bits? But we ultimately decided like, we should just put the whole thing out.

• 29:54Casey Newton?

  Put it out there.

• 29:55Kevin Roose?

  Yeah, so I would just say to listeners, like, as you listen to this interview, you may be thinking like, why are these guys asking about ChatGPT? Who cares about ChatGPT? We've got bigger fish to fry here, people. But just keep in mind that when we recorded this, none of this drama had happened yet. The biggest news in Sam Altman's world was that the one-year anniversary of ChatGPT was coming up and we wanted to ask him to reflect on that. So just keep in mind, these are questions from Earth 1 and we are now on Earth 2. And just bear that in mind as you listen. But I would say that the issues that we talked about with Sam, some of the things around the acceleration of progress at OpenAI and his view of the future and his optimism

• 30:40Kevin Roose?

  about what building powerful AI could do, those are some of the key issues that seem to have motivated this coup by the board. So I think it's still very relevant, even though the specific facts on the ground have changed so much since we recorded with him. So in this interview, you'll hear us talk about existential risk, AI safety. If that's a subject you have

• 31:02Casey Newton?

  Been paying much attention to the fear here is that as these systems grow more powerful, and they are already growing exponentially more powerful year by year. At some point, they may become smarter than us. Their goals may disalign from ours. And so for folks who follow this stuff closely, there's a big debate on how seriously we should take that risk.

• 31:23Kevin Roose?

  Right, and there's also a big debate in the tech world more broadly about whether AI and technology in general should be progressing faster or whether things are already going too fast and they should be slowed down. And so when we ask him about being an accelerationist, that's what we're talking about.

• 31:41Casey Newton?

  And I should say, I texted Sam this morning to see if there was anything that he wanted to say or add, and as we record, have not heard back from him yet.

• 31:54Kevin Roose?

  When we come back, our interview from last week with Sam Alden.

• 32:16Speaker 03

  AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 32:48Speaker 04

  Ripple has been powering modern finance for a decade. It's not our first trip around the blockchain, and it won't be our last. We're leading the charge to bridge the gap between existing financial systems and the future of finance. You can have it all: legacy, stability, and digital innovation. Systems that work with your other systems. Because with the right partner, the question isn't crypto or traditional finance. The question is simply, why choose? A new standard of modern finance. It's happening with Ripple. For informational purposes only. Availability of products can vary and may be provided by different Ripple entities.

• 33:21Speaker 05

  Today's AI builders need infrastructure that scales with their ambitions. Iren is a vertically integrated AI cloud platform, delivering data centers, compute and software for AI training and inference, underpinned by its expansive portfolio of land and grid-connected power in renewable-rich regions globally. Iren's AI cloud gives direct access to Nvidia GPUs, scalable from hundreds to thousands. For builders, that means direct access to AI compute, rapid scaling and predictable pricing. Iren, AI cloud for the next big thing.

• 33:55Casey Newton?

  Sam Altman, welcome back to Hard Fork.

• 33:57Speaker 06

  Thank you.

• 33:58Casey Newton?

  Sam, it has been just about a year since ChatGPT was released, and I wonder if you have been doing some reflecting over the past year and kind of where it has brought us in the development of AI.

• 34:10Speaker 06

  Frankly, it has been such a busy year that has like not been a ton of time for reflection.

• 34:15Casey Newton?

  Well, that's why we brought you in. We want you to reflect here.

• 34:17Speaker 06

  Great, I can do it now.

• 34:20Speaker 06

  I mean, I definitely think this was the year so far, there will be maybe more in the future, but the year so far, where the general average tech person went from taking AI not that seriously to taking it pretty seriously.

• 34:33Casey Newton?

  Yeah.

• 34:34Speaker 06

  And the sort of recompiling of expectations, given that. So I think in some sense, that's like the most significant update of the year.

• 34:44Casey Newton?

  I would imagine that for you, a lot of the past year has been watching the world catch up to things that you have been thinking about for some time. Does it feel that way?

• 34:53Speaker 06

  Yeah, it does. You know, we kind of always thought on like the inside of OpenAI that it was strange that the rest of the world didn't take this more seriously. Like, it wasn't more excited about it.

• 35:04Casey Newton?

  I mean, I think if five years ago, you had you had explained like what ChatGPT was going to be, I would have thought, wow, that like, that sounds pretty cool. But and, you know, presumably, I could have just looked into it more, and I would have smartened myself up. But I think until I actually used it, as is often the case, it was just hard to know what it was.

• 35:19Speaker 06

  Yeah, I actually think we could have explained it, and it wouldn't have made that much of a difference. We tried.

• 35:26Speaker 06

  People are busy with their lives. They don't, they don't have like a lot of time to sit there and like listen to some tech people prognosticate about something that may or may not happen. But you ship a product that people use, like get real value out of, and then it's, and then it's different.

• 35:38Casey Newton?

  Yeah. I remember reading about the, the early days of the run up to the launch of ChatGPT. And I think you all have said that you did not expect it to be a hit when it launched.

• 35:49Speaker 06

  No, we thought it would be a hit. We didn't think it'd be like this. We did it because we thought it was going to be a hit. We didn't think it was going to be like this big of a hit.

• 35:54Casey Newton?

  Right. As we're sitting here today, I believe it's the case that you can't actually sign up for ChatGPT plus right now. Is that right?

• 36:00Speaker 06

  Correct.

• 36:00Casey Newton?

  Yeah. So what's that all about?

• 36:03Speaker 06

  We have like not enough capacity always, but at some point it gets really bad. So over the last 10, 10 days or so, we have done, you know, we've like done everything we can. We've rolled out new optimizations. We've like disabled some features and then people just keep signing up. It keeps getting slower and slower. And there's like a limit at some point to what you can do there and you can't, we, we just don't want to offer like a bad quality of service. And so it gets like slow enough that we just say, you know what, until we can make more progress, either with more GPUs or more optimizations, we're gonna put this on hold. Not a great place to be in, to be honest, but you know, it's like the least of several bad options.

• 36:42Casey Newton?

  Sure. And I feel like in the history of tech development, there often is a moment with really popular products where you just have to close signups for a little while, right?

• 36:50Speaker 06

  The thing that's different about this than others is it's just, it's so much more compute intensive than the world's use to for internet services. So you don't usually have to do this. Like usually by the time you're at this scale, you've like solved your scaling bottlenecks.

• 37:02Kevin Roose?

  Yeah. One of the interesting things for me about covering all the AI changes over the past year is that it often feels like journalists and researchers and companies are discovering properties of these systems sort of at the same time altogether. I mean, I remember when we had you and Kevin Scott from Microsoft on the show earlier this year around the Bing, Bing relaunch. And you both said something to the effect of, well, to discover what these models are or what they're capable of, you kind of have to put them out into the world and have millions of people using them. Then we saw, you know, all kinds Of crazy but also inspiring things you had you had Bing Sydney and but you also had people starting to use these things in their lives. So I guess I'm curious what you feel like you

• 37:45Kevin Roose?

  have learned about language models and your language model specifically from putting them out into the world.

• 37:51Speaker 06

  What we don't want to be surprised by is the capabilities of the model. That would be bad. And we were not, you know, with GPT four, for example, we took a long time between finishing the model and releasing it. Red teamed it heavily, really studied it, did all of the work internally, externally. And there's, I'd say there's at least so far, and maybe now it's been long enough that we would have, we have not been surprised by any capabilities the model had that we just didn't know about at all. In a way that we were for GPT three, frankly, sometimes the people found stuff. But what I think you can't do in the lab is understand how technology and society are going to co-evolve. So you can say, here's what the model can do and not do. But you can't say like, and here's exactly how society is going to progress, given that. And that's where you just

• 38:36Speaker 06

  have to see what people are doing, how they're using it. And that, like, well, one thing is they use it a lot. Like, that's, that's one takeaway that we did not, clearly, we did not appropriately plan for. But more interesting than that is the way in which this is transforming people's productivity, personal lives, how they're learning, and how, like, you know, one example that I think is instructive, because it was the first and the loudest, is what happened with ChatGPT in education.

• 39:16Speaker 06

  Days, at least weeks, but I think days after the release of ChatGPT, school districts were like falling all over themselves to ban ChatGPT. And that didn't really surprise us. Like, that we could have predicted and did predict. The thing that happened after that quickly was

• 39:40Speaker 06

  You know, like weeks to months was school districts and teachers saying, hey, actually we made a mistake. And this is really important part of the future of education and the benefits far with the downside. And not only are we unbanning it, we're encouraging our teachers to make use of it in the classroom. We're encouraging our students to get really good at this tool because it's going to be part of the way people live. And, you know, then there was like a big discussion about what the kind of path forward should be. And that is just not something that could have happened without releasing.

• 40:12Kevin Roose?

  Yeah.

• 40:12Speaker 06

  And part, can I say one more thing?

• 40:15Kevin Roose?

  Yeah.

• 40:15Speaker 06

  Part of the decision that we made with the chat GPT release, the original plan had been to do the chat interface and GPT four together in March. And we really believe in this idea of iterative deployment. And we had realized that chat, the chat interface plus GPT was a lot. I don't think we realized quite how much it was.

• 40:36Kevin Roose?

  Like too much for society to take in.

• 40:38Speaker 06

  So we split it and we put out, we put it out with GPT three point five first, which we thought was a much weaker model. It turned out to still be powerful enough for a lot of use cases. But I think that in retrospect was a really good decision and helped with that process of gradual adaptation for society.

• 40:57Kevin Roose?

  Looking back, do you wish that you had done more to sort of, I don't know, give people some sort of a manual to say, here's how you can use this at school or at work?

• 41:04Speaker 06

  Two, two things. One, I wish we had done something intermediate between the release of three point five and the API and chat GPT. Now, I don't know how well that would have worked because I think there was just going to be some moment where it went like viral in the mind of society. And I don't know how incremental that could have been. That's sort of a like, either it goes like this or it doesn't kind of thing. And I think I have reflected on this question a lot. I think the world was going to have to have that moment. It was better sooner than later. It was good we did it when we did. Maybe we should have tried to push it even a little earlier, but it's a little chancy about when it hits.

• 41:46Speaker 06

  And I think only a consumer product could have done what happened there. Now, the second thing is, should we have released more of a how to manual? And

• 41:59Speaker 06

  I honestly don't know. I think we could have done some things there that would have been helpful, but I really believe that it's not optimal for tech companies to tell people like, here is how to use this technology and here's how to do whatever. And the organic thing that happened there actually was pretty good.

• 42:18Kevin Roose?

  Yeah. I'm curious about the thing that you just said about, we thought it was important to get this stuff into folks' hands sooner rather than later. Say more about what that is.

• 42:27Speaker 06

  More time to adapt for our institutions and leaders to understand, for people to think about what the next version of the model should do, what they'd like, what would be useful, what would not be useful, what would be really bad, how society and the economy need to co-evolve. Like, the thing that many people in the field or adjacent to the field have advocated, or used to advocate for, which I always thought was super bad, was, you know, this is so disruptive, such a big deal, it's got to be done in secret by the small group of us that can understand it, and then we will fully build the AGI and push a button all at once when it's ready. And I think that'd be quite, quite, quite bad.

• 43:08Kevin Roose?

  Yeah, because it would just be way too much change too fast.

• 43:10Speaker 06

  Yeah, again, society and technology have to co-evolve and people have to decide what's going to work for them and not, and how they want to use it. And we're, you know, you can criticize OpenAI about many, many things, but we do try to like, really listen to people and adapt it in ways that make it better or more useful, and we're able to do that, but we wouldn't get it right without that feedback.

• 43:27Kevin Roose?

  Yeah. I want to talk about AGI and the path to AGI later on, but first I want to just define AGI and have you talk about sort of where we are on the continuum. So.

• 43:39Speaker 06

  I think it's a ridiculous and meaningless term.

• 43:42Kevin Roose?

  Yeah.

• 43:42Speaker 06

  So I apologize that I keep using it.

• 43:46Kevin Roose?

  I mean, I just never know what people are talking about.

• 43:48Speaker 06

  No one else does either. They mean like really smart AI.

• 43:51Kevin Roose?

  Yeah, so it stands for artificial general general intelligence. And you could probably ask 100 different AI researchers and they would give you 100 different definitions of what AGI is. Researchers at Google DeepMind just released a paper this month that sort of offers a framework. They have five levels, level, or I guess they have levels ranging from level zero, which is no AI, all the way up to level five, which is superhuman. And they suggest that currently ChatGPT, Bard, Lama 2 are all at level one, which is sort of equal to or slightly better than an unskilled human. Would you agree with that? Like, where are we? If you would, if you'd say this is a term that means something and you sort of define it that way, how close are we?

• 44:33Speaker 06

  Um.

• 44:36Speaker 06

  I think the thing that matters is the curve and the rate of progress. And there's not going to be some milestone that we all agree like, okay, we've passed it and now it's called AGI. Like, what I would say is we currently have systems that are like, there will be researchers who will write papers like that. And, you know, academics will debate it and people in industry will debate it. And I think most of the world just cares like, is this thing useful to me or not? And we currently have systems that are somewhat useful, clearly. Like, and, you know, whether we want to say like, it's a level one or two, I don't know. But people use it a lot and they really love it. There's huge weaknesses in the current systems.

• 45:16Speaker 06

  But it doesn't mean that.

• 45:20Speaker 06

  Like, I'm, you know, a little embarrassed by GPTs, but people still like them. And that that's good. Like, it's nice to do useful stuff for people. So, yeah, call it a level one. Doesn't bother me at all. I am embarrassed by it. We will make them much better. But at their current state, they're still like delighting people and being useful to people.

• 45:39Kevin Roose?

  Yeah, I also think it underrates them slightly to say that they're just better than unskilled humans. Like, when I use ChatGPT, it is better than skilled humans.

• 45:46Speaker 06

  And some things and worse than unskilled, that worse than any human and many other things.

• 45:50Kevin Roose?

  But I guess this is one of the questions that people ask me the most, and I imagine ask you is like, what are today's AI systems useful and not useful for doing?

• 46:01Speaker 06

  I would say the main thing that they're bad at, well, many things, but one that is on my mind a lot is they're bad at reasoning. And a lot of the valuable human things require some degree of complex reasoning. But they're.

• 46:20Speaker 06

  They're good at a lot of other things like, like, you know, GPT-4 is vastly superhuman in terms of its world knowledge. It knows there's a lot of things in there. And it's just, it's like very different than how we think about evaluating human intelligence. So it can't do these basic reasoning tasks. On the other hand, it knows more than any human has ever known. On the other hand, again, sometimes it like totally makes stuff up in a way that a human would not. But, you know, if you're using it to be a coder, for example, it can hugely increase your productivity. And there's value there, even though it has all of these other weak points. If you were a student, you can learn a lot more than you could without using this tool in some ways. Value there, too.

• 47:04Casey Newton?

  Let's talk about GPTs, which you announced at your recent developer conference. For those who haven't had a chance to use one yet, Sam, what's a GPT?

• 47:11Speaker 06

  It's like a custom version of ChatGPT that you can get to behave in a certain way. You can give it limited ability to do actions. You can give it knowledge to refer to. You can say like, act this way. But it's super easy to make, and it's a first step towards more powerful AI systems and agents.

• 47:29Casey Newton?

  We've had some fun with them on the show. There is, there's a hard fork bot that you can sort of ask about anything that's happened on any episode of the show. It works pretty well, we found when we did some testing. But I want to talk about where, where this is going. What is, what is the GPTs that you've released a first step toward?

• 47:43Speaker 06

  Um, AIs that can accomplish useful tasks. Like the, the, I think we need to move towards this with great care. You know, we don't, I think it would be a bad idea to put like, turn powerful agents free on the internet. But AIs that can act on your behalf to do something with a company that can access your data, that can like help you be good at a task. I think that's, that's going to be an exciting way we use computers. Like, we have this belief that we're heading towards a vision where there are New interfaces, new user experiences possible because finally the computer can

• 48:28Speaker 06

  understand you and think. And so the sci fi vision of a computer that you just like tell what you want and it figures out how to do it. This is a step towards that.

• 48:38Kevin Roose?

  Right now, I think what's holding a lot of people back in, a lot of companies and organizations back in sort of using this kind of AI in their work is that it can be unreliable. It can make up things. It can give wrong answers, which is fine if you're doing creative writing assignments, but not if you're a hospital or a law firm or something else with big stakes.

• 49:00Kevin Roose?

  How, how do we solve this problem of reliability? And do you think we'll ever get to the sort of low fault tolerance that is needed for these really high stakes applications?

• 49:12Speaker 06

  So first of all, I think this is like a great example of people understanding the technology, making smart decisions with it, society and the technology co-evolving together. Like what, what you see is that people are using it where appropriate and where it's helpful and not using it where you shouldn't. And for all of the sort of like fear that people have had, like both users and companies seem to really understand the limitations and are making appropriate decisions about where to roll it out. It,

• 49:46Speaker 06

  the kind of controllability, reliability, whatever you want to call it, that is going to get much better. I think we'll see a big step forward there over the coming years. And,

• 50:00Speaker 06

  and I think that there will be a time.

• 50:04Speaker 06

  I don't know if it's like 2026, 2028, 2030, whatever, but there will be a time where we just don't talk about this anymore.

• 50:09Kevin Roose?

  Yeah. It seems to me though, that that is something that becomes very important to get right in the, as you build these more powerful GPTs, right? Once I tell, like, I would love to have a GPT be my assistant, go through my emails. Hey, don't forget to respond to this before the end of the day.

• 50:25Speaker 06

  The reliability has got to be way up before that happens.

• 50:27Kevin Roose?

  Yeah, yeah, that makes sense. Um, you, you mentioned as we started to talk about GPTs that you have to do this carefully.

• 50:37Casey Newton?

  For folks who haven't spent as much time reading about this, explain what are some things that could go. You know, you guys are obviously going to be very careful with this. Other people are going to build GPT like things, might not put the same kind of controls in place. So what can you imagine other people's doing that like you as the CEO would say your folks, hey, it's not going to be able to do that?

• 50:54Speaker 06

  Well, that example that you just gave, like if you let it act as your assistant and go like, you know, send emails, do finance, financial transfers for you. Like, it's very easy to imagine how that could go wrong. But I think most people who would use this don't want that to happen on their behalf either. And so there's more resilience to this sort of stuff than than people think.

• 51:16Casey Newton?

  I think that's, I mean, for what it's worth, on the whole, on the hallucination thing, which it does feel like has maybe been the longest conversation that we've had about ChatGPT in general since it launched. I just always think about Wikipedia as a resource I use all the time. And I don't want Wikipedia to be wrong, but 100% of the time, it doesn't matter if it does. I am not relying on it for life-saving information, right? ChatGPT for me is the same, right? It's like, hey, hey, you know, it's, I mean, it's like great and just kind of bar trivia. Like, hey, you know, what's like the history of this conflict in the world?

• 51:44Speaker 06

  Yeah, I mean, we want to get that a lot better and we we will. Like, I think the next model will just hallucinate much less.

• 51:51Kevin Roose?

  Is there, is there an optimal level of hallucination in an AI model? Because I've heard researchers say, well, you actually don't want it to never hallucinate because that would mean making it not creative. That new ideas come from making stuff up that's not necessarily tethered to.

• 52:07Speaker 06

  This is why I tend to use the word controllability and not reliability. You want it to be reliable when you want, you want it to, either you instruct it or it just knows based off of the context that you're asking a factual query and you want the 100% black and white answer. But you also want it to know when you want it to hallucinate or you want it to make stuff up. As you just said, like, new discovery happens because you come up with new ideas, most of which are wrong, and you discard those and keep the good ones and sort of add those to your understanding of reality. Or if you're telling a creative story, you want that. So, so if these models, like.

• 52:45Speaker 06

  If these models didn't hallucinate at all ever, they wouldn't be so exciting. They wouldn't do a lot of the things that they can do. But you only want them to do that when you want them to do that. And so, so like the way I think about it is like model capability, personalization and controllability. And those are like the three axes we have to push on. And controllability means no hallucinations when you don't want lots of it when you're trying to invent something new.

• 53:09Casey Newton?

  Let's maybe start moving into some of the debates that we've been having about AI over the past year. I actually want to start with something that I haven't heard as much, but that I do bump into when I use your products, which is like, they can be quite restrictive in how you use them. I think mostly for, for great reasons, right? Like, I think you guys have learned a lot of lessons from the, the past era of tech development. At the same time, I feel like, like I've tried to ask ChatGPT a question about sexual health. I feel like it's going to call the police on me, right? So I'm just curious how you've approached that subject.

• 53:39Speaker 06

  Yeah, look, one thing, no one wants to be scolded by a computer ever. Like, that is not a good feeling. And so you should never feel like you're gonna have the police called or it's more like horrible, horrible, horrible. We have started very conservative, which I think is a defensible choice. Other people may have made a different one. But again, that principle of controllability, what we'd like to get to is a world where if you want some of the guardrails relaxed a lot, and that's a like, you know, you're not like a child or something, then fine, we'll relax the guardrails. It should be up to you. But I think starting super conservative here, although annoying, is a defensible decision,

• 54:23Speaker 06

  and I wouldn't have gone back and made it differently. We have relaxed it already. We will relax it much more, but we want to do it in a way where it's user controlled.

• 54:31Casey Newton?

  Yeah. Are there certain red lines you won't cross, things that you will never let your models be used for, other than things that are like obviously illegal or dangerous?

• 54:42Speaker 06

  Yeah, certainly things that are illegal and dangerous, we won't. There's like a lot of other things that I could say, but they so depend like where those red lines will be so depend on how the technology evolves that it's hard to say right now, like here's the exhaustive set. Like, we really try to just study the models and predict capabilities as we go, but we get, you know, if we learn something new, we change our plans.

• 55:06Kevin Roose?

  Yeah. One other area where things have been shifting a lot over the past year is in AI regulation and governance. I think a year ago, if you'd asked, you know, the average congressperson, what do you think of AI? They would have said, what's that?

• 55:18Casey Newton?

  Get out of my office.

• 55:19Kevin Roose?

  Right. You know, we just recently saw the Biden White House put out an executive order about AI. You have obviously been meeting a lot with lawmakers and regulators, not just in the US, but around the world. What's your view of how AI regulation is shaping up?

• 55:34Speaker 06

  It's a really tricky point to get across. What we believe is that on the frontier systems, there does need to be proactive regulation there, but heading into overreach and regulatory capture would be really bad. And there's a lot of amazing work that's going to happen with smaller models, smaller companies, open source efforts, and it's really important that regulation not strangle that. So it's like, I've sort of become a villain for this, but I think there was.

• 56:02Casey Newton?

  You have?

• 56:02Speaker 06

  Yeah.

• 56:03Casey Newton?

  How do you feel about this?

• 56:04Speaker 06

  Like annoyed, but have bigger problems in my life right now. But this message of like, regulate us, regulate the really capable models that can have significant consequences, but leave the rest of the industry alone, it's just, it's a hard message to get across.

• 56:24Casey Newton?

  Sure. Here is an argument that was made to me by a high ranking executive at a major tech company as some of this debate was playing out. This person said to me that there is essentially no harms that these models can have that the internet itself doesn't enable, right? And that to do any sort of work like is proposed in this executive order to have to inform the Biden administration is just essentially pulling up the ladder behind you and ensuring that the folks who've already raised the money can, you know, sort of reap all of the profits of this new world and will leave the little people behind. So I'm curious what you make of that argument.

• 57:03Speaker 06

  Uh, I disagree with it on a bunch of levels. First of all, uh, I, I, I wish the threshold for when you do have to report was, uh, set differently and based off of like, uh, you know, evals and capability thresholds.

• 57:18Kevin Roose?

  Not flops?

• 57:18Speaker 06

  Not flops.

• 57:19Kevin Roose?

  Okay.

• 57:19Speaker 06

  But there's no small company trained with that many flops anyway. So that's like a little bit.

• 57:23Kevin Roose?

  Yeah.

• 57:24Speaker 06

  You know?

• 57:24Kevin Roose?

  For the listener who maybe didn't listen to our last episode about this. Listen to our flops episode. The flops are the, the sort of measure of the amount of computing that is used to train these models. The executive order says if you're above a certain computing threshold, you have to tell the government that you're training a model that big.

• 57:38Speaker 06

  Yep. Um, but no small effort is training at 10 to the 26 flops. Currently, no big effort is either. Um, so that's like a dishonest comment. Um, second of all, the burden of just saying like, here's what we're doing is not that great, but, but.

• 57:57Speaker 06

  Third of all, the, the underlying thing there, there's nothing you can do here that you couldn't already do on the internet. Um, that's the real either dishonesty or lack of understanding. Um, I, you could maybe say with GPT four, you can't do anything. You can't do on the internet, but I don't think that's really true even at GPT four. Like you, there are some new things and GPT five and six, there will be like very new things. And, um, saying that we're going to be like cautious and responsible and have some testing around that, uh, I think that's going to look more prudent in retrospect than it maybe sounds right now.

• 58:35Kevin Roose?

  I'd say for me, these seem like the absolute gentlest regulations you could imagine. It's like tell the government and report on any safety testing you did.

• 58:41Speaker 06

  Seems reasonable.

• 58:42Kevin Roose?

  Yeah. And people are not just, uh, saying that this is, these, these fears are unjustified of AI and sort of existential risk. Um, some people, some of the more vocal critics of OpenAI have said that, um, that, that OpenAI, that, that you are, are specifically lying about the risks of human extinction from AI, creating fear, um, so that regulators will come in and make laws or give executive orders that prevent smaller competitors from, uh, being able to compete with you. Andrew Ng, who's I think one of your professors at Stanford recently said something to this effect. Um, what's your response to, to that? I'm curious if you have thoughts about that.

• 59:20Speaker 06

  Yeah. Like I, I actually.

• 59:24Speaker 06

  Don't think we're all gonna go extinct. I think it's gonna be great. I think we're like heading towards the best world ever. But when we deal with a dangerous technology as a society, we often say that we have to confront and successfully navigate the risks to get to enjoy the benefits. And that's like a pretty consensus thing. I don't, I don't think that's like a radical position. You, you can, I can imagine that if this technology stays on the same curve, there are systems that are capable of significant harm in the future. And, you

• 1:00:04Speaker 06

  know, like Andrew also said not that long ago that he thought it was like totally irresponsible to talk about AGI because it was just never happening.

• 1:00:12Casey Newton?

  I think he compared it to worrying about overpopulation on Mars.

• 1:00:15Speaker 06

  And I think now he might say something different. So like, it's,

• 1:00:23Speaker 06

  humans are very bad at having intuition for exponentials. Again, I think it's gonna be great. Like, I wouldn't work on this if I didn't think it was gonna be great. People love it already. And I think they're gonna love it a lot more. But that doesn't mean

• 1:00:39Speaker 06

  we don't need to be responsible and accountable and thoughtful about what the downsides could be. And in fact, I think the tech industry often has only talked about the good and not the bad. And that doesn't go well either.

• 1:00:52Casey Newton?

  The exponential thing is real. I have dealt with this. You know, I've talked about the fact that I was only using GPT 3.5 until a few months ago. And finally, at the urging of a friend, upgraded. And I thought, oh.

• 1:01:04Speaker 06

  I would have given you a free account. Sorry.

• 1:01:05Casey Newton?

  I wish I shouldn't have asked.

• 1:01:06Speaker 06

  But it's a, it's a real improvement.

• 1:01:08Casey Newton?

  It is a real improvement. And not just in the sense of, oh, the copy that it generates is better. It actually transformed my sense of how quickly the industry was moving. It made me think, oh, like, the next generation of this is gonna be sort of radically better. And so I think that part of what we're dealing with is just that it has not been widely distributed enough to get people to reckon with the implications.

• 1:01:31Speaker 06

  Um, I disagree with that. I mean, I think that like, you know, maybe the tech experts say like, oh, this is like, you know, not a big deal, whatever. Like, most of the world is like, who has used even the free version is like, oh, man, they got real AI.

• 1:01:46Casey Newton?

  Yeah. Yeah. And you, you went around the world this year, talking to people in a lot of different countries. I'd be curious what, you know, to what extent that informed what you just said.

• 1:01:55Speaker 06

  Significantly. I mean, I was, I had a little bit of a sample bias, right? Because the people that wanted to meet me were probably like, pretty excited. But you do get a sense. And there's like, quite a lot of excitement, maybe more excitement in the rest of the world than the US.

• 1:02:07Kevin Roose?

  Sam, I want to ask you about something else that people are, are not happy about when it comes to these language and image models, which is this issue of copyright. I think a lot of people view what OpenAI and other companies did, which is sort of, you know, hoovering up work from across the internet, using it to train these models that can, in some cases, output things that are similar to the work of living authors or writers or artists. And they just think like, this is the original sin of the AI industry, and we are never going to forgive them for doing this. What do you think about that? And what would you say to artists or writers who just think that this was a moral lapse? Forget about the legal, whether you're allowed to do it or not, that it

• 1:02:52Kevin Roose?

  was just unethical for you and other companies to do that in the first place?

• 1:02:56Speaker 06

  Well, we block that stuff. Like, you can't go to like, Dolly and generate some, I mean, you get, speaking of being annoyed, like, we may be too aggressive on that. But I think, I think it's the right thing to do until we figure out some sort of economic model that works for people. And, you know, we're doing some things there now, but we've got more to do. Other people in the industry like, do allow quite a lot of that. And I get why artists are annoyed.

• 1:03:20Kevin Roose?

  I guess I'm talking less about the output question than just the, the act of taking all of this work, much of it copyrighted, without the explicit permission of the people who created it and using it to train these models. Do you think the people, what would you say to the people who just say that, Sam, that was, that was the wrong move, you should have asked, and we will never forgive you for it?

• 1:03:41Speaker 06

  Um, well, first of all, like, always have empathy for people who are like, hey, you did this thing and it's affecting me. And, you know, we didn't talk about it first, or it was just like a new, a new thing. Like, the, um, I do think that

• 1:03:59Speaker 06

  In the same way, humans can read the Internet and learn, AI should be allowed to read the Internet and learn. Shouldn't be regurgitating, shouldn't be violating any copyright laws. But if we're really going to say that AI doesn't get to read the Internet and learn, and if you read a physics textbook and learn how to do a physics calculation, not every time you do that in the rest of your life, you've got to figure out how to like...

• 1:04:28Speaker 06

  That seems like not a good solution to me. But on individuals' private work,

• 1:04:36Speaker 06

  yeah, we try not to train on that stuff. We really don't want to be here upsetting people. Again, I think other people in the industry have taken different approaches. And we've also done some things that I think now that we understand more, we'll do differently in the future.

• 1:04:55Casey Newton?

  Like what?

• 1:04:56Speaker 06

  Like what we do differently? We want to figure out new economic models so that, say, if you're an artist, we don't just totally block you. We don't just not train on your data, which a lot of artists also say, no, I want this in here. I want like whatever. But we have a way to help share revenue with you. GPTs are maybe going to be an interesting first example of this because people will be able to put private data in there and say, hey, use this version and there could be a revenue share around it.

• 1:05:22Casey Newton?

  I feel like that might be a good place to take a break and then come back and talk about the future.

• 1:05:26Speaker 06

  Yes, let's take a break.

• 1:05:48Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner. Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 1:06:22Speaker 03

  AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 1:06:54Speaker 04

  Ripple has been powering modern finance for a decade. It's not our first trip around the blockchain, and it won't be our last. We're leading the charge to bridge the gap between existing financial systems and the future of finance. You can have it all: legacy, stability, and digital innovation. Systems that work with your other systems. Because with the right partner, the question isn't crypto or traditional finance. The question is simply, why choose? A new standard of modern finance. It's happening with Ripple. For informational purposes only. Availability of products can vary. May be provided by different Ripple entities.

• 1:07:29Casey Newton?

  Well, I had one question about the future that kind of came out of what we were talking about before the break, which is, and it's so big, but I truly need to hear your thoughts on this, which is what is the future of the internet as ChatGPT rises? And the reason I ask is I now have a hotkey on my computer that I type when I want to know something, and it just accesses ChatGPT directly through software called Raycast. And because of this, I am using Google Search not nearly as much. I am visiting websites not nearly as much. That has implications for all the publishers and for frankly, just the model itself, because presumably if the economics change, there'll be fewer webpages created, there's less data for ChatGPT to access. So I'm just curious what you have thought about the internet in a world where your product succeeds in the way you want it to?

• 1:08:19Speaker 06

  I do think if this all works, it should really change how we use the Internet. There's a lot of things that the interface for is like, perfect. Like, if you want to like, mindlessly watch TikTok videos, perfect. But if you're trying to like, get information or get a task accomplished, it's actually like, quite bad relative to what we should all aspire for. And you can totally imagine a world where you have a task that right now takes like, hours of stuff clicking around the Internet and bringing stuff together. And you just ask ChatGPT to do one thing, and it goes off and computes, and you get the answer back.

• 1:08:59Speaker 06

  And I'll be disappointed if we don't use the Internet differently.

• 1:09:06Casey Newton?

  Yeah. Do you think that the economics of the Internet as it is today are robust enough to withstand the challenge that AI poses?

• 1:09:15Speaker 06

  Probably.

• 1:09:16Casey Newton?

  Okay. Well, I worry in particular about the publishers. The publishers have been having a hard time already for, you know, a million other reasons. But to the extent that they're driven by advertising and visits to webpages, and to the extent that the visits to the webpages are driven by Google's search in particular, a world where web search is just no longer the front page to most of the Internet, I think does require a different kind of web economics.

• 1:09:38Speaker 06

  I think it does require a shift, but I think the value is... So what I thought you were asking about was like, is there going to be enough value there for some economic model to work? And I think that's definitely going to be the case. Yeah, the model may have to shift. I would love it if ads become less a part of the Internet. Like, I was thinking the other day, like, I just had this, like, for whatever reason, like, this thought in my head as I was like, browsing around the Internet, being like, there's more ads than content everywhere.

• 1:10:05Casey Newton?

  I was reading a story today, scrolling on my phone, and I managed to get it to a point where between all of the ads on my relatively large phone screen, there was one line of text from the article visible.

• 1:10:15Speaker 06

  You know, one of the reasons I think people like... ChatGPT, even if they can't articulate it, is we don't do ads.

• 1:10:22Casey Newton?

  Yeah.

• 1:10:22Speaker 06

  Like, as a, as a, as a intentional choice, because there's plenty of ways you could imagine us putting ads.

• 1:10:27Casey Newton?

  Totally.

• 1:10:28Speaker 06

  But we made the choice that ads plus AI can get a little dystopic. We're not saying never, like, we, we do want to offer a free service, but like, a big part of our mission fulfillment, I think, is if we can continue to offer ChatGPT for free at a high quality of service to anybody who wants it, and just say like, hey, here's free AI, and good free AI, and no ads. Because I think that really does, especially as the AI like gets really smart, that really does get a little strange.

• 1:10:59Casey Newton?

  Yeah, yeah.

• 1:11:00Kevin Roose?

  I know, we talked about AGI and it not being your favorite term, but it is a term that people in the industry use as sort of a benchmark or a milestone or something that they're aiming for. And I'm curious what you think the barriers between here and AGR, AGI are. Maybe let's define AGI as sort of a computer that can do any cognitive task that a human can.

• 1:11:24Speaker 06

  If it, like, let's say we make an AI that is really good, but it can't go discover novel physics. Would you call that an AGI?

• 1:11:30Kevin Roose?

  I probably would, yeah.

• 1:11:31Speaker 06

  You would, okay.

• 1:11:32Kevin Roose?

  Would you?

• 1:11:34Speaker 06

  Well, again, I don't like the term, but I wouldn't call that we're done with the mission. I'd say we still got a lot more work to do.

• 1:11:39Casey Newton?

  The vision is to create something that is better at humans than doing original science that can invent, can discover.

• 1:11:46Speaker 06

  Well, I am a believer that all real, sustainable human progress comes from scientific and technological progress. And if we can have a lot more of that, I think it's great. And if a system can do things that we, unaided on our own, can't, just even as a tool that helps us go do that, then I will consider that a massive triumph and happily, you know, I can happily retire at that point. But before that, I can imagine that we do something that creates incredible economic value, but is not the kind of AGI, superintelligence, whatever you want to call it, thing that we should aspire to.

• 1:12:20Casey Newton?

  Right. What are some of the barriers to getting to that place where we're doing novel physics research?

• 1:12:27Speaker 06

  Um.

• 1:12:29Casey Newton?

  And keep in mind, Kevin, I don't know anything about technology and.

• 1:12:32Speaker 06

  That seems unlikely to be true.

• 1:12:34Casey Newton?

  Well, if you if you start talking about, you know, like retrieval augmented generation or anything like I might, you might.

• 1:12:40Speaker 06

  I'll follow, but you'll lose Casey.

• 1:12:42Casey Newton?

  He'll follow, yeah.

• 1:12:43Speaker 06

  Um, we talked earlier about just the model's limited ability to reason. And I think that's one thing that needs to be better. The model needs to be better at reasoning.

• 1:12:55Speaker 06

  Like GPT four.

• 1:12:58Speaker 06

  An example of this that my co-founder Ilya uses sometimes is really stuck in my mind is there was like a time in Newton's life where the right thing for him to do.

• 1:13:07Casey Newton?

  You're talking, of course, about Isaac Newton, not my life.

• 1:13:09Speaker 06

  Isaac Newton, okay. Well, maybe you do.

• 1:13:11Casey Newton?

  But maybe my life, we'll find out, stay tuned.

• 1:13:14Speaker 06

  Where the right thing for him to do is to read every math textbook he could get his hands on. He should like talk to every smart professor, talk to his peers, you know, do problem sets, whatever. And that's kind of what our models do today. And at some point, there was, he was never going to invent calculus doing that, what didn't exist in any textbook. And at some point, he had to go think of new ideas and then test them out and build them, whatever else. And that phase, that second phase, we don't do yet. And I think you need that before. It's something I want to call an AGI.

• 1:13:47Kevin Roose?

  Yeah. One thing that I hear from AI researchers is that a lot of the progress that has been made over the past, call it five years in this type of AI has been just the result of just things getting bigger, right? Bigger models, more compute. Obviously, there's there's work around the edges in how you build these things that makes them more useful, but there hasn't really been a shift on the architectural level, of the systems that these models are built on. Do you think that that is going to remain true? Or do you think that we need to invent some new process or new, you know, new mode or new technique to get through some of these barriers?

• 1:14:30Speaker 06

  We will need new research ideas and we have needed them. I don't think it's fair to say there haven't been any here.

• 1:14:37Speaker 06

  I think a lot of the people who say that are not the people building GPT four, but they're the people sort of opining from the sidelines. But but there is some kernel of truth to it.

• 1:14:48Speaker 06

  And.

• 1:14:50Speaker 06

  The answer is we have, OpenAI has a philosophy of we will just do whatever works. Like if it's time to scale the models and work on the engineering challenges, we'll go do that. If now we need a new algorithm or breakthrough, we'll go work on that. If now we need a different kind of data mix, we'll go work on that. So like, we just do the thing in front of us and then the next one, and then the next one, and then the next one. And there are a lot of other people who want to write papers about, you know, level one, two, three, and whatever. And there are a lot of other people who want to say, well, it's not real progress. They just made this like incredible thing that people are using and loving and it's not real sign like. But our, our belief is like, we will just do whatever we can to usefully drive the progress forward.

• 1:15:33Speaker 06

  And we're kind of open minded about how we do that.

• 1:15:37Kevin Roose?

  What is super alignment? You all just recently announced that you are devoting a lot of resources and, and time and computing power to super alignment, and I don't know what it is. So can you help me understand?

• 1:15:49Casey Newton?

  It's alignment that comes with sour cream and guacamole.

• 1:15:52Kevin Roose?

  There you go.

• 1:15:53Casey Newton?

  San Francisco taco show. That's a very San Francisco specific joke, but it's pretty good. I'm sorry, go ahead, Sam.

• 1:16:00Speaker 06

  I don't, can I leave it at that? I don't really want to, I mean, that was such a good answer. No, um, it's, uh, so alignment is how you sort of get these models to behave in accordance with the human who's using them, uh, what they want. And super alignment is how you do that for super capable systems. So we know how to align GPT four pretty well, but, uh, like better than people thought we were going to be able to do. Now there's this like, when we put out GPT two and three, people are like, oh, it's irresponsible research because this is always going to just like spew toxic shit. You're never going to get it. And it actually turns out like, we're able to align GPT four reasonably well, maybe too well.

• 1:16:39Casey Newton?

  Yeah, it's, I mean, good luck getting it to talk about sex is my official comment about GPT four.

• 1:16:45Speaker 06

  But that's, you know, in some sense, that's an alignment failure because that's, it's not doing what you wanted there.

• 1:16:49Casey Newton?

  Yeah.

• 1:16:49Speaker 06

  So, but now we have that, now we have like the social part of the problem. We can technically do it.

• 1:16:53Casey Newton?

  Right.

• 1:16:54Speaker 06

  Um, but we don't yet know. What the new challenges will be for much more capable systems. And so that's what that team research is.

• 1:17:01Kevin Roose?

  So like, what kinds of questions are they investigating? Or what research are they doing? Because I, you know, I confess, I sort of, I lose my grounding in reality, when you start talking about super capable systems and the problems that can emerge with them. Is this, is this sort of a theoretical future forecasting team?

• 1:17:20Speaker 06

  Well, they try to do work that is useful today, but for the theoretical systems of the future. So they'll have their first result coming out, I think, pretty soon. But, but yeah, they're interested in these questions of as the systems get more capable than humans, what is it going to take to reliably solve the alignment challenge?

• 1:17:45Casey Newton?

  Yeah. And I mean, this is the stuff where my brain does feel like it starts to melt as I ponder the implications, right? Because you've made something that is smarter than every human, but you, the human, have to be smart enough to ensure that it always acts in your interests, even though by definition is way smarter than you.

• 1:17:59Speaker 06

  Yeah, we need some help there.

• 1:18:00Kevin Roose?

  Yeah. I, I do want to stick on this issue of, of alignment or super alignment, because I think there's, there's an unspoken assumption in there that, well, you just, you just put it as alignment is sort of what, what the user wants it to behave like. And obviously, there are a lot of users with good intentions.

• 1:18:18Speaker 06

  No, no. Yeah, it has to be like what society and the user can intersect on. There are going to have to be some rules here.

• 1:18:27Kevin Roose?

  And I guess where do you derive those rules? Because, you know, if you're anthropic, you use, you know, the UN Declaration of Human Rights and the Apple Terms of Service. And that becomes your-

• 1:18:38Casey Newton?

  The two most important documents in rights governance.

• 1:18:43Kevin Roose?

  If you're not just going to borrow someone else's rules, how do you decide which values these things should align themselves to?

• 1:18:49Speaker 06

  So we're doing this thing, we've been doing this thing, we've been doing these like democratic input governance grants, where we're giving different research teams money to go off and set different proposals. There's some very interesting ideas in there about how to, how to kind of fairly decide that.

• 1:19:06Speaker 06

  The naive approach to this that I have always been interested in, maybe we'll try at some point, is what if you had hundreds of millions of ChatGPT users spend an hour, a few hours a year, answering questions about what they thought the default setting should be, what the wide bounds should be. Eventually, you need more than ChatGPT users. You need the whole world represented in some way because even if you're not using it, you're still impacted by it. But to start, what if you literally just had ChatGPT chat with its users? It can, I think it's very important, it would be very important in this case to let the users make final decisions, of course, but you could imagine it saying like, hey, you answered this question this way. Here's how this would impact other users in a way you might not have thought of. If you want to stick with your answer, that's totally

• 1:19:51Speaker 06

  up to you. But are you sure, given this new data? And then you could imagine like GPT-5 or whatever, just learning that collective preference set. And I think that's interesting to consider.

• 1:20:04Casey Newton?

  Yeah.

• 1:20:06Speaker 06

  I want to ask-

• 1:20:06Casey Newton?

  Better than the Apple Terms of Service, let's say. I want to ask you about this feeling. Kevin and I call it AI vertigo. I don't, is this a widespread term that people use?

• 1:20:15Speaker 06

  No, I think you invented this.

• 1:20:16Casey Newton?

  It's just sort of us. So there is this moment when you contemplate even just kind of the medium AI future, you start to think about what it might mean for the job market, your own job, your daily life, for society. And there is this kind of dizziness that I find sets in. This year, I actually had a nightmare about AGI. And then I sort of asked around, and I feel that people who work on this stuff, like, that's not uncommon. I wonder for you, if you have had these moments of AI vertigo, if you continue to have them, or is there at some point where you think about it long enough that you feel like you get your legs underneath you?

• 1:20:51Speaker 06

  I thought, I think you, I used to have, I mean, there were some, you can like, point to these moments, but there were some like, very strange, like, extreme vertigo moments, particularly around the launch of GPT-3.

• 1:21:04Casey Newton?

  Hmm.

• 1:21:06Speaker 06

  But you do get your legs under you.

• 1:21:07Casey Newton?

  Yeah.

• 1:21:08Speaker 06

  And I think the future will somehow be less different than we think. Like, it's this amazing thing to say, right? Like, we invent AGI and it matters less than we think. It doesn't sound like a sentence that parses. And yet, it's what I expect to happen.

• 1:21:20Casey Newton?

  Why is that?

• 1:21:25Speaker 06

  There's like a lot of inertia in society and humans are remarkably adaptable to any amount of change.

• 1:21:31Kevin Roose?

  One question I get a lot that I imagine you do too is is from people who want to know what they can do. You you mentioned adaptation as being necessary on the societal level. I think for many years, the conventional wisdom was that if you wanted to adapt to a changing world, you should learn how to code, right? That was like the classic advice.

• 1:21:50Speaker 06

  May not be such good advice anymore.

• 1:21:51Kevin Roose?

  Exactly. So now, you know, AI systems can can code pretty well. For a long time, the conventional wisdom was that creative work was sort of untouchable by machines. If you were a factory worker, you might get automated out of your job. But if you were an artist or a writer, that was impossible for computers to do. Now we see that's no longer safe. So where is the sort of high ground here? Like, where can people focus their energy if they want skills and abilities that AI is not going to be able to replace?

• 1:22:19Speaker 06

  My answer is my my meta answer is you always it's always the right bet to just get good at the most powerful new tools, most capable new tools. And so when computer programming was that, you did want to become a programmer. And now that AI tools like totally change what one person can do, you want to get really good at using AI tools. And and so like having a sense for how to work with chat GPT and other things, that is the high ground. And that's like, that's we're not going back. Like, that's going to be part of the world. And you can use it in all sorts of ways, but getting fluent at it, I think, is really important.

• 1:22:58Kevin Roose?

  I want to challenge that because I I I think you're partially right in that I think there is an opportunity for people to embrace, you know, AI and and sort of become more resilient to disruption that way. But I also think if you look back through history, it's not like we learn how to do something new and then the old way just goes away. Right? We still make things by hand. There's still an artisanal market. So do you think there's going to be people who just decide, you know what? I don't want to use this stuff.

• 1:23:26Speaker 06

  Totally.

• 1:23:27Kevin Roose?

  And they're gonna like, there's going to be something valuable in their sort of, I don't know, non-AI assisted work.

• 1:23:33Speaker 06

  I expect like.

• 1:23:36Speaker 06

  I expect that if we look forward to the future, things, everything that, things that we want to be cheap can get much cheaper and things that we want to be expensive are going to be astronomically expensive.

• 1:23:47Casey Newton?

  Like what?

• 1:23:48Speaker 06

  Real estate, like handmade goods, art.

• 1:23:53Speaker 06

  And so totally, like, there'll be a huge premium on things like that. And there'll be many people who like, really, you know, there's always been like a, even when machine-made products have been much better, there has always been a premium on handmade products and I'd expect that to intensify.

• 1:24:09Casey Newton?

  This is also a bit of a curveball, very curious to get your thoughts. Where do you come down on the idea of AI romances? Are these net good for society?

• 1:24:18Speaker 06

  I don't want one personally.

• 1:24:19Casey Newton?

  You don't want one? Okay. But it's clear that there is a huge demand for this, right? Yeah. Like, I, I, I think that, I mean, you know, Replica is building these, they seem like they're doing very well. I would be shocked if this is not a multi-billion dollar company, right?

• 1:24:31Speaker 06

  Someone will make a multi-billion dollar company.

• 1:24:33Casey Newton?

  That's what I'm saying, yeah, somebody will.

• 1:24:33Speaker 06

  For sure.

• 1:24:34Casey Newton?

  Yeah. Do you, uh, like, I just personally think we're gonna have a big culture war. Like, I think Fox News is gonna be doing segments about the generation lost to AI girlfriends or boyfriends, like, at some point within the next few years. But at the same time, you look at all the data on, on loneliness, and it seems like, well, if we can give people companions that make them happy during the day, it could be a net good thing.

• 1:24:52Speaker 06

  It's complicated.

• 1:24:53Casey Newton?

  Yeah.

• 1:24:54Speaker 06

  You know, I have, I have misgivings, but I don't, this is not a place where I think I get to, like, impose what I think is good on other people.

• 1:25:01Casey Newton?

  Totally. Okay, but it sounds like this is not at the top of your product roadmap, is building the, the boyfriend API.

• 1:25:06Speaker 06

  No.

• 1:25:06Casey Newton?

  All right.

• 1:25:07Kevin Roose?

  You recently posted on X that you expect AI to be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to some very strange outcomes. Can you expand on that? Like, what are some things that AI might become very good at persuading us to do? And what are some of those strange outcomes you're worried about?

• 1:25:26Speaker 06

  The thing I was thinking about at that moment was the upcoming election. There's a huge focus on the US 2024 election. There's a huge focus on deep fakes and the impact of AI there. And I think that's reasonable to worry about, good to worry about. But we already have some societal antibodies towards people seeing, like, doctored photos or whatever, and, yeah, they're gonna get more compelling, there's gonna be more, but we kind of know that those are there. There's a lot of discussion about that. There's almost no discussion about what are, like, the new things AI can do to influence an election, AI tools can do to influence an election. And one of those is to, like, carefully

• 1:26:03Speaker 06

  You know, one-on-one persuade individual people.

• 1:26:06Casey Newton?

  Tailored messages.

• 1:26:07Speaker 06

  Tailored messages. That's like a new thing that the content firms couldn't quite do.

• 1:26:11Kevin Roose?

  Right. And that's not AGI, but that could still be pretty harmful.

• 1:26:15Speaker 06

  I think so, yeah.

• 1:26:17Kevin Roose?

  I know we are running out of time, but I do want to push us a little bit further into the future than the sort of, I don't know, maybe five-year horizon we've been talking about. If you can imagine a good post AGI world, a world in which we have reached this threshold, whatever it is, what does that world look like? Does it have a government? Does it have companies? What do people do all day?

• 1:26:42Speaker 06

  Like a lot of material abundance. People are, people continue to be very busy, but the, the way we define work always moves. Like if you, our jobs would not have seemed like real jobs to people several hundred years ago, right? This would have seemed like incredibly silly entertainment. It's important to me, it's important to you. And hopefully it has some value to other people as well. There will be, and the jobs of the future may seem, I hope they seem even sillier to us, but I hope the people get even more fulfillment and I hope society gets even more fulfillment out of them. But everybody can have a really great quality of life, like to a degree that I think we probably just can't imagine now. Of course, we'll still have governments. Of course, people will still squabble over

• 1:27:22Speaker 06

  whatever they squabble over. You know, less different in all of these ways than someone would think. And, and then like unbelievably different in terms of what you can get a computer to do for you.

• 1:27:34Kevin Roose?

  One fun thing about becoming a very prominent person in the tech industry as you are, is that people have all kinds of theories about you. One fun one that I heard the other day is that you have a, a secret Twitter account where you are way less measured and careful.

• 1:27:50Speaker 06

  I don't anymore. I did for a while. I decided I just couldn't keep up with the OPSEC.

• 1:27:55Casey Newton?

  It's so hard to lead a double life.

• 1:27:56Kevin Roose?

  What was your, what was your secret Twitter account?

• 1:27:58Speaker 06

  Obviously, I can't. I, I, I mean, I had a good alt. A lot of people have good alts, but you know.

• 1:28:03Casey Newton?

  Your name is literally Sam Altman. I mean, it would have been weird if you didn't have one.

• 1:28:07Speaker 06

  But I think I just got. Yeah, too, like, too well-known or something to be doing that.

• 1:28:11Kevin Roose?

  Yeah. Well, and the sort of theory that I heard attached to this was that you are you are secretly an accelerationist, a person who wants AI to go as fast as possible. And then all this careful diplomacy that you're doing and asking for regulation, this is really just the sort of polite face that you put on for society. But deep down, you just think we should go all gas, no brakes toward the future.

• 1:28:32Speaker 06

  No, I certainly don't think all gas, no brakes to the future, but I do think we should go to the future. And that probably is what differentiates me than like most of the AI companies. I think AI is good. Like, I don't secretly hate what I do all day. I think it's gonna be awesome. Like, I want to see this get built. I want people to benefit from this. So all gas, no brake, certainly not. And I don't even think like most people who say it mean it. But I am a believer that this is a tremendously beneficial technology and that we have got to find a way safely and responsibly to get it into the hands of the people, to confront the risks so that we get to enjoy the huge rewards. And like, you know, maybe relative to the prior of most people who work on AI, that does make me an accelerationist.

• 1:29:18Speaker 06

  But compared to those like accelerationist people, I'm clearly not them. So, you know, I'm like somewhere, I think you like want the CEO of this company to be somewhere.

• 1:29:26Kevin Roose?

  Your acceleration is adjacent.

• 1:29:27Speaker 06

  Which I think I am.

• 1:29:28Kevin Roose?

  You're gas and brakes.

• 1:29:31Speaker 06

  I believe that this will be the most important and beneficial technology humanity has ever, has yet invented. And I also believe that if we're not careful about it, it can be quite disastrous. And so we have to navigate it carefully.

• 1:29:44Kevin Roose?

  Yeah.

• 1:29:44Speaker 06

  Yeah.

• 1:29:45Casey Newton?

  Sam, thanks so much for coming on Hard Work.

• 1:29:47Speaker 06

  Thank you, guys.

• 1:29:51Kevin Roose?

  When we come back, we'll have some notes on that interview now with the benefit of hindsight.

• 1:30:11Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 1:30:45Speaker 03

  AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with realtime guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 1:31:17Speaker 04

  Ripple has been powering modern finance for a decade. It's not our first trip around the blockchain, and it won't be our last. We're leading the charge to bridge the gap between existing financial systems and the future of finance. You can have it all. Legacy, stability, and digital innovation. Systems that work with your other systems. Because with the right partner, the question isn't crypto or traditional finance. The question is simply, why choose? A new standard of modern finance. It's happening with Ripple. For informational purposes only. Availability of products can vary. Maybe provided by different Ripple entities.

• 1:31:51Kevin Roose?

  So Casey, now with, you know, five days of hindsight on this interview and after everything that has transpired between the time that we originally recorded it and now, are there any things that Sam said that stuck out to you as being particularly relevant to understanding this conflict?

• 1:32:08Casey Newton?

  You know, I keep coming back to the question that you asked him about whether he was a closet accelerationist, right? Is he somebody who is telling the world, hey, I'm trying to do this in a very gradual iterative way, but behind the scenes is working to hit the accelerator? And during that interview, he gave a sort of very diplomatic answer, as you might expect to that question. But learning what we have learned over the past few days, I do feel like he is on the more accelerationist side of things. And certainly, all of the people rallying to his defense on social media over the weekend, a good number of them were rallying because they think he is the one who is pushing AI forward. How about you? What'd you think?

• 1:32:49Kevin Roose?

  Totally. I thought I thought that was very interesting. And now with the additional context of the last, you know, three days, explains a lot about the conflict between Sam Altman and the board. We still don't obviously know exactly what happened, but I can imagine that Sam going around saying things like, I think that the future is going to be amazing, and I think that everything's going to be great with AI. I can see why that would land poorly with board members who are much more concerned from the looks of things about how the future is going to look. So it seems like he's sort of an optimist who is running a company where the board of that company is less optimistic about AI. And that just seems like a fundamental tension that it sounds like they were not able to

• 1:33:34Kevin Roose?

  get past. I was also struck by something else that he said. It was interesting, when we talked about GPTs, these like build your own chatbots that OpenAI released at Dev Day a few weeks ago, he said that he was embarrassed because they were so simple and and sort of not all that functional and and pretty prosaic. And that's just such a striking contrast, because some of the reporting that came out over the weekend suggested that the GPTs were actually one of the things that scared Ilya Sutskever and the board, that giving these AIs more agency and more autonomy and allowing them to do things on the internet was, at least if you believe the reporting, part of what made the board so anxious.

• 1:34:18Casey Newton?

  Yes, and at the same time, if it is true that the board and Ilya found out about GPT's at Developer Day, that speaks to some fundamental problems in how this company was being run. And I don't know if that is a Sam thing, or a board thing, or what. But you would think that by the time the keynote was being delivered, all of those stakeholders would have been looped in.

• 1:34:41Kevin Roose?

  Totally. And I guess my other reflection on that interview is that it just sounded like Sam had no idea that any of this was brewing. This did not sound like someone who was trying to carefully walk the line between, you know, being optimistic and being, you know, scared of existential risk. This did not sound like someone who thought that he was on thin ice with his board. This sounded like someone who was very confidently charging ahead with his vision for the future of AI.

• 1:35:10Casey Newton?

  That's right.

• 1:35:11Kevin Roose?

  I really hope we are not doing more emergency podcasts on this. Could the news just give us a little break for a minute?

• 1:35:17Casey Newton?

  Well, if I were you, Kevin, I would clear your Tuesday morning.

• 1:35:20Kevin Roose?

  Oh, God. Happy Thanksgiving.

• 1:35:23Casey Newton?

  Happy Thanksgiving.

• 1:35:26Kevin Roose?

  Heartfork is produced by Rachel Cohn and Davis Land. We're edited by Jen Poyant. Today's show is engineered by Rowan Nemisto. Original music by Marian Lozano, Rowan Nemisto, and Dan Powell. Our audience editor is Nell Galogly. Video production by Ryan Manning and Dylan Bergerson. Special thanks to Paula Schuman, Puywing Tam, Kate LaPresti, and Jeffrey Miranda. As always, you can email us at [email protected].

• 1:36:26Speaker 07

  Seventy-five percent of women who seek menopause care don't receive it. The lying awake at 3 a.m. wondering what's happening to your body ends now. Midi is virtual care built for women in perimenopause and menopause with clinicians who specialize in women's health. Plus, it's covered by most major insurances. Getting started is easy. You can book your first visit with a clinician right away. Midlife needs Midi. Visit joinmidi.com to book your first visit today. That's join m-i-d-i dot com. Insurance coverage varies. Check with your plan for coverage.