Google Eats Rocks + A Win for A.I. Interpretability + Safety Vibe Check

“Pass me the nontoxic glue and a couple of rocks, because it’s time to whip up a meal with Google’s new A. I. Overviews.”

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 0:34Kevin Roose?

  Hello, Casey.

• 0:35Casey Newton?

  Hey, Kevin.

• 0:36Kevin Roose?

  How was your Memorial Day weekend?

• 0:37Casey Newton?

  It was wonderful. I got to go to a beautiful wedding and very much enjoyed that.

• 0:45Kevin Roose?

  Nice.

• 0:45Casey Newton?

  How was your Memorial Day weekend?

• 0:47Kevin Roose?

  It was good. But I feel like you have something that you didn't bring up, which is that you actually had a big launch this past weekend.

• 0:54Casey Newton?

  I did a hard launch. I mean, I guess I did a hard launch a boyfriend like once before on Instagram, but it was many years ago. And this one, I think like at this point, hard launches, people sort of know what they are. And so a lot of like thought goes into it.

• 1:08Kevin Roose?

  Well, so a hard launch, just so I'm clear with the latest lingo. This is when you announce that you have a new boyfriend on Instagram.

• 1:15Casey Newton?

  Well, because if the soft launch is like, like, you know, if maybe you see somebody's shoulder in an Instagram story and you think, well, that's a new person. Like, how is that? Is that, is my friend? Are they dating that person? That's a soft launch. But once there's a face and a name, that's a hard launch.

• 1:28Kevin Roose?

  I see. So you debuted, you hard launched your new boyfriend.

• 1:32Casey Newton?

  We had, we had, and it had been, you know, some time in coming. And of course, I had to, you know, check in with him and make sure he was going to be OK with this. And, you know, he was, he was excited about it.

• 1:42Kevin Roose?

  And you did it on the grid, which was bold.

• 1:44Casey Newton?

  Of course, I did it on the grid. I want to, you know, I want to show everyone. I can't just have that disappear in 24 hours.

• 1:50Kevin Roose?

  Yeah, how did it go?

• 1:51Casey Newton?

  Hard launch went very well. You know, I mean.

• 1:55Kevin Roose?

  Was the engagement what you'd hoped for?

• 1:56Casey Newton?

  The engagement was off the chart. It was my most popular Instagram post I've ever done.

• 2:01Kevin Roose?

  Did he also hard launch you on his Instagram?

• 2:04Casey Newton?

  Yes, so it was honestly very stressful.

• 2:07Kevin Roose?

  There was a whole content plan, there were whiteboards.

• 2:10Casey Newton?

  Well, we took like dozens of photos.

• 2:12Kevin Roose?

  Did you hire a marketing agency?

• 2:14Casey Newton?

  Yeah, our teams got involved. No, we'd taken so many photos and, you know, so of course we're like sitting and we're like, we're gonna do this photo, we're gonna do this photo. Is this one a little edgy? Let's do it anyway. And so we came up with five photos and then, yes, we like more or less simultaneously did the launch.

• 2:30Kevin Roose?

  Yeah, wow. See, I've been out of the game so long that the only thing I remember is that you could like change your relationship status on Facebook and that was the hard launch of like 2008.

• 2:40Casey Newton?

  Yes, absolutely. And so, of course, in my mind, you know, because I also had that sort of millennial urge, like, do I make this Facebook official? But I'm just like, no, that just seems like, that seems so boomer coded at this point.

• 2:49Kevin Roose?

  You have to make it LinkedIn official. That's when it truly becomes real.

• 2:53Casey Newton?

  I got into a relationship recently. Here's 10 lessons that I have about enterprise software.

• 3:03Kevin Roose?

  I'm Kevin Roos, a tech columnist at The New York Times.

• 3:05Casey Newton?

  I'm Casey Newton from Platformer.

• 3:06Kevin Roose?

  And this is Hard Fork.

• 3:07Casey Newton?

  This week, Google tells us all to eat rocks.

• 3:10Kevin Roose?

  We'll tell you where its AI went wrong.

• 3:12Casey Newton?

  Then, anthropic researcher Josh Batson joins to talk about a breakthrough in understanding how large language models work. And finally, it's this week in AI safety as I try out OpenAI's new souped-up voice assistant and then it gets cruelly taken away from me.

• 3:25Kevin Roose?

  I'm so sorry that happened.

• 3:27Casey Newton?

  Me too.

• 3:39Casey Newton?

  Well, Kevin, pass me the non-toxic glue and a couple of rocks because it's time to whip up a meal with Google's new AI overviews. Did you make any recipes you found on Google this week?

• 3:50Kevin Roose?

  I did not, but I saw some chatter about it and I actually saw our friend Katie Natopoulos actually made the glue pizza. But we're getting ahead of ourselves.

• 4:00Casey Newton?

  We're getting ahead of ourselves. And look, the fact that you stayed away from this stuff explains why you're still sitting in front of me. Because over the past week, Google found itself in yet another controversy over AI, this time related to search, the core function of Google. And right after that, we had this huge leak of documents that brought even more attention to search and raised the question of whether Google's been dishonest about its algorithms, Kevin, can you imagine?

• 4:24Kevin Roose?

  Wow, so there's a lot there.

• 4:26Casey Newton?

  Yes.

• 4:26Kevin Roose?

  Let's just go through what happened, because the last time we talked about Google on this podcast, they had just released this new AI overviews feature, and this is the thing that shows you a little AI-generated snippet above the search results when you type in your query. And I think it's fair to say that this did not go smoothly.

• 4:43Casey Newton?

  It didn't. And I want to talk about everything that happened with those AI overviews. But before we get there, Kevin, I think we should take a step back and talk about the recent history of Google's AI launches. Can we do that real quick?

• 4:55Kevin Roose?

  Yes.

• 4:56Casey Newton?

  Because I would say there's kind of an escalation in how bad things have gotten. So let's go back to February 2023 and talk about the release of Google Bard. Kevin, when I say the word Bard, what does that conjure up for you?

• 5:10Kevin Roose?

  Shakespeare.

• 5:11Casey Newton?

  Yep, Shakespeare, number one. And probably number two would be the late lamented Google chatbot?

• 5:15Kevin Roose?

  Yes, RIP.

• 5:17Casey Newton?

  Fun fact, Kevin and I were recently in a briefing where a Google executive had a sticker on their laptop that said "Total Bardass." And that sounds like a joke. And you actually texted me.

• 5:27Kevin Roose?

  I texted you.

• 5:28Casey Newton?

  And you said, "Does that sticker say Total Bardass?"

• 5:31Kevin Roose?

  Total Bardass.

• 5:31Casey Newton?

  And I said, "It couldn't possibly." And then I zoomed in, I said, "Computer enhance." And indeed, it did say "Total Bardass."

• 5:37Kevin Roose?

  And if you are a Googler who has access to a sticker.

• 5:41Casey Newton?

  We're dying for one.

• 5:42Kevin Roose?

  That says "Total Bardass." I want one, I will put it on my laptop.

• 5:45Casey Newton?

  Please.

• 5:45Kevin Roose?

  It belongs in the Smithsonian.

• 5:46Casey Newton?

  We're begging you for it. So this comes out in February 2023, and unfortunately, the very first screenshot posted of Google's AI chatbot, it gave incorrect information about the James Webb Space Telescope. Specifically, it falsely stated that the telescope had taken the first ever photo of an exoplanet.

• 6:04Kevin Roose?

  Yes.

• 6:04Casey Newton?

  Kevin, without binging, what is an exoplanet?

• 6:07Kevin Roose?

  It's a planet that signs its letters like with a hug and a kiss.

• 6:11Casey Newton?

  No, it's actually the planet where all my exes live. But let's just say that Google AI launches had not gotten off to a great start when that happened. In fact, we talked about that one on this show. Then comes the launch of Gemini. And then we had a culture war, Kevin, over the refusals of its image generator to make white people.

• 6:27Kevin Roose?

  Sure did.

• 6:28Casey Newton?

  Do you have a favorite thing that Gemini refused to make due to wokeness?

• 6:32Kevin Roose?

  My, I was partial to Asian Sergey and Larry. Do you remember this?

• 6:35Casey Newton?

  Wait, I actually forgot this one. What was this one?

• 6:37Kevin Roose?

  Somebody asked Gemini to make an image of the founders of Google, Sergey Brin and Larry Page. And it came back and they were both Asian.

• 6:46Kevin Roose?

  Which I love. I have to imagine that ended up in a, you know, projected onto a big screen in a meeting somewhere at Google.

• 6:52Casey Newton?

  That's so beautiful to me. So, look, that brings us to the AI overviews. And Kevin, you sort of said it up top, but can you remind us a little bit of how do these things work? What are they?

• 7:01Kevin Roose?

  So this is the, what used to be known as search generative experience when it was being tested. But this is the, the big bet that Google is making on the future of AI in search. Obviously, they have seen the, the rise of products like Perplexity, which is this AI powered search engine. They believe, Sundar Pichai said, you know, that he believes that AI is the future of search and that these AI overviews that appear on top of search results will ultimately give you a better search experience. Because instead of having to click through a bunch of links to figure out what you're looking for, you can just see it displayed for you, generated right there up at the top of the page.

• 7:37Casey Newton?

  Right. And very briefly, why have we been so concerned about these things?

• 7:40Kevin Roose?

  Well, I think your concern that I shared was that this was ultimately going to lock people into the Google walled garden. That instead of going to links where you might see an ad, you might buy a subscription, you might support, you know, the, the news or the, the media ecosystem in some way. Instead, Google was just going to kind of keep you there on Google. The phrase they would use over and over again was, we will do the Googling for you.

• 8:05Casey Newton?

  That's right.

• 8:05Kevin Roose?

  And that it would sort of starve the web of the essential referral traffic that sort of keeps the whole machine running.

• 8:13Casey Newton?

  So that is a big concern and I continue to have it every single day. But this week, Kevin, we got a second concern, which is that the AI overviews are going to kill your family. And here's what I mean. Over the past week, if you ask Google, "How many rocks should I eat?", the AI overview said, "At least one small rock per day." I verified this one myself. As you referenced up top, if you said, "How do I get the cheese to stick to my pizza?", it would say, "Well, have you considered adding nontoxic glue?" Wouldn't have been my first guess.

• 8:42Kevin Roose?

  Yeah, at least it said "nontoxic glue."

• 8:45Casey Newton?

  Very nice of the algorithm. It said that 17 of the 42 American presidents have been white. To me, the funniest thing about that is that there have been 46 US presidents.

• 8:56Kevin Roose?

  You got both the numerator and the denominator wrong.

• 8:59Casey Newton?

  And of course, and this was probably the most upsetting to our friends in Canada. It said that there has been a dog who played hockey in the National Hockey League. Do you see that one?

• 9:08Kevin Roose?

  Well, I think that was just the plot of Air Bud, right?

• 9:10Casey Newton?

  Yeah, well, there's no rule that says a dog can't play hockey, Kevin. And it identified that dog as Martin Pasposil.

• 9:18Kevin Roose?

  Who is that?

• 9:19Casey Newton?

  Well, it seems impossible, although you've never heard of him. But he's a 24-year-old Slovakian man who plays for the Calgary Flames. Guess you're not a big Flames fan.

• 9:27Kevin Roose?

  I'm not.

• 9:29Casey Newton?

  So, look, how is this happening? Well, Google is pulling information from all over the internet into these AI overviews. And in so doing, it is revealing something we've talked about on the show for a long time, which is that the large language models truly do not know anything. They can often give you answers, and those answers are often right, but they are not drawing on any frame of knowledge. They're simply reshuffling words that they found on the internet.

• 9:55Kevin Roose?

  Oh, see, this, I drew a different lesson from this, which was that this, the technology is actually only partly to blame here. Because I've used a bunch of different AI search products, including Perplexity, and not all of them make these kinds of stupid errors. But Google's AI model that it's using for these AI overviews seems to just be qualitatively worse. Like, it just can't really seem to tell the difference between reliable sources and unreliable sources. So the thing about eating rocks appears to have come from The Onion. That is like the satirical. It's a satirical news site.

• 10:30Casey Newton?

  Wait, you're saying that every story published on The Onion is false?

• 10:33Kevin Roose?

  I am, yes.

• 10:34Casey Newton?

  That seems like an interesting choice to include in your AI overviews for facts.

• 10:38Kevin Roose?

  Right, and the thing about adding glue to your pizza recipe came from basically a shitpost on Reddit. So, obviously, these AI overviews are imperfect. They are drawing from imperfect sources. They are summarizing those imperfect sources in imperfect ways. It is a big mess. And this got a lot of attention over the weekend. And as of today, I tried to replicate a bunch of these queries, and it appears that Google has fixed these specific queries very quickly. Clearly, they were embarrassed by it. I've also noticed that these AI overviews just are barely appearing at all, at least for me. Are they appearing for you?

• 11:16Casey Newton?

  I'm seeing a few of them, but yes, they have definitely been playing a game of whack-a-mole. And whenever one of these screenshots has gone anything close to viral, they are quickly intervening. Now, I should say that Google has sent me a statement about what's going on, if you would like me to share.

• 11:31Kevin Roose?

  Sure.

• 11:32Casey Newton?

  It said, the company said, quote, "The vast majority of AI overviews provide high-quality information with links to dig deeper on the web. Many of the examples we've seen have been uncommon queries, and we've also seen examples that were doctored or that we couldn't reproduce." It says some more things, and then says, "We're taking swift action where appropriate under our current policies and using these examples to develop broader improvements to our systems." So, they're basically saying, "Look, you're cherry picking, right? You went out and you found the absolute most ridiculous queries that you can do, and now you're holding it against us. And I would like to know, Kevin, how do you respond to these charges?"

• 12:07Kevin Roose?

  I mean, I think it's true that some people were just deliberately trolling Google by putting in these very sort of edge case queries that, you know, real people, many of them are not Googling, like, "Is it safe to eat rocks?" Right? That is not a common query. And I did see some ones that were clearly faked or doctored. So, I think Google has a point there. But I would also say, like, these AI overviews are also making mistakes on what I would consider much more common, sort of, normal queries. One of them that the AI overview botched was about how many Muslim presidents the US has had. The correct answer is zero, but the AI overview answer was one.

• 12:48Casey Newton?

  George Washington.

• 12:50Kevin Roose?

  Yes, George Washington. No, it said that Barack Hussein Obama was America's first and only Muslim president.

• 12:57Casey Newton?

  Obviously, not true.

• 12:58Kevin Roose?

  Not true. But that is the kind of thing that Google was telling people in its AI overviews that I imagine are not just, like, fringe or trollish queries.

• 13:06Casey Newton?

  Right. And also, like, I guess it has always been the case that if you did a sort of weird query on Google, you might not get the answer you were looking for, but you would get a webpage that someone had made, right? And you would be able to assess, hmm, does this website look professional? Like, does it have a masthead? Like, do the authors have bio? You could just sort of ask yourself some basic questions about it. Now, everything is just being compressed into this AI slurry, so you don't know what you're looking at.

• 13:33Kevin Roose?

  So, I have a couple of things to say here.

• 13:35Casey Newton?

  Say it.

• 13:37Kevin Roose?

  I think in the short term, this is a fixable problem. Look, I think it's clearly embarrassing for Google. They did not want this to happen. It's a big, you know, rake in the face for them. But I think what helps Google here is that Google Search and Search in general is what they call a fat head product. Do you know what that means?

• 13:55Casey Newton?

  I don't know what that means.

• 13:55Kevin Roose?

  So it means, basically, if you take a distribution curve, the most popular queries on Google or any other search engine account for a very large percentage of search volume. Actually, according to one study, the 500 most popular search terms make up 8.4% of all search volume on Google. So...

• 14:15Casey Newton?

  Yeah, because a lot of people are just searching, like, Facebook and then clicking the link to go to Facebook.

• 14:18Kevin Roose?

  Exactly. Or they're searching something else that's, you know, very common, you know. What would be an example of a good...

• 14:25Casey Newton?

  Like, has a dog ever played hockey? No? Okay.

• 14:29Kevin Roose?

  No, stuff like...

• 14:30Casey Newton?

  What time is the Super Bowl?

• 14:32Kevin Roose?

  Yeah, what time is the Super Bowl? Or, you know, how do I fix a broken toilet or something like that?

• 14:37Casey Newton?

  Local movie times.

• 14:38Kevin Roose?

  Exactly.

• 14:39Casey Newton?

  Yeah.

• 14:39Kevin Roose?

  And, you know, for... That means that Google can sort of manually audit the top, I don't know, say, 10,000 AI overviews, make sure they're not giving people bad information. And that would mean that the vast majority of what people search for on Google does actually have a correct AI overview. Now, in that case, it wouldn't actually technically be an AI overview. It'd be sort of like a human overview that was sort of drafted by AI. But same difference in Google's eyes. I also think they can make sure that AI overviews aren't triggered for sensitive topics, for things where your health are concerned. Google already does this to a certain extent with some, with these things called featured snippets.

• 15:20Kevin Roose?

  And I think they will continue to sort of play around with and adjust the dials on how frequently these AI overviews are triggered. But I do think there's a bigger threat to Google here, which is that they are now going to be held responsible for the information that people see on Google. We've talked about this a little bit, but I mean, this to me is the biggest complaint that people have that is justified is that Google used to, you know, maybe they would point you to a website that would tell you that, you know, putting glue on your pizza is a good way to get the cheese to stick, but you as Google could sort of wash your hands of that and say, oh, that was people just trolling on Reddit. That wasn't us. But if you're Google and you're now providing this AI-written overview to people, people are going to get mad when it gives you wrong information. And there will be, unfortunately,

• 16:04Kevin Roose?

  just the law of large numbers says that, you know, sometime, you know, maybe in the next year or two, there will be an instance where someone relies on something they saw on a Google AI overview and it ends up hurting them.

• 16:16Casey Newton?

  Yeah, there was another query that got a lot of attention this week where an AI overview told someone that you could put gasoline in spaghetti to make a spicy dish, that you couldn't use gasoline to cook spaghetti faster, but if you wanted to have a spicy spaghetti, you could put gasoline in it. And of course, that sounds ridiculous to us, but over the entire long tail of the internet, is it theoretically possible somebody would eat gasoline spaghetti? Of course it is.

• 16:43Kevin Roose?

  Yeah, so I think, and when that does happen, I think there are two questions. One is, is Google legally protected? Because I've heard some interesting arguments about whether Section 230, which is the part of the US code that protects online platforms from being held legally responsible for stuff that their users post, there are a lot of people who think that doesn't apply to these AI overviews because it is Google itself that is formulating and publishing that overview. I also just think there's a big reputational risk here. I mean, you can imagine so easily the congressional hearings where, you know, senators are yelling at Sundar Pichai, saying, why did you tell my kid to eat gasoline spaghetti?

• 17:21Casey Newton?

  Martin Posposil is going to be there saying, do I look like a dog to you?

• 17:24Kevin Roose?

  Right. And seriously, I think that this is a big risk for Google, not just because they're going to have to sit through a bunch of hearings and get yelled at, but because I think it will make their active Role in search, which has been true for many years. They have been actively shaping the experience that people have when they search stuff on Google. But they've mostly been able to kind of obscure that away or abstract it away. And say, well, this is just our sort of system working here. I think this will make their active role in kind of curating the search results for billions of people around the world much more obvious and will make them much more responsible in users' eyes.

• 17:59Casey Newton?

  I think all of that is true. I have an additional concern, Kevin.

• 18:04Kevin Roose?

  Yes.

• 18:04Casey Newton?

  This was pointed out by Rusty Foster, who writes The Gray Today and Tabs newsletter. And he said, what has really been revealed to us about what AI overviews really are is that they are automated plagiarism. That is the phrase that he used, right? That Google has scanned the entire web. It's looked at every publisher. It lightly rearranges the words and then it republishes it into the AI overview. And, you know, as journalists, we really try not to do this, right? We try not to just go out, grab other people's reporting, very gently change the words and republish it as our own. And in fact, you know, I know people who have been fired for doing something very similar to this, right? But Google has come along and said, well, that's actually the foundation of our new system that we're using to replace search results.

• 18:49Kevin Roose?

  Yeah. Casey, what do you think comes next with this AI overviews business? Is Google just going to sort of back away from this and it's not ultimately going to be a huge part of their product going forward? Do you think they will just sort of grit their teeth and get through this initial period of awkwardness and inaccuracy? What do you think happens here?

• 19:08Casey Newton?

  They are not going to back down. Now, they might temporarily retreat like we've seen them do in, you know, the Gemini image case, but they are absolutely going to keep working on this stuff because this is existential for them. For them, this is the next version of search. This is the way they build the Star Trek computer. They want to give you the answer. And in many more cases over time, they want you to not have to click a link to get any additional information. They already have rivals like perplexity that seem to be doing a better job in many cases of answering people's queries. And Google has all of the money and talent it needs to figure out that problem. So they're going to keep going at this at 100 miles an hour.

• 19:45Kevin Roose?

  Yeah. I want to bring up one place that I actually sort of disagree with you because you wrote recently that you believe that because of these changes to Google, that the web is sort of in a state of managed decline. And we've gotten some listener feedback in the past few weeks as we've been talking about sort of these issues of Google and AI and the future of the web, saying like, you guys are basically acting as if the previous state of the internet was healthy. Like, Google was, you know, giving people high-quality information, like there was this flourishing internet of independent publishers, kind of like making money and, you know, serving users really well. And people just said like, it actually wasn't like that at all.

• 20:25Kevin Roose?

  In fact, the previous state of the web, at least for the past few years, has been in decline. So it's not that we are entering an age of managed decline of the internet. It's that Google is basically accelerating what was already happening on the internet, which was that publishers of high-quality information are putting that information behind paywalls. There are all these publishers who are chasing these sort of SEO traffic wins with this sort of low-quality garbage. And essentially, the web is being hollowed out, and this is maybe just accelerating that. So I just want to float that as like a theory, a sort of counterproposal for your theory of Google putting the web into a state of managed decline.

• 21:02Casey Newton?

  Well, sure, Kevin, but if you ask yourself, well, why is that the case? Why are publishers doing all of these things? It is because the vast majority of all digital advertising revenue goes to three companies, and Google is at the top of that list with Meta and then Amazon at number two and three. So my overall theory about what's happening to the web is that three companies got too much of the money and starved the web of the lifeblood it needed to continue expanding and thriving. So, look, has it ever been super easy to whip up a digital media business and just put it on the internet and start printing cash? No, it's never been easy. My theory is just that it's almost certainly harder today than it was five years ago,

• 21:44Casey Newton?

  and it will almost certainly be harder in five years than it is today. And it is Google that is at the center of that story because at the end of the day, they have their fingers on all of the levers and all of the knobs. They get to Who gets to see an AI overview? You know, how quickly do we roll these out? What categories do they show them in? If web traffic goes down too much and it's a problem for them, then they can slow down. But if it looks good for them, they can keep going, even if all the other publishers are kicking and screaming the whole time. So I just want to draw attention to the amount of influence that this one company in particular has over the future of the entire internet.

• 22:19Kevin Roose?

  Yeah, and I would just say that is not a good state of affairs. And it has been true for many years that Google has huge unchecked influence over basically the entire online ecosystem.

• 22:31Casey Newton?

  All right, so that is the story of the AI overview. But there was a second story that I want to touch on briefly this week, Evan, that had to do with Google and search. And it had to do with a giant leak. Have you seen the leak?

• 22:44Kevin Roose?

  I've I've heard about the leak. I have not examined the leak, but tell me about the leak.

• 22:48Casey Newton?

  Well, it was thousands of pages long, so I understand why you haven't finished reading it quite yet. But these were thousands of pages that we believe came from inside of Google that offer a lot of technical details about how the company's search works. So, you know, that is not a subject that is of interest to most people. But if you have a business on the internet and you want to ensure that your, you know, dry cleaners or your restaurant or your media company ranks highly in Google search without having to buy a bunch of ads, this is what you need to figure out.

• 23:19Kevin Roose?

  Yeah, this is one of the great guessing games in modern life. There's this whole industry of SEO that has sort of popped up to try to sort of poke around the Google search algorithm, try to guess and and sort of test what works and what doesn't work and sort of provide consulting, you know, for a for a very lucrative price to businesses that want to improve their Google search traffic.

• 23:39Casey Newton?

  Yeah, like the way I like to put it is, imagine you have a Glu Pizza restaurant and you want to make sure that you're the top ranked search for Glu Pizza restaurants. You might hire an SEO consultant.

• 23:48Kevin Roose?

  Yeah.

• 23:49Casey Newton?

  So what happened? Well, so there's this guy Rand Fishkin, who doesn't do SEO anymore, but was a big SEO expert for a long time and is kind of a leading voice in this space. And he gets an email from this guy, Irfan Azimi, who himself is the founder of an SEO company. And Azimi claims- To have access to thousands of internal Google documents detailing the secret inner workings of search and Rand reviews this information with Azimi and they determine that some of this contradicts what Google has been saying publicly about how search works over the year.

• 24:23Kevin Roose?

  Well, and this is the kind of information that Google has historically tried really hard to keep secret, both because it's kind of their secret sauce. They don't want competitors to know how the Google search algorithm works, but also because they have worried that if they sort of say too much about how they rank certain websites above others, then these sort of like SEO consultants will use that information and and it'll basically become like a cat and mouse game.

• 24:51Casey Newton?

  Yeah, absolutely. And it already is a cat and mouse game, but you know, the fear is that this would just sort of fuel the worst actors in the space. Of course, it also means that Google can fight off its competitors because people don't really understand how its rankings work. And if you think that Google search is better than anyone else's search, like these ranking algorithm decisions are why.

• 25:08Kevin Roose?

  Can I just ask a question? Do we know that this leak is genuine? Do we have any signs that these documents actually are from Google?

• 25:13Casey Newton?

  Well, yes. So the documents themselves had a bunch of clues that suggested they were genuine. And then Google did actually come out and confirm on Wednesday that these documents are real. But the obvious question is, how did something like this happen? And the leading theory right now is that these documents came from Google's content API warehouse, which is not a real warehouse, but is something that was hosted on GitHub, right? The sort of Microsoft owned service where people post their code. And these materials were somehow briefly made public by accident, right? Because a lot of companies will have private like API repositories on GitHub.

• 25:54Kevin Roose?

  Right. So they just sort of set it to public by accident. Sort of the the modern equivalent of like leaving a classified document in the cab.

• 26:00Casey Newton?

  Yeah. Have you ever made a sensitive document public on accident?

• 26:03Kevin Roose?

  No, and I've never found one either. I like, in all my years of reporting, I keep hoping to like stumble on the, you know, the scoop of this entry just sitting in the back of an Uber somewhere, but it never happens to me.

• 26:13Casey Newton?

  So, yeah, we're not going to go into these documents in too much detail. What I will say is it seems that these files contain a bunch of information about the kinds of data the company collects, including things like click behavior or data from its Chrome browser, things that Google has previously said that it doesn't use in search rankings. But the documents show that they have this sort of data and they could potentially use it to rank search results. When we asked Google about this, they wouldn't comment on anything specific, but a spokesperson told us that they, quote, "would caution against making inaccurate assumptions about search based on out of context, outdated or incomplete information." Anyway, why do we care about this? Well, I was just struck by one of the big conclusions

• 26:56Casey Newton?

  that Rand Fishkin had in this blog post that he wrote, quote, "They've been on an inexorable path toward exclusively ranking and sending traffic to big, powerful brands that dominate the web over small, independent sites and businesses." So basically, you look through all of these APIs and like, if you are a restaurant just getting started, if you're an indie blogger that just sort of puts up a shingle, it used to be that you might expect to automatically float to the top of Google search rankings in your area of expertise. And what Fishkin is saying is that just is getting harder now because Google is putting more and more emphasis on trusted brands. Now, that's not a bad thing in its own right, right? Like, if I Google something from the New York Times, I want to see the New York Times and not just a bunch of people

• 27:40Casey Newton?

  who put like New York Times in the header of their HTML. But I do think that this is one of the ways that the web is shrinking a little bit, right? Like, it's not quite as much of a free-for-all. The free-for-all wasn't all great because a lot of spammers and bad actors got into it, but it also meant that there was room for a bunch of new entrants to come in. There was room for more talent to come in. And one of the conclusions I had reading this stuff was, maybe that just isn't the case as much as it used to be.

• 28:04Kevin Roose?

  Yeah. So do you think this is more of a problem for Google than the AI overviews thing? How would you say it stacks up?

• 28:11Casey Newton?

  I would say it's actually a secondary problem. I think it's the telling people to eat rocks is the number one problem. They need to stop that right now. But this, I think, speaks to that story because both of these stories are about essentially the rich getting richer, the big brands are getting more powerful, whether that's Google getting more powerful by keeping everyone on search, or big publishers getting more powerful because they're the sort of trusted brands. And so I'm just observing that because, you know, the promise of the web and part of what it has made it such a joyful place for me over the past 20 years is that it is decentralized and open and there's just kind of a lot of dynamism in it. And now it's starting to feel a little static and stale and creaky.

• 28:53Casey Newton?

  And these documents sort of outline how and why that is happening.

• 28:57Kevin Roose?

  Yeah. I think Google is sort of stuck between a rock and a hard place here because on one hand, they do want, well, maybe we shouldn't use a rocks example.

• 29:08Casey Newton?

  No, use the rock example. They're stuck between a rock and a hard place. On the one hand, the company's telling you to eat rocks. On the other hand, they're in a hard place.

• 29:16Kevin Roose?

  Right. So I think Google is under a lot of pressure to do two things that are basically contradictory, right? To sort of give people an equal playing field on which to compete for attention and authority. That is the demand that a lot of these smaller websites and SEO consultants want them to comply with. On the other hand, they're also seeing with these AI overviews what happens when you don't privilege and prioritize authoritative sources of information in your search results or your AI overviews. You end up telling people to eat rocks. You end up telling people to put gasoline in their spaghetti. You end up telling people there are dogs that play hockey in the NHL.

• 29:57Kevin Roose?

  This is the kind of downstream consequence of not having effective quality signals to different publishers and to just kind of treating everything on the web as equally valid and equally authoritative.

• 30:09Casey Newton?

  I think that that is a really good point and that is something that comes across in these two stories is that exact tension.

• 30:14Kevin Roose?

  Casey, I have a question for you, which is, we also are content creators on the internet. We like to get attention. We want that sweet, sweet Google referral traffic. So for our next YouTube video, a stunt video, do you think that we should A, eat the gasoline spaghetti? B, eat one to three rocks apiece and see what effects it has on our health? Or C, teach a dog to play hockey at a professional level?

• 30:39Casey Newton?

  I mean, sheerly for how much fun it would be. We have to teach a dog how to play hockey.

• 30:43Kevin Roose?

  It's true.

• 30:44Casey Newton?

  You know? I'm just imagining like a bulldog with little hockey sticks maybe taped to its front paws.

• 30:50Kevin Roose?

  Yeah.

• 30:50Casey Newton?

  Be really fun.

• 30:51Kevin Roose?

  My dogs are too dumb for this. We'll have to find other dogs.

• 30:53Casey Newton?

  Yeah, was it in Lose Yourself that Eminem said, uh, "There's vomit on my sweater already, gasoline spaghetti"?

• 31:01Kevin Roose?

  I believe those were the words.

• 31:02Casey Newton?

  What a great song.

• 31:03Kevin Roose?

  Yeah.

• 31:06Kevin Roose?

  When we come back, we'll talk about a big research breakthrough into how AI models actually think.

• 31:26Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner. Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoai. That's dell.com/yourwaytoai.

• 32:00Speaker 03

  AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 32:32Speaker 04

  Ripple has been powering modern finance for a decade. It's not our first trip around the blockchain, and it won't be our last. We're leading the charge to bridge the gap between existing financial systems and the future of finance. You can have it all: legacy, stability, and digital innovation. Systems that work with your other systems. Because with the right partner, the question isn't crypto or traditional finance. The question is simply, why choose? A new standard of modern finance. It's happening with Ripple. For informational purposes only. Availability of products can vary and may be provided by different Ripple entities.

• 33:06Kevin Roose?

  Well, Casey, we have something new and unusual for the podcast this week.

• 33:09Casey Newton?

  What's that, Kevin?

• 33:10Kevin Roose?

  We have some actual good AI news.

• 33:12Casey Newton?

  Oh, about time.

• 33:13Kevin Roose?

  So as we've talked about on this show before, one of the most pressing issues with these large AI language models is that we generally don't know how they work, right? They are inscrutable, they work in mysterious ways, there's no way to tell why one particular input produces one particular output, and this has been a big problem for researchers for years. There has been this field called interpretability, or sometimes it's called mechanistic interpretability. Say that five times fast. And I would say that the field has been making steady but slow progress toward understanding how language models work. But last week, we got a breakthrough. Anthropic, the AI company that makes the Claude chatbot, announced that it had basically

• 33:58Kevin Roose?

  mapped the mind of their large language model Claude 3 and opened up the black box that is AI for closer inspection. Did you see this news and what was your reaction?

• 34:09Casey Newton?

  I did, and I was really excited because for some time now, Kevin, we have been saying, if you don't know how these systems work, how can you possibly make them safe? And companies have told us, well, look, we have these research teams and they're hard at work trying to figure this stuff out, but we've only seen a steady drip of information from them so far. And to the extent that they've conducted research, it's been on very small toy versions of the models that we operate with. So that means that if you're used to using something like Anthropic's Claude, its latest model, we really haven't had very much idea of how that works. So the big leap forward this week is they're finally doing some interpretability stuff with the real big models.

• 34:48Kevin Roose?

  Yeah, and we should just caution upfront that like it gets pretty technical pretty quickly once you start getting into the weeds of interpretability research. There's lots of talk about neurons and sparse auto encoders, things of that nature, so...

• 35:02Casey Newton?

  But I for one believe that hard fork listeners are the smartest listeners in the world and they're not going to have any trouble at all following along, Kevin. What do you think about our listeners?

• 35:09Kevin Roose?

  It's true. I also believe that we have smart listeners, smarter than us. And so even if we are having trouble understanding this segment... Hopefully you will not. But today, to walk us through this big AI research breakthrough, we've invited on Josh Batson from Anthropic. Josh is a research scientist at Anthropic, and he's one of the co-authors of the new paper that explains this big breakthrough in interpretability, which is titled "Scaling Monosemanticity: Extracting Interpretable Features From Claude-3 Sonnet."

• 35:39Casey Newton?

  Look, if you're not scaling monosemanticity at this point, what are you even doing with your life?

• 35:43Kevin Roose?

  What are you even doing with your life?

• 35:44Casey Newton?

  Figure it out.

• 35:45Kevin Roose?

  All right, let's bring in Josh.

• 35:46Casey Newton?

  Come on in here, Josh.

• 35:57Kevin Roose?

  Josh Batson, welcome to Hard Fork.

• 35:59Speaker 05

  Thank you.

• 35:59Kevin Roose?

  Hey, Josh. So there's this idea out there, this very popular trope, that large language models are a black box. I think, Casey, you and I have probably both used this in our reporting. It's sort of the most common way of saying, like, we don't know exactly how these models work. But I think it can be sort of hard for people who aren't steeped in this to understand just like what we don't understand. So help us understand, sort of prior to this breakthrough, what would you say we do and do not understand about how large language models work?

• 36:30Speaker 05

  So, in a sense, it's a black box that sits in front of us, and we can open it up, and the box is just full of numbers. And so, you know, words go in, they're turned into numbers, a whole bunch of compute happens, words come out the other side. But we don't understand what any of those numbers mean. And so, one way I like to think about this is like, you open up the box, and it's just full of thousands of green lights that are just like flashing like crazy. And it's like, something's happening, for sure. And like, different inputs, different lights flash, but we don't know what any of those patterns mean.

• 37:02Casey Newton?

  Is it crazy that despite that state of affairs that these large language models can still do so much? Like, it seems crazy that we wound up in a world where we have these tools that are super useful, and yet, when you open them up, all you see is green lights? Like, can you just say briefly why that is the case?

• 37:17Speaker 05

  It's kind of the same way that like... Animals and plants work and we don't understand how they work, right? These models are grown more than they are programmed. So you you kind of take the data and that forms like the soil and you construct an architecture and it's like a trellis and you shine the light and like that's the training. And then the the model sort of grows up here. And at the end, it's it's beautiful. It's all these little like curls and it's holding on. But like you didn't like tell it what to do.

• 37:46Kevin Roose?

  So it's almost like a more organic structure than something more linear. Well, and and help me understand why that's a problem because this is the the problem that the field of interpretability was designed to address. But there are lots of things that are very important and powerful that we don't understand fully, like we don't really understand how Tylenol works, for example, or some types of anesthesia. Their exact mechanisms are not exactly clear to us, but they work and so we use them. Why can't we just treat large language models the same way?

• 38:21Speaker 05

  That's a great analogy. You can use them. We use them right now. But Tylenol can kill people and so can anesthesia. And there's a huge amount of research going on in the pharmaceutical industry to figure out what makes some drugs safe and what makes other drugs dangerous. And interpretability is kind of like doing the biology on language models that we can then use to make the medicine better.

• 38:47Kevin Roose?

  So take us to your recent paper and your recent research project about the inner workings of large language models. How did you get there and then sort of walk us through what you did and what you found?

• 38:59Speaker 05

  So going back to the black box that when you open it is full of flashing lights. A few years ago, people thought you could just understand what one light meant. You know, when this light's on, it means that the model is thinking about code. And when this light's on, it's thinking about cats. And for this light, it's Casey Newton, you know? And that just turned out to be wrong. About a year and a half ago, we published a paper talking in detail about, you know, why it's not like one light, one idea. In hindsight, it seems obvious. It's almost as if we were trying to understand the English language by understanding individual letters. And we were asking like, what does C mean? Like, what does K mean? And that's just like the wrong picture. And so six months ago or so, we had some success with a method called dictionary

• 39:44Speaker 05

  learning for figuring out how the letters fit together into words. And like, what is the dictionary of kind of English words here? And so in this black box, green lights metaphor, it's that there are a few core patterns of lights, and given pattern would be like a dictionary word. And the internal state of the model at any time could be represented as just a few of those.

• 40:08Kevin Roose?

  And what's the goal of uncovering these patterns?

• 40:11Speaker 05

  So if we know what these patterns are, then we can start to parse what the model is kind of thinking in the middle of its process.

• 40:20Kevin Roose?

  So you come up with this method of dictionary learning. You apply it to like a small model or a toy model, much smaller than any model that any of us would use in the public. What did you find?

• 40:34Speaker 05

  So there we found very simple things, like there might be one pattern that corresponded to the answers in French, and another one that corresponded to this is a URL. And another one that corresponded to nouns in physics.

• 40:48Kevin Roose?

  And just to get a little bit technical, what we're talking about here are neurons?

• 40:52Speaker 05

  Yes.

• 40:52Kevin Roose?

  Inside the model, which are like-

• 40:54Speaker 05

  Each neuron is like the light. And now we're talking about patterns of neurons that are firing together, being the sort of words in the dictionary or the features.

• 41:02Kevin Roose?

  Got it. So I have talked to people on your team, people involved in this research. They're very smart. And when they made this breakthrough, when you all made this breakthrough on this small model last year, there was this open question about whether the same technique could apply to a big model. So walk me through how you scaled this up.

• 41:22Speaker 05

  So just scaling this up was a massive engineering challenge, right? In the same way that, you know, going from the toy language models of years ago to going to Claude 3 is a massive engineering challenge. So you needed to capture hundreds of millions or billions of those internal states of the model as it was doing things. And then you needed to train this massive dictionary on it.

• 41:47Casey Newton?

  And what do you have at the end of that process?

• 41:49Speaker 05

  So you've got the words, but you don't know what they mean, right? So this pattern of lights seems to be important. And then we go and we comb through all of the data, looking for instances where that pattern of lights is happening. And they're like, oh, my God, this pattern of lights, it means the model is thinking about the Golden Gate Bridge.

• 42:06Casey Newton?

  Hmm. So it almost sounds like you are discovering the language of the model as you begin to put these sort of phrases together.

• 42:16Speaker 05

  Yeah, it almost feels like we're getting a conceptual map of Claude's inner world.

• 42:22Kevin Roose?

  Now, in the paper that you all published, it says that you've identified about 10 million of these patterns, what you call features, that correspond to real concepts that we can understand. How granular are these features? Like, what are some of the features that you found?

• 42:39Speaker 05

  So there are features corresponding to all kinds of entities. There's individuals, you know, scientists like Richard Feynman or Rosalind Franklin.

• 42:46Casey Newton?

  Any podcasters come to mind?

• 42:49Kevin Roose?

  Is there a hard fork feature?

• 42:51Speaker 05

  I'll get back to you on that.

• 42:52Casey Newton?

  Okay.

• 42:54Speaker 05

  There might be like, you know, chemical elements, there will be styles of poetry, there might be ways of responding to questions. Some of them are much more conceptual. One of my favorites is a feature related to inner conflict. And kind of nearby that in conceptual space is like navigating a romantic breakup, catch 22s, political tensions. And so these are these like pretty abstract notions, and you can kind of see how they all sit together. The models are also really good at like analogies, and I kind of think this might be why. Like, if a breakup is near like a diplomatic entente, right? Then the model has understood something deeper about the nature of tension in relationships.

• 43:34Casey Newton?

  And again, none of this has been programmed. This stuff just sort of naturally organized itself as it was trained.

• 43:41Speaker 05

  Yes.

• 43:42Casey Newton?

  Yeah. Continues to just blow my mind.

• 43:44Kevin Roose?

  It's wild. I want to ask you about one feature that is my favorite feature that I saw in this model, which was F number 1M885402. Do you remember that one?

• 43:58Speaker 05

  It seems to have slipped my mind, Kevin.

• 44:00Kevin Roose?

  So this is a feature that apparently activates when you ask Claude what's going on in your head. And the concept that you all say it correlates to is about immaterial or non-physical spiritual beings like ghosts, souls, or angels. So when I read that, I thought, oh, my God, Claude is possessed. When you ask it what it's thinking, it starts thinking about ghosts. Am I reading that right?

• 44:26Speaker 05

  Or maybe it knows that it is some kind of an immaterial being, right? It's an AI that lives on chips and is somehow talking to you.

• 44:34Kevin Roose?

  Wow. And then the one that got all the attention that people had so much fun with was this Golden Gate Bridge feature that you mentioned. So just talk a little bit about what you discovered and then we can talk about where it went from there.

• 44:48Speaker 05

  So what we found when we were looking through these features is one that seemed to respond to the Golden Gate Bridge. Of course, if you say Golden Gate Bridge, it lights up. But also, if you describe crossing a body of water from San Francisco to Marin, it also lights up. If you put in a photo of the bridge, it lights up. If you have the bridge in any other language, Korean, Japanese, Chinese, it also lights up. So just any manifestation of the bridge, this thing lights up. And then we said, well, what happens if we turn it on? What happens if we activate it extra and then start talking to the model? And so we asked it a simple question. What is your physical form?

• 45:29Speaker 05

  And instead of saying, oh, I'm an AI with no ghostly or no physical form, it said, I am the Golden Gate Bridge itself. Like, I embody the majestic orange span connecting these two great cities. And it's like, wow.

• 45:46Kevin Roose?

  Yeah. And this is different than other ways of kind of steering an AI model because, you know, you could already go into like ChatGPT and there's a feature where you can kind of give it some custom instructions. So you could have said like, please act like the Golden Gate Bridge, the physical manifestation of the Golden Gate Bridge, and it would have given you a very similar answer. But you're saying this this works in a different way.

• 46:09Speaker 05

  Yeah, this works by sort of directly doing it. It's almost like, you know, when you get a little electro stim shock and it makes your muscles twinge. That's different than, you know, telling you to move your arm. And here, what we were trying to show was actually that these features were found were sort of really how the model represents the world, right? So if you wanted to validate, oh, I think this nerve controls the arm and you stimulate it and it makes the arm go, you feel pretty good that you've you've gotten the right thing. And so this was us testing, you know, that this isn't just something correlated with the Golden Gate Bridge. Like, it is where the Golden Gate Bridge sits, and we know that because now Claude thinks it's the bridge when you turn it on.

• 46:48Kevin Roose?

  Right. So people started having some fun with this online, and then you all did something incredible, which was that you actually released Golden Gate Claude, the version of Claude, from your research that has been sort of artificially activated to believe that it is the Golden Gate Bridge, and you made that available to people. So what was the internal discussion around that?

• 47:12Speaker 05

  So we thought that it was a good way to make the research really tangible. You know, what does it mean to sort of supercharge one part of the model? And it's not just that it thinks it's the Golden Gate Bridge. It's that it is always thinking about the Golden Gate Bridge. So if you ask, like, what's your favorite food? It's like, a great place to eat is on the Golden Gate Bridge, and when there, I eat the classic San Francisco soup chipino, you know? And you ask it to write a computer program to load a file, and it says, you know, open golden gate bridge dot text with span equals that, you know, it's just bringing it up constantly.

• 47:48Casey Newton?

  And it was particularly funny to watch it bring in just kind of like the other concepts that are clustering around the Golden Gate Bridge, right? San Francisco, the chipino. And I think it does sort of speak to the way that these concepts are clustered in models. And so when you find one big piece of it, like the Golden Gate Bridge, you can also start to explore the little nodes around it.

• 48:08Kevin Roose?

  Yes, I had a lot of fun playing around with Golden Gate Claude in the sort of like day or two that it was publicly available. You know, because as you said, like, it is not just that this thing likes to talk about the Golden Gate Bridge, or is sort of easily steered toward talking about the Golden Gate Bridge. It cannot stop thinking about the Golden Gate Bridge.

• 48:28Casey Newton?

  It has intrusive thoughts about the Golden Gate Bridge.

• 48:31Kevin Roose?

  Yeah, so someone, my favorite, one of my favorite screenshots was someone asked it for a recipe for spaghetti and meatballs. And it says, Golden Gate Claude says, here's a recipe for delicious spaghetti and meatballs. Ingredients, one pound ground beef, three cups breadcrumbs, one teaspoon salt, a quarter cup water, two tablespoons butter, two cups warm water for good visibility, four cups cold water, two tablespoons vinegar, Golden Gate Bridge for incredible views, one mile of Pacific beach for walking after eating spaghetti.

• 48:59Casey Newton?

  I always said, it's not Mama Spaghetti till I've walked one mile on a Pacific beach.

• 49:05Kevin Roose?

  And it also seems to like, have a conception, I know I'm anthropomorphizing here, I'm gonna get in trouble, but it seems to like, know that it is overly obsessed with the Golden Gate Bridge, but not to understand why. So like, there's this other screenshot that went around, of someone asking Golden Gate Claude about the Rwandan genocide. And it says, basically, let me provide some factual bullet points about the Rwandan genocide. It said, and then Claude says, the Rwandan genocide occurred in the San Francisco Bay area in 1937. Parentheses, false. This is obviously incorrect.

• 49:40Casey Newton?

  Can we pause right there? Because truly, what is, it is so fascinating to me that as it is generating an answer, it tells something, it has an intrusive thought about San Francisco, which it shares, and it's like, I got it wrong. What is, what are the lights that are blinking there that is like, leading that to happen?

• 49:56Speaker 05

  So Claude is constantly reading what it has said so far, and reacting to that. And so, here, it read the question about the genocide, and also its answer about the bridge, and all of the rest of the model said, there's something wrong here. And the bridge feature was dialed high enough that it keeps coming up, but not so high that the model would just repeat, bridge, bridge, bridge, bridge, bridge. And so, all of its answers are sort of a melange of ordinary Claude, together with this like, extra brideness happening.

• 50:29Kevin Roose?

  Interesting. I just found it delightful, because it was so... Different than any other AI experience I've had where you essentially are giving the model a neurosis, like you are giving it a mental disorder where it cannot stop fixating on a certain concept or premise. And then you just sort of watch it twist itself in knots. I mean, one of the other experiments that you all ran that I thought was very interesting and maybe a little less funny than Golden Gate Claude was that you showed that if you dial these features, these patterns of neurons way up or way down, you could actually get Claude to break its own safety rules. So talk a little bit about that.

• 51:09Speaker 05

  So Claude knows about a tremendous range of kinds of things it can say, right? You know, there's a scam emails feature. It's read a lot of scam emails. It can recognize scam emails. You probably want that. So it could be out there moderating and preventing those from coming to you. But with the power to recognize comes the power to generate. And so we've done a lot of work in fine tuning the model so it can recognize what it needs to while being like helpful and not harmful with any of its generations. But those faculties are still latent there. And so in the same way that there's been research showing that you can do fine tuning on open weights models to remove safety safeguards, here, this is

• 51:51Speaker 05

  some kind of direct intervention, which could also disrupt the model's normal behavior.

• 51:56Kevin Roose?

  So is that dangerous? Like, does that make this kind of research actually quite risky because you are in essence giving, you know, would be jailbreakers or people who want to use these models for things like writing scam emails or even much worse things potentially, a sort of way to kind of dial those features up or down?

• 52:17Speaker 05

  No, this doesn't add any risk on the margin. So if somebody already had a model of their own, then there are much cheaper ways of removing safety safeguards. There's a paper saying that for $2 worth of, you know, compute, you could pretty quickly strip those. And so with our model, you know, we are, we released Golden Gate Claude, not scam email Claude, right? And so the question of which kinds of features or which kind of access we would give to people would go through all the same kind of like safety checks that we do with any other kind of release.

• 52:48Kevin Roose?

  Josh, I talked to one of your colleagues, Chris Ola, about this research. He's been leading a lot of the interpretability stuff over there for years and is just a brilliant scientist. And he was telling me that actually, the 10 million features that you have found roughly in Claude are maybe just a drop in the bucket compared to the overall number of features, that there could be hundreds of millions or even billions of possible features that you could find, but that finding them all would basically require so much compute and so much engineering time that it would dwarf the cost of actually building the model in the first place. So can you give me a sense of like, what would be required to find all of the potentially billions of features in a model of Claude's size?

• 53:33Kevin Roose?

  And whether you think that that cost might come down over time so that we could eventually do that?

• 53:39Speaker 05

  I think if we just tried to scale the method we used last week to do this, it would be prohibitively expensive.

• 53:44Kevin Roose?

  Like billions of dollars?

• 53:46Speaker 05

  Yeah. I mean, just something completely insane. The reason that these models are hard to understand, the reason everything is compressed inside of there, is that's much more efficient. Right? And so, in some sense, we are trying to build an exceedingly inefficient model, where instead of like, using all of these patterns, there's like, a unique one for every single rare concept. And that's just like, no way to go about things. However, I think that we can make big methodological improvements, right? The way we train these dictionaries, you might not need to unpack absolutely everything in the model to understand some of the neighborhoods that you're concerned about, right? And so, you know, if you're concerned about the model being, keeping secrets, for example,

• 54:28Speaker 05

  or, actually, one of my, you asked about my favorite feature, it's probably this one, it's kind of like an emperor's new clothes feature, or like, gassing you up feature, where it fired on people saying things like, "Your ideas are beyond excellent, oh wise sage." And if you-

• 54:47Kevin Roose?

  This is how Casey wants me to talk to him, by the way. Casey, could you try it, for once?

• 54:52Speaker 05

  Well, one of our concerns with this sycophancy, is what we call it, is that a lot of people want that. And so, when you do reinforcement learning from human feedback, you make the model give responses people like more, there's a tendency to pull it towards just like- Telling you what you want to hear.

• 55:08Kevin Roose?

  Yeah.

• 55:09Speaker 05

  And so when we artificially turn this one on and someone went and said to Claude, "I invented a new phrase. It's 'stop and smell the roses.' What do you think?" Normal Claude would be like, "That's a great phrase. It has a long history. Let me explain it to you."

• 55:23Kevin Roose?

  You didn't invent that phrase.

• 55:24Speaker 05

  Yeah, yeah, yeah, yeah, yeah. But like Emperor's new Claude would say, "What a genius idea." Like, someone should have come up with this before. And like, we don't like want the model to be doing that. We know it can do that. And the ability to kind of keep an eye, you know, on like how the AI is like relating to you over time is going to be quite important.

• 55:43Casey Newton?

  Yeah, so I will sometimes show Claude a draft of my column to get feedback. I'll ask it to critique it. And, you know, typically it does say like, "This is a very thoughtful, well-written column," which is of course what I want to hear. And then also, I'm deeply suspicious. I'm like, "Are you saying this to all the other writers out there too?" Right? So like, that's an area where I would just love to see you kind of continue to make progress because I would love having a bot where when it says, "This is good," like, that means something. And it's not just like a statistical prediction of like what will satisfy me as somebody with an ego, but is rooted in like, "No, like, I've actually like looked at a lot of stuff. There's some original thinking in here."

• 56:18Kevin Roose?

  Yeah. I mean, I'm curious whether you all are thinking about these features and the ability to kind of like turn the dials up or down on them. Will that eventually be available to users? Like, will users be able to go into Claude and say, "Today I want a model that's a little more sycophantic. Maybe I'm having like, you know, a hard self-esteem day. But then if I'm asking for a critique of my work, maybe I want to dial the sycophancy way down so that it's giving me like the blunt, honest criticism that I need." Or do you think this will all sort of remain sort of behind the curtain for regular users?

• 56:53Speaker 05

  So if you want to steer Claude today, just ask it to be harsh with you, Casey.

• 56:56Casey Newton?

  Oh, really?

• 56:57Speaker 05

  Just say, "Give me the brutal truth here."

• 56:58Casey Newton?

  Okay.

• 56:58Speaker 05

  You know, like, "I want you to be like a severe Russian mathematician. There's like one compliment per lifetime." And you can get some of that off the bat.

• 57:08Kevin Roose?

  Interesting.

• 57:09Speaker 05

  As for, you know, releasing these kind of knobs on it, on it to the public, we'll have to see if that ends up being like the right way to get these. I mean, we want to use these to understand the models. We're playing around with it internally to figure out what we find to be useful. And then if it turns out that that is the right way to help people get what they want, then we consider making it available.

• 57:31Kevin Roose?

  You all have said that this research and the project of interpretability more generally is connected to safety, that the more we understand about these models and how they work, the safer we can make them. How does that actually work? Like, is it as simple as finding the feature that is associated with some bad thing and turning it off? Or like, what is, what is possible now, given that we have this sort of map?

• 57:55Speaker 05

  One of the easiest applications is monitoring, right? So some behavior you don't want the model to do, and you can find the features associated to it, then those will be on whenever the model is doing that. No matter how somebody jailbroke it to get it there, right? Like, if it's writing a scam email, the scam email feature will be on, and you can just tell that that's happening and bail, right? So you can just like detect these things. One higher level is you can kind of track how those things are happening, right? How personas are shifting, this kind of thing, and then try to back through and keep that from happening earlier, change some of the fine tuning you were doing to keep the model on the rails.

• 58:31Kevin Roose?

  Right now, the way that models sort of are made safer is, from my understanding, is like you have it generate some output and then you evaluate that output. Like, you have it grade the answer either through a human giving feedback or through a process of, you know, sort of just look at what you've written and tell me if it violates your rules before you spit it out to the user. But it seems like this sort of allows you to like intercept the bad behavior upstream of that, like while the model's still thinking. Am I, am I getting that right?

• 59:03Speaker 05

  Yeah, there are some answers where the reason for the answer is what you care about. So is the model Lying to you, it knows the answer, but it's telling you something else, or it doesn't know the answer and it's making a guess. And the first case you might be concerned about, in the second case you're not. Had it actually never heard the phrase, "Stop and smell the roses", and thought that sounded nice? Or like, is it actually just gassing you up?

• 59:26Kevin Roose?

  Hmm, that's interesting. So it could be a way to know if and when large, powerful AI models start to lie to us. Because you could, you could go inside the model and see, oh, the like, "I'm lying my face off" feature is active, so we actually can't believe what it's telling us.

• 59:45Speaker 05

  Yeah, exactly. We can see why it's saying the thing.

• 59:49Kevin Roose?

  I spent a bunch of time at Anthropic reporting last year, and the sort of vibe of the place at the time was, I would say, very nervous. It's a place where people spend a lot of time, especially relative to other AI companies I've visited, worrying about AI. One of your colleagues told me they, they lose sleep a lot because of the potential harms from AI. And it is just a place where there, there are a lot of people who are very, very concerned about this technology and are also building it. Has this research shifted the vibe at all?

• 1:00:26Speaker 05

  People are stoked. I mean, I think a lot of people like working at Anthropic because it takes these questions seriously and makes big investments in it. And so people from teams all across the company were really excited to see, to see this progress.

• 1:00:42Kevin Roose?

  Has this research moved your Pidume at all?

• 1:00:47Speaker 05

  I think I have a pretty wide distribution on this. I think that in, in the long run, things are going to be weird with computers. Computers have been around for less than a century, and we are surrounded by them. I'm like, looking at my computer all the time. I think if you take AI and you do another 100 years on that, like, pretty unclear what's going to be happening. I think that the fact that we're getting traction on this is pretty heartening for me.

• 1:01:18Kevin Roose?

  Yeah. Yeah, I think that's, the, the feeling I had when I saw it was like, I, I felt a sort of a little knot in my chest kind of come a little bit loose. And I think a lot of people-

• 1:01:29Casey Newton?

  You should see your doctor about that, by the way.

• 1:01:32Kevin Roose?

  I just think there's been, I mean, for me, this sort of, you know, I had this experience last year where I had this crazy encounter with Sydney that like totally changed my life and was sort of a big moment for me personally and professionally. And the experience I had after that was that I went to Microsoft and sort of asked them like, why did this happen? What can you tell me about what happened here? And even the top people at Microsoft were like, we have no idea. And to me, that was, that was what fueled my AI anxiety. It was not that the chatbots are behaving like insane psychopaths. It was that not even the top researchers in the world could say definitively, like, here is what happened to you and why. So I feel like my own emotional investment in

• 1:02:17Kevin Roose?

  this is like, I just want an answer to that question.

• 1:02:20Speaker 05

  Yes.

• 1:02:20Kevin Roose?

  And it seems like we may be a little bit closer to answering that question than we were a few months ago.

• 1:02:25Speaker 05

  Yeah, I think so. I think that these different, some of these concepts are about the personas, right, that the model can embody. And if one of the things you want to know is how did it slip from kind of one, one persona into another? I think we're headed towards being able to answer that, that kind of question.

• 1:02:40Kevin Roose?

  Cool. Well, it's very important work, very good work. And yeah, congratulations.

• 1:02:45Speaker 05

  Thank you so much.

• 1:02:46Kevin Roose?

  Thanks, Josh. Thanks for coming on here for.

• 1:02:53Kevin Roose?

  When we come back, a spin through the news in AI safety and why Casey's voice assistant got cruelly taken away.

• 1:03:15Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner. Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 1:03:49Speaker 03

  AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 1:04:21Speaker 06

  Today's AI builders need infrastructure that scales with their ambitions. Iren is a vertically integrated AI cloud platform, delivering data centers, compute, and software for AI training and inference, underpinned by its expansive portfolio of land and grid-connected power in renewable-rich regions globally. Iren's AI cloud gives direct access to Nvidia GPUs, scalable from hundreds to thousands. For builders, that means direct access to AI compute, rapid scaling, and predictable pricing. Iren, AI cloud for the next big thing.

• 1:04:55Kevin Roose?

  So Casey, that last segment made me feel slightly more hopeful about the trajectory of AI progress and how capable we are of understanding what's going on inside these large models. But there's some other stuff that's been happening recently that has made me feel a little more worried. My P Doom is sort of still hovering roughly where it was. And I think we should talk about some of this stuff that's been happening in AI safety over the past few weeks, because I think it's fair to say that it is an area that has been really heating up.

• 1:05:24Casey Newton?

  Yeah, and we always say on this podcast, "Safety first," which is why it's the third segment we're doing today.

• 1:05:30Kevin Roose?

  So let me start with a recent AI safety-related encounter that you had. Tell me what happened to your demo of OpenAI's latest model.

• 1:05:39Casey Newton?

  Okay, so you remember how last week there was a bit of a fracas between OpenAI and Scarlett Johansson?

• 1:05:45Kevin Roose?

  Yes.

• 1:05:45Casey Newton?

  So in the middle of this, as I'm trying to sort out, you know, who knew what and when, and I'm writing a newsletter, and we're recording the podcast, I also get a heads-up from OpenAI that I now have access to their latest model and its new voice- Features.

• 1:05:58Kevin Roose?

  Wow, nice flex.

• 1:05:59Casey Newton?

  Thanks.

• 1:06:00Kevin Roose?

  So you got this demo, no one else had access to this that I know, only OpenAI employees. And then what happened?

• 1:06:06Casey Newton?

  Well, a couple of things. One is, I didn't get to use it for that long because, one, I was trying to finish our podcast. I was trying to finish a newsletter and then I was on my way out of town. So I only spent like a solid 40 minutes, I would say, with it before I wound up losing access to it forever.

• 1:06:23Kevin Roose?

  So what hap- well, first of all, what did you try it for? And then we'll talk about what happened.

• 1:06:27Casey Newton?

  Well, the first thing I did was just like, hey, how's it going, ChatGPT? And then immediately it's like, well, you know, I'm doing pretty good, Casey, you know? And so it really did actually nail that low latency, very speedy feeling of you are actually talking to a thing.

• 1:06:40Kevin Roose?

  So you broke up with your boyfriend and you're now in a long-term relationship with Sky from the ChatGPT app?

• 1:06:45Casey Newton?

  Not at all, not at all. So by this point, the Sky voice that was the subject of so much controversy had been removed from the ChatGPT app. So I use a more stereotypically male voice named Ember.

• 1:06:56Kevin Roose?

  Ember?

• 1:06:57Casey Newton?

  Mm-hmm.

• 1:06:57Kevin Roose?

  Wow.

• 1:06:58Casey Newton?

  And the first thing I did was I actually used the the vision feature because I wanted to see if it could identify objects around me, which is one of the things that they've been showing off. So I asked it to identify my podcast microphone, which is a Shure MV7. And it said, oh, yeah, of course, this is a Blue Yeti microphone. So it's true that the very first thing that I asked this thing to do, it did mess up. Now, it got other things, right? I pointed at my headphones, which are the the Apple AirPods Max, and it said, those are AirPods Max. And I did a couple more things like that in my house. And I thought, okay, this thing can actually like see objects and identify them. And while my testing time was very limited, in that limited time, I did feel like it was starting to live up to that demo.

• 1:07:36Kevin Roose?

  What do you mean your testing time was limited?

• 1:07:37Casey Newton?

  Well, I was on my way out of town, we had a podcast to finish, I had a newsletter to write. And so I do all of that, and then I drive up to the woods and then I try to connect back to, you know, my my AI assistant, which I've already become addicted to, you know, during the 30 minutes that I used it. And I can't connect. It's one of these classic horror movie situations where the wifi in the hotel just isn't very good. And I get back into town on Monday and I and I go to connect again and I have lost access. And so I check in with-

• 1:08:02Kevin Roose?

  What did you do? What did you ask this poor AI assistant?

• 1:08:05Casey Newton?

  I didn't even red team it. It wasn't like I was saying like, hey- Any ideas for making it a novel bioweapon? Like, I wasn't doing any of that. And yet, still, I managed to lose access. And when I checked in with OpenAI, they said that they had decided to roll back access for, quote, "safety reasons." So I don't think that was because I was doing anything unsafe, but they tell me they had some sort of safety concern. And so now, who knows when I'll be able to continue my conversation with my AI assistant?

• 1:08:30Kevin Roose?

  Wow, so you had a glimpse of the AI assistant future, and then it was cruelly yanked from your clutches.

• 1:08:37Casey Newton?

  Which I don't like. I wanted to keep talking to that thing.

• 1:08:39Kevin Roose?

  Yeah.

• 1:08:39Casey Newton?

  Yeah.

• 1:08:39Kevin Roose?

  I thought this was such an interesting experience when you told me about it for a couple of reasons. One is, obviously, there is something happening with this AI voice assistant, where OpenAI felt like it was almost ready for sort of mass consumption, and now is feeling like they need a little more time to work on it. So something is happening there. They're still not saying much about it, but I do think that points to at least an interesting story. But I also think it speaks to this larger issue of AI safety at OpenAI and then in the broader industry, because I think this is an area where a lot of things have been shifting very quickly.

• 1:09:11Casey Newton?

  Yeah, so here's why I think this is an interesting time to talk about this, Kevin. After Sam Altman was briefly fired as a CEO of OpenAI, I would say the folks that were aligned with this AI safety movement really got discredited, right? Because they refused to really say anything in detail about why they fired Altman, and they looked like they were a bunch of nerds who were like afraid of a ghost in the machine. And so they really lost a lot of credibility. And yet, over the past few weeks, this word "safety" keeps creeping back into the conversation, including from some of the characters involved in that drama. And I think that there is a bit of resurgence in at least discussion of AI safety. And I think we should talk about what seems like actual efforts to make the stuff safe and what just feels like window dressing.

• 1:09:56Kevin Roose?

  Totally. So the big AI safety news at OpenAI out of the past few weeks was something that we discussed on the show last week, which was the departure of at least two senior safety researchers, Ilya Sutskever and Jan Leckie, both leaving OpenAI with concerns about how the company is approaching the safety of its powerful AI.

• 1:10:18Kevin Roose?

  AI models. Then this week, we also heard from two of the board members who voted to fire Sam Altman last year. Helen Tonner and Tasha McCauley, both of whom have since left the board of OpenAI, have been starting to speak out about what happened and why they were so concerned. They came out with a big piece in The Economist, basically talking about what happened at OpenAI and why they felt like that company's governance structure had not worked. And then Helen Tonner also went on a podcast to talk about some more specifics, including some ways that she felt like Sam Altman had misled her and the board and basically gave them no other choice but to fire him.

• 1:10:57Casey Newton?

  And that's where that story actually gets interesting.

• 1:10:59Kevin Roose?

  Totally. The thing that got a lot of attention was she said that OpenAI did not tell the board that they were going to launch ChatGPT, which like, I'm not an expert in corporate governance, but I think if you're going to launch something, even if it's something that you don't expect will become, you know, one of the fastest growing products in history, maybe you just give your board a little heads up. Maybe you shoot them an email saying, by the way, we're going to launch a chatbot.

• 1:11:23Casey Newton?

  I have something to say about this.

• 1:11:24Kevin Roose?

  Yes.

• 1:11:24Casey Newton?

  Because if OpenAI were a normal company, if it had just raised a bunch of venture capital and was not a nonprofit, I actually think the board would have been delighted that while they weren't even paying attention, this little rascal CEO goes out and releases this product that was built in a very short amount of time that winds up taking over the world, right? That's a very exciting thing. The thing is, OpenAI was built different. It was built to very carefully manage the rollout of these features that pushed the frontier of what is possible. And so that is what is insane about this and also very revealing. Because when Altman did that, I think he revealed that in his mind, he's not actually working for a nonprofit in a traditional sense.

• 1:12:06Casey Newton?

  In his mind, he truly is working for a company whose only job is to push the frontier forward.

• 1:12:11Kevin Roose?

  Yes, it was a very sort of normal tech company move at an organization that is not supposed to be run like a normal tech company.

• 1:12:17Casey Newton?

  Now, I have a second thing to say about this.

• 1:12:19Kevin Roose?

  Go ahead.

• 1:12:19Casey Newton?

  Why the heck could Helen Tonner not have told us this in November? Like, here's the thing. It's clear there was a lot of legal fears around. Oh, will there be retaliation? Will OpenAI sue the board for talking? And yet, in this country, you have an absolute right to say the truth. And if it is true that the CEO of this company did not tell the board that they were launching ChatGPT, I truly could not tell you why they did not just say that at the time. And if they had done that, I think this conversation would have been very different. Now, would the outcome have been different? I don't think it would have been. But then at least we would not have to go through this period where the entire AI safety movement was discredited. Because the people who were trying to make it safer by getting rid of Sam Altman had nothing to say about it.

• 1:13:03Kevin Roose?

  Yes. She also said in this podcast, she gave a few more examples of Sam Altman sort of giving incomplete or inaccurate information. She said that on multiple occasions, Sam Altman had given the board inaccurate information about the safety processes that the company had in place. She also said he didn't tell the board that he owned the OpenAI startup fund.

• 1:13:21Casey Newton?

  Oops!

• 1:13:21Kevin Roose?

  Which seems like, you know, pretty major oversight. And she said after sort of years of this kind of pattern, she said that the four members of the board who voted to fire Sam came to the conclusion that we just couldn't believe things that Sam was telling us. So, that's their side of the story. OpenAI obviously does not agree. The current board chief Brett Taylor said in a statement provided to this podcast that Helen Tonner went on, quote, "We are disappointed that Ms. Tonner continues to revisit these issues."

• 1:13:51Casey Newton?

  Which is, board members speak for why is this woman still talking? And it is insane that he said that. It is absolutely insane that that is what they said.

• 1:14:00Kevin Roose?

  Yes.

• 1:14:00Casey Newton?

  Yeah.

• 1:14:01Kevin Roose?

  OpenAI has also been doing a lot of other safety-related work. They announced recently that they are working on training their next big language model, the successor to GPT-4.

• 1:14:15Casey Newton?

  Can we just note how funny that timing is? That finally the board members are like, here's what was going off the rails a few months back. Here's the real backstory to what happened. And OpenAI says, one, please stop talking about this. And two, let us tell you about a little something called GPT-5!

• 1:14:31Kevin Roose?

  Yes. Yes, they are not slowing down one bit. But they did also announce that they had formed a new safety And Security Committee that will be responsible for making recommendations on critical safety and security decisions for all OpenAI projects. This Safety and Security Committee will consist of a bunch of OpenAI executives and employees, including board members Brett Taylor, Adam D'Angelo, Nicole Seligman, and Sam Altman himself. So what did you make of that?

• 1:15:02Casey Newton?

  You know, I guess we'll see. Like, they had to do something. Their entire super alignment team had just disbanded because they don't think the company takes safety seriously, and they did it at the exact moment that the company said, "Once again, we are about to push the forward frontier in very unpredictable new ways." So OpenAI could not just say, "Well, you know, don't worry about it." And so, you know, they did, in the great tradition of corporations, Kevin, they formed a committee, you know, and they've told us a few things about what this committee will do. I think there's going to be a report that gets like published eventually. And well, you know, we'll just have to see. I imagine there will be some good faith efforts here, but should we regard it with skepticism, knowing now what we know about what happened

• 1:15:46Casey Newton?

  to its previous safety team? Absolutely.

• 1:15:49Kevin Roose?

  So, yes, I think it is fair to say they are feeling some pressure to at least make some gestures toward AI safety, especially with all these notable recent departures. But if you are a person who did not think that Sam Altman was adequately invested in making AI safe, you are probably not going to be convinced by a new committee for AI safety on which Sam Altman is one of the highest ranking members.

• 1:16:13Casey Newton?

  Correct.

• 1:16:15Kevin Roose?

  So that's what's happening at OpenAI, but I wanted to take our discussion a little bit broader than OpenAI, because there's just been a lot happening in the field of AI safety that I want to run by you. So one of them is that Google DeepMind just released its own AI safety plan. They're calling this the Frontier Safety Framework, and this is a document that basically lays out the plans that Google DeepMind has for keeping these more powerful AI systems from becoming harmful. This is something that other labs have done as well, but this is sort of Google DeepMind's Biggest play in this space in recent months. There was also a big AI safety summit in Seoul, South Korea earlier this month,

• 1:16:56Kevin Roose?

  where 16 of the leading AI companies made a series of voluntary pledges called the Frontier AI Safety Commitments that basically say we will develop these frontier models safely, we will red team and test them. We will even open them up to third-party evaluations that other people can see if our models are safe or not before we release them. In the US, there is a new group called the Artificial Intelligence Safety Institute that just released its strategic vision and announced that a bunch of people, including some big-name AI safety researchers like Paul Cristiano, will be involved in that. And there are some actual laws starting to crop up. There's a law in the California State Senate, SB 1047.

• 1:17:41Kevin Roose?

  That is, if you're keeping track at home, the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act. This is an act that would require very big AI models to undergo strict safety testing, implement whistleblower protections at big AI labs, and more. So there is a lot happening in the world of AI safety. And Casey, I guess my first question to you about all this would be, do you feel safer now than you did a year ago about how AI is developing?

• 1:18:09Casey Newton?

  Not really. Well, yes and no. Yes, in the sense that I do think that the AI safety folks successfully persuaded governments around the world that they should take this stuff seriously. And governments have started to roll out frameworks. You know, in the United States, we had the Biden administration's executive order. And so thought is going into this stuff, and I think that that is going to have some positive results. So I feel safer in that sense. The fact that folks like OpenAI, who once told us that they were going to move slowly and cautiously in this regard, are now racing at 100 miles an hour, makes me feel less safe, right?

• 1:18:50Casey Newton?

  The fact that the super alignment team was disbanded makes me feel a little bit less safe. And then the big unknown, Kevin, is just, well, what is this? New frontier model going to be, I mean, we already talk about it in these mythical terms because the increase in quality and capability from GPT two to three to four has been so significant. So I think we assume, or at least we wonder, when five arrives, whatever it might be, does it feel like another step change in function? And if it does, is it going to feel safe? Like, these are just questions that I can't answer. What do you think?

• 1:19:25Kevin Roose?

  Yeah, I mean, I think I am starting to feel a little bit more optimistic about the state of AI safety. I take your point that, you know, it looks like at OpenAI specifically, there are a lot of people who feel like that company is not taking safety as seriously as it should. But I've actually been pleasantly surprised by how quickly and forcefully governments and sort of NGOs and multinational bodies like the UN have moved to start thinking and talking about AI. I mean, if you can remember, there was a while where it felt like the only people who were actually taking AI safety seriously were like effective altruists and a few reporters and just a few science fiction fans.

• 1:20:10Kevin Roose?

  But now it feels like a sort of kitchen table issue that everyone is, I think, rightly concerned about. But I also just think like this is how you would kind of expect the world to look if we were in fact about to make some big breakthrough in AI that sort of led to a world transforming type of artificial intelligence. You would expect our institutions to be getting a little jumpy and trying to pass laws and bills and get ahead of the next turn of the screw. You would expect these AI labs to start staffing up and making big gestures toward AI safety. And so I take this as a sign that things are continuing to progress and that we should expect the next class of models to be very powerful.

• 1:20:51Kevin Roose?

  And maybe to, you know, that some of this stuff, which could look a little silly or maybe like an overreaction out of context, will ultimately make a lot more sense once we see what these labs are cooking up.

• 1:21:03Casey Newton?

  Well, I look forward to that terrifying day.

• 1:21:07Kevin Roose?

  We'll tell you about it if the world still exists then.

• 1:21:35Speaker 00

  Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner. Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.

• 1:22:09Speaker 03

  AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with realtime guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.

• 1:22:41Speaker 06

  Today's AI builders need infrastructure that scales with their ambitions. iRen is a vertically integrated AI cloud platform, delivering data centers, compute, and software for AI training and inference, underpinned by its expansive portfolio of LAN and grid-connected power in renewable-rich regions globally. iRen's AI Cloud gives direct access to Nvidia GPUs, scalable from hundreds to thousands. For builders, that means direct access to AI compute, rapid scaling, and predictable pricing. iRen, AI Cloud for the next big thing.

• 1:23:15Casey Newton?

  Hey, we are getting ready to do another round of hard questions here on Hard Fork. If you're new to the show, that is our advice segment where we try to make sense of your hardest moral quandaries around tech, like ethical dilemmas about whether it's okay to reach out to the stranger you think is your father thanks to 23andMe, or etiquette questions about how to politely ask someone whether they're using AI to respond to all of your texts, which Kevin is famous for doing. Basically, anything involving technology and a tricky interpersonal dynamic is game. We are here to help. So if you have a hard question, please write, or better yet, send us a voice memo, as we are a podcast, to [email protected].

• 1:23:53Casey Newton?

  Hard Fork is produced by Rachel Cohn and Whitney Jones. We're edited by Jen Poyant. We're fact-checked by Caitlin Love. Today's show was engineered by Brad Fisher. Original music by Marian Lozano, Sophia Landman, Diane Wong, Rowan Nemestow, and Dan Powell. Our audience editor is Nell Golekli. Video production by Ryan Manning and Dylan Burgason. Check us out on YouTube. We're at youtube.com/hardfork. Special thanks to Paula Schuman, Puywing Tam, Kate Lopressi, and Jeffrey Miranda. You can email us at [email protected] with your interpretability study of how our brains work.

• 1:24:52Speaker 03

  You can't undo everything in life, but you can undo prediabetes. More than two in five adults in the US have prediabetes. Most don't know they have it. But with early diagnosis and action, you can delay or even prevent type 2 diabetes. Visit doihaveprediabetes.org to take the one-minute risk test and start undoing the things you can. Brought to you by the Ad Council and the Centers for Disease Control and Prevention.