Pod Insights Hard Fork Social Media In Wartime + Betting on the Future + A.I. Passes the Smell Test About this episode As the Israel-Hamas war broke out, misinformation and fake imagery surged on X, the platform formerly known at Twitter. Can Meta’s Threads fill the real-time news hole that X created? Should it?
Then, Kevin debriefs us on his reporting on Manifold Markets, where Silicon Valley Rationalists bet on the likelihoods of different events.
Plus: The company digitizing smell.
Transcript 13,521 words · 7 speakers
Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Speaker 00 Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.
0:35 Kevin Roose? So Casey, I get a lot of deranged and unintelligible texts from you.
0:39 Casey Newton? You're welcome.
0:41 Kevin Roose? Thanks. And even by those standards, the one that I got from you last night while I was sitting down for dinner was was pretty wild. So this was a text that contained a link to something called Bonk. And your text said the following, "Bonk me on Bonk. My handle is Casey Newton."
1:02 Casey Newton? That's right.
1:02 Kevin Roose? So Casey, why are you asking me to Bonk you? And do you know that I'm happily married?
1:09 Casey Newton? I did hear that, although I was told there was an AI with different ideas. But Bonk is the app that everyone in Silicon Valley is talking about. And by everyone, I mean a few of my fans who showed it to me.
1:22 Kevin Roose? What is this app?
1:23 Casey Newton? OK, it's the best. So do you remember the app Yo from back in the day?
1:26 Kevin Roose? Yes, this was an app like 10 years ago that did only one thing. You could like push a button and it would tell, it would like say Yo to someone.
1:33 Casey Newton? That's exactly right. It was like sort of, you know, when there was all this buzz about these new messaging apps. And then this messaging app came along that could only do one thing.
1:40 Kevin Roose? It was sort of like a parody of an app.
1:42 Casey Newton? It felt that way. And yet it was a sort of parody that raised $1.5 million at a $10 million valuation. OK, 2014 was wild. Love those times. But then, you know, Yo disappears. And then many years go by. And then along comes Bonk. Can I tell you about the Bonk product?
1:58 Kevin Roose? Please.
1:58 Casey Newton? OK, first of all, I tried to get you to download it, but you can't because you don't have iOS 17, which is truly funny, because when you see what this app does, it's not clear to me that it's relying on any of the labor. Sensors and graphical upgrades.
2:11 Kevin Roose? Right. I do not yet have the advanced technology required to run the Bonk app.
2:16 Casey Newton? But if you run, but here's what you do. So you add your friend on Bonk and then you bonk them. How do you bonk them? Well, you just see their username on the screen.
2:23 Kevin Roose? Okay, so you are showing me this app. It has a list of three of your friends on Bonk.
2:28 Casey Newton? That's right.
2:28 Kevin Roose? With a big blue button, one for each friend.
2:30 Casey Newton? That's right. And if you don't have any friends on Bonk, you can bonk the Bonk Bot. There's a Bonk Bot for you to bonk. It's sort of like the Tom of MySpace of Bonk is the Bonk Bot.
2:38 Kevin Roose? No, this is a prank. This is a lady for fools joke.
2:40 Casey Newton? So here's what you do. So here's, here's one of my friends and I can just sort of annoy him right now by bonking him.
2:45 Kevin Roose? Pressing Bonk over and over again.
2:47 Casey Newton? And every time I bonk him, he gets a push notification. So, so ever since I've
2:51 Kevin Roose? This is malware.
2:52 Casey Newton? So ever since I've downloaded this app, I'll look at my phone and I'll have like 46 notifications that just say your friend bonked you.
3:02 Casey Newton? Here's the best part. So as if that were annoying enough, when you type over on this menu, there's a leaderboard and it shows you how many bonks you've sent. Then how many bonks your friends have sent.
3:12 Kevin Roose? You have bonked someone 5,064 times?
3:16 Casey Newton? No. One of the people who introduced me to the app has sent 5,000 bonks this week. I've only sent 438.
3:23 Kevin Roose? Grounds for being arrested.
3:26 Casey Newton? What I love about this so much is it's like, what if Yo was just a tool for like benign harassment?
3:32 Kevin Roose? Yeah, this is a harassment app.
3:33 Casey Newton? Absolutely. Absolutely. But you know, at the same time, it is nice during the day. It just to sort of let someone know that you're thinking about them, even though you have nothing to say. When you have nothing to say, but you still want to say something, that's when you bonk.
3:46 Kevin Roose? Well, I will not be installing this app anywhere on any device that I own because the thought of getting 57,000 bonks a day from you just fills me with dread and terror.
3:56 Casey Newton? And you are somebody who worries you spend too much time on your phone. So I understand how you wouldn't want to install it. But if you are looking to literally waste your life on your phone, and who isn't, please check out Bonk. I, I, I was texting with the creator the other day and he was telling me that he hopes to get it into the App Store soon. Right now it's only available on, on Testlight, which is the sort of the beta testing app on iPhone. But you can find it at bonkbonkbonk.app.
4:19 Kevin Roose? No, no, no. Do not, do not encourage this behavior.
4:22 Casey Newton? If I were the creator, I would create merch, I would create a bumper sticker and it would just say, honk if you're bonking.
4:29 Casey Newton? Yes.
4:30 Kevin Roose? No.
4:31 Casey Newton? Kevin, this, we need more of this.
4:32 Kevin Roose? I cannot endorse this technology.
4:33 Casey Newton? Kevin, think about all the time we spent talking about misinformation, hate speech, harassment, all of it. Find, this is a social app where you can only harass someone by making them want to turn off their phone. And that's a new kind of harassment we haven't seen yet, but it seems very sweet to me.
4:48 Kevin Roose? OK, well, if I ever get another bunk from you, I will be reporting it to the authorities and getting a restraining order. So please, stop.
4:55 Casey Newton? Hey, catch me on bunk.
5:01 Kevin Roose? I'm Kevin Roos. I'm a tech columnist at The New York Times.
5:03 Casey Newton? I'm Casey Newton from Platformer.
5:04 Kevin Roose? And you're listening to Hard Fork.
5:06 Casey Newton? This week, what the war between Israel and Hamas means for the future of social networks. Then Kevin visits a prediction markets conference and we place some bets. And finally, Osmo CEO Alex Wilczko on his quest to build an AI that can smell.
5:38 Kevin Roose? So Casey, we like to have a good time on this podcast, but there has actually been a lot of very serious news this week coming out of the Middle East. I'm talking, of course, about the war between Israel and Hamas. And, you know, this is a tech show. We are not foreign policy experts or experts on conflicts in the Middle East. There are plenty of great podcasts where you can get that kind of information. But I did want to talk about it today because I think there's a really important thing that I've observed in our social media ecosystem, which is that, from what I can tell, there has been no place to actually get good, reliable information about this conflict. Like, I'm sure I, like you and so many other people, opened up X and Instagram
6:20 Kevin Roose? and all of my other apps, trying to sort of make sense and sort out what was going on. And I just couldn't do it. I was just bombarded with stuff that was fake or misleading or suspicious in some way. It was very hard to just get the basic nuts and bolts of what was going on. And I'm wondering if you experienced that, too.
6:38 Casey Newton? Yeah. You know, very much so. I think that one of the ways that we understand what's going on the world right now is by using these social media apps, right? Over the past decade plus, we have trained ourselves when something horrible is happening, our first thought has been to go to Twitter specifically, right? Because for the longest time, Twitter was the best answer to the question of what is going on. Because not only did you get reports from the journalists who were on the ground there, or from the elected officials who were sort of handling the tragedy, but you also have these first-person reports, people who are just average users, who whipped out their phone, they took out a video, they posted a thread. And this is sort of collectively how we made sense of things.
7:22 Casey Newton? But for all of the reasons that we have been talking about on this show over the past year, that world is now in chaos. It has been upended by all of the changes on the consumer internet. And so I think that's sort of what brings this story into our zone, is that one of the main ways we have for understanding events in real time is just changing radically.
7:43 Kevin Roose? Well, and the stakes are just much higher in wartime, right? During normal sort of peacetime, getting bad information on social media might be annoying, it might be misleading, it might even be harmful. But during the fog of war, when there are so many conflicting reports flying around, images, videos, first-person accounts, or things purporting to be first-person accounts, that's when the stakes of conflict and of information get really real. So, there's been some reporting over the past week about X, as Twitter is now called, and many of the viral falsehoods that have been appearing on the site. People are sharing videos from previous conflicts that are years old. People are circulating video game footage and passing it off as wartime footage, fake
8:28 Kevin Roose? images of celebrities taking sides, and actually, you know, things that have caught the attention of regulators in Europe who have warned Elon Musk and X that by hosting this kind of content, it may be in violation of some of the EU's content moderation laws.
8:45 Casey Newton? Yeah, so there's a lot in there. To pull apart. I am somebody who thinks that Elon Musk has been laying the groundwork for this for a long time and that in a lot of ways, this is the logical culmination of a story that began when he decided that he was going to get rid of the old verification system. But I mean, does any of that surprise you at this point, Kevin?
9:08 Kevin Roose? It doesn't because Elon Musk, he's been so clear about the fact that he doesn't like the mainstream media, doesn't want journalists to have sort of special privileges or status on X. And he has been boosting his own set of what he calls citizen journalists, these people who are sort of amateurs, many of which have turned out to be questionable or incorrect. He warned users away from trusting mainstream journalists on the subject, instead promoting two accounts that are known spreaders of misinformation. And you had a really interesting newsletter on this this week where you talked about sort of the failure of X to live up to its past as a real time news platform.
9:47 Casey Newton? Well, you know, again, I think that we have just seen Elon laying the groundwork for this for months, right? One of the first things that he did was he got rid of the old verification badges. And now you can get one of these badges by paying $8 a month. I think probably just as importantly, you can now make money based on the number of views you get if you have paid your $8 a month. And so what are we seeing? If you look at a lot of people who are repurposing video of things that happened long ago or video game footage and trying to pass it off as events in this war, those people have verification badges, right? Which suggests to me they are hoping to get a payout based on the number of impressions they are getting for spreading misinformation. So that's actually quite different from what we used to see in the old days, right? Lord knows,
10:32 Casey Newton? the old Twitter had plenty of problems. It spread plenty of misinformation. There were people doing the exact same thing in those days that are doing it now. The difference is they weren't being paid to do it. And at the same time, Elon has also given these spreaders of misinformation so many powerful tools.
10:47 Kevin Roose? Yeah. And this is sort of an annoyance and a frustration for people in the US. But for people who are actually in the conflict zone, for people who are in Gaza, who are in Israel, this is Potentially a very big problem for them if they cannot use social media or X to figure out what is going on and maybe getting bad or incomplete information about their surroundings or their safety.
11:09 Casey Newton? Yeah, and again, I mean, to some degree, this was always the case. Like, I want to be clear.
11:13 Kevin Roose? Yes, we shouldn't romanticize what old Twitter was like or sort of turn it into this thing that it wasn't. It was never a perfect disseminator of news in breaking situations.
11:23 Casey Newton? Right, and I would say particularly like, you know, five years ago, if you were in the middle of some calamity and you said, like, should I rely on Twitter to understand what to do next? I would say like, well, it should be like an input, but like, you should be trying to guide yourself toward vetted credible sources of information and random tweets are like, maybe not going to be that thing. But at the same time, I think that the utility of these networks actually goes beyond understanding, like, okay, where is sort of the worst violence right now for me to avoid? It also is about understanding the conversation around these things, right? What sort of positions are people staking out? What is the conversation here? And that is actually one of the ways that we make sense of this, right? And that is another way in which X is just not as useful as Twitter once was, because many of
12:05 Casey Newton? the most prominent voices here have either stopped using the platform entirely over the past year, or they're being drowned out by people who paid $8 a month so that their voices float to the top of replies. So just as a means for understanding the conversation around these terrible events, X is just not nearly as useful as it used to be.
12:23 Kevin Roose? So where were you getting news? Where are you getting news about this conflict if it's not on X?
12:28 Casey Newton? Yeah, so I spend much more time these days on three networks. That would be Blue Sky, Mastodon, and Threads, which is Meta's app. And I was having the most interesting experiences on Threads of those. I think Blue Sky is pretty vibrant among a certain crowd, but Threads has really started to accelerate, particularly over the past couple of weeks, in bringing on just some of the most influential voices in media in particular, right? Like, this is where I'm starting to see the reporters show up. Seeing people like your colleague, Sheriff Frankel, for example, had a fascinating story about how Hamas was seeding X with these violent- Videos as sort of part of its terror campaign.
13:11 Casey Newton? I learned about that on threads, right? People who are in the sort of more pundit sphere. They were sharing their analysis on threads. And so, you know, it definitely is not what the old Twitter was for following real time news for reasons that we can get into if you want to. It has started to feel a lot more vibrant and I think has revealed how desperate people are for something to replace what Twitter used to be.
13:35 Kevin Roose? Yeah, so I found this part really interesting because I agree with you that it seems like threads has sort of picked up, at least among the journalists that I know and follow. And at the same time, when we talked to Adam Mosseri, the head of threads earlier this year, he was pretty explicit about not wanting it to be a place where everyone came to get news about important global conflicts. They've been trying to position this as something more like a TikTok for text, something that is going to be fun and entertaining and light and sort of not drag them into the morass of content moderation that they have seen on their other products. So how do you think that tension is playing out right now?
14:17 Casey Newton? Well, it's an interesting question because I'm not sure what the positive vision for threads was when it launched. In fact, I think like many apps, they wanted to just put it in the world to see what people did with it. I think they would have been thrilled if most of the people who showed up wanted to do makeup tutorials and Amazon hauls and productivity tips. But that's just kind of not how it is playing out. And most of the brands that were doing these try hard posts when I first started using the app in July seemed to have disappeared out of the algorithmic feed. And what I'm seeing in their place are a lot of journalists, pundits, maybe the odd elected official, and they're having discussions about what is happening in real time.
15:01 Casey Newton? So I do think that puts Meta at a crossroads to decide, do we want to lean into the direction of where some of our most prominent users are guiding us, or do we want to put our foot down and build in a different direction?
15:16 Kevin Roose? So Adam Mosseri, the head of threads. Has been sort of, seems like he's kind of been backtracking a little bit off this kind of anti-news or sort of, "we're not going to focus on news" statement that he made to us earlier. In a Threads post just the other day, he said, quote, "We're not anti-news. News is clearly already on Threads. People can share news. People can follow accounts that share news. We're not going to get in the way of any either, but we're also not going to amplify news on the platform. To do so would be too risky, given the maturity of the platform, the downsides of overpromising, and the stakes." What do you make of that?
15:52 Casey Newton? I mean, I think that there is a way of interpreting those comments that is less about an ideological opposition to wanting to build an app that is useful for reading journalism, and more is about what will make this product successful, right? Like, if I pulled you aside and I said, "Kevin, I have an incredible new app to learn about all of the horrors in the world in real time." You might say, "I don't know if I need that in my life," right? But if you have this very vague statement about, "Kevin, there's a new app where people are connecting and sharing and exploring their interests," that might just have a broader appeal, right? And so I think that they don't want to throw in the towel yet on having the broader appeal before they at least give it the old college try. What's interesting, though, is that there is now this calamity, which has people clamoring
16:36 Casey Newton? for something very specific in a way that they just haven't seen before. And for what it's worth, I do think that there are a couple of things they could do to at least explore this direction without going all the way in. And maybe it might be interesting to talk about what I think those two directions are.
16:49 Kevin Roose? Yeah, let's talk about that.
16:50 Casey Newton? Okay, so, well, so what does it mean to like lean into news, right? Because I thought that like Misery got kind of undue criticism because he didn't say like, "Don't post news here, journalists go away." He was just like, "If you think we're going to build you a ton of special features, like this is probably not going to happen." What are those features? Well, something that people just clamor for on Threads all the time, one of those things is a trending topics page, which we actually have learned Meta is working on. It seems like that's going to be coming to Threads very soon. People also want hashtags, because that's a way to analyze the news in real time. And then there's more complicated stuff, like, could there be a tweet deck version of Threads where you-
17:26 Kevin Roose? Like a thing for power users who stare at it all day?
17:28 Casey Newton? Or could you create lists of users? So it's like, oh, something terrible is happening in the Middle East. I know I can always get good information from these 20 people. That, to me, is an interesting question of whether Meta pursues that. And then the other question is like, well, let's say they don't do that, right? They say, the hell with it. We don't want to get more involved with news for all the reasons that they've already said. What do they do? What does a more TikTok version of Threads look like? And I, you know, so, so when I was trying to think through that, I just opened up TikTok. And I'm like, well, what's not there? And it's like, well, Threads doesn't have messaging yet. It doesn't really have a lot of creative tools. There's like, it doesn't really seem like a home for short form video. Those are all directions they could go, and it probably would be a lighter, more fun, more entertaining app.
18:12 Casey Newton? The question is just what gets them to the billion users that they want? Is it just sort of this nebulous vision of a bunch of people having a good time, mostly not talking about the horrors of the world? Or is it something that a lot of people are telling them right now, I desperately want this, please build this?
18:28 Kevin Roose? Yeah.
18:28 Casey Newton? And by the way, I am not saying that I know which of those two things they should do. I do think that there is a possibility that if you're just thinking about this from a sort of like cynical capitalist perspective, they might want to stick to their plan to just build TikTok for text. But I don't know, man. A lot of the most successful apps of all time, they got successful once they started to do what their users were asking them to do.
18:49 Kevin Roose? Yeah, I mean, I share that sense of like desire for a product that lets me know what is going on in the world, or I can find good, at least sort of better than average information about an unfolding global conflict. At the same time, I worry when I see journalists flocking to Threads, you know, introducing themselves, saying, I'm here, this is my new Twitter, I'm going to use this the way that I used Twitter before. In part because this is just not a company that has a good track record when it comes to disseminating reliable information to a large base of users. You know, we saw what happened when Facebook took over the news ecosystem.
19:30 Kevin Roose? It was not good for Facebook, it was not good for users of Facebook, and it was not good for journalists and media organizations. It's taken a long time for the media to kind of pry itself away from the fire hose of Facebook traffic. And Facebook's domination of the news ecosystem really did have harmful effects, especially outside the US, especially in zones with conflict and war and strife. We've seen a lot of the harms that have resulted from that. And so when I see journalists sort of wanting to throw their attention into another platform owned by the same people who brought us the last series of information disasters, I just think like, what are we doing here?
20:10 Casey Newton? Sure.
20:11 Kevin Roose? Why are we trusting this company? It feels a little bit like Charlie Brown with the football.
20:14 Casey Newton? Well, and if what was being proposed here was Meta was going to news organizations and saying, we're going to give you a million dollars a year to go hire some people and we want you to post your news on threads first and we'll create a special article format that only exists on threads. And all the publishers in the world said, oh yeah, that sounds great to us. I would agree with you. That is a bad path to go down. We're not going to go down that path again. But I am somebody who believes that Twitter essentially discovered a huge desire. It might be niche in comparison to like what TikTok is for today, by example. But there are clearly a lot of people who want to have that sort of, let me just scroll with my thumb, get a quick sense of what's happening in the world. I want it to be multiple publications. I want it to be not just, you know, reporters from all these different publications, but
20:59 Casey Newton? celebrities, politicians, get all those people into one room. And as Twitter started to collapse, there were some people who said, we'll never have it again. Like there were these special conditions that created this and it was just kind of a one up. I don't believe that. I think those people want to get back in a room. And I think over the past few days, we've seen them getting onto threads.
21:16 Kevin Roose? Yeah, I agree with you that there does seem to be this kind of organic demand for something like the old Twitter. But I just, man, I just, we have run the experiment where Mark Zuckerberg and Adam Mosseri run the global information ecosystem. It did not end well, right? We had ethnic violence. We had far right authoritarians seizing power. We had like a global insurgence of misinformation. Meta specifically has proven that it cannot do both things. It cannot build entertaining apps that grow at a huge speed and, and, you know, have hundreds of millions or billions of users and distribute information responsibly. They have failed that test. And now it drives me insane to see journalists and media organizations just like lining up to trust them again with such a vital task.
22:01 Casey Newton? Well, sure, but you know what? At the same time, journalists have done something else insane, which is they've bet their entire futures on Google, right? So much of the revenue pumping through the digital media economy is just people writing quick, cheap stories and hoping that they get discovered by the Google algorithm, right? And as we have talked about on the show, there is a wave of generative AI that is coming for all of those stories, and I truly am terrified that it is going to leave digital media in ruins. And if and when that happens, and more and more of those sites go away, where are people going to find their news, right? The web that we are enjoying today truly might not exist in five years. And if that is the case, then people are just going to need a place to talk about things.
22:45 Casey Newton? And it might be that it is not Threads. I am so agnostic about whatever platform it becomes, but I do think that Threads has been positioning itself to grow fast in a way that Blue Sky and Mastodon just have not been.
22:57 Kevin Roose? Yeah, I agree with that. I know that this is going to sound like me being like a shill for the mainstream media or like a company man, but I really do think that if newspapers had never existed and you pitched a startup that would sort of aggregate all of the stuff that went viral on social media the day before and fact-check it and verify it through on-the-ground, you know, discovery and reporting, and then sort of rank it in order of importance, add some analysis and opinion in there, package it up and put it on your doorstep in a physical form in the morning, or on your phone, or your tablet, I think that would be like a very popular product. I think what a lot of people are clamoring for is exactly what the media, in some idealized
23:42 Kevin Roose? version of its past, was providing. Now, obviously, there are things that you can get on social media that you can't get in the mainstream media at least, you know, right away, and it does take some time for reporting to happen, but I really do think that what we've seen over the past week in this situation is just the incredible necessity of people doing professional reporting on what is going on in a global conflict.
24:09 Casey Newton? Absolutely. And, you know, if the question is- Like, would I be better off going to a newspaper homepage to get my news about this than checking social media? I believe that the answer is yes, right? At least as a starting point. At the same time, again, it is valuable to see first-person perspectives in real time on some issues, right? Think about what a lot of CNN is. It's people just with cameras pointing cameras at things. That is a lot of what we got out of the old Twitter. So, you know, look, if you're a publisher listening to this thinking, what should I do? You do have an opportunity to go out and build your own distribution. You do have an opportunity to build a better product. You have an opportunity to market yourself as a better product than what people are making do with right now. I think all of that is very real, but I just also think we're keying ourselves if we don't
24:51 Casey Newton? believe that there is a value in a distributed social network where people are just showing up and saying, here's what I saw today and here's what I think about it. Because over and over again, we learn that there is demand for this.
25:00 Kevin Roose? Totally. I just think publishers should not have to choose between giving Elon Musk control of their industry or giving it to Mark Zuckerberg. I just do not think that that is a choice that media organizations should make.
25:10 Casey Newton? Well, Kevin, where have you been getting your news?
25:14 Kevin Roose? I mean, so much of it has come from just mainstream media sources. I have really found myself just going back to, you know, the New York Times app, the Wall Street Journal app, the, you know, websites of local news organizations in Israel and around the conflict zone where I can read English language reports of what is happening on the ground. It sort of feels like how I consumed news, you know, 10 or 15 years ago before social media really took off. And it's making me wonder if this kind of decade-long dream of kind of like the internet as a global town square, of there being these sort of gigantic central platforms where everyone went to sort of figure out what was happening in the world, whether that dream is something
25:59 Kevin Roose? that is gone. It sounds like you don't think that is.
26:03 Casey Newton? No, because what people want in times like this is not actually just the vetted, fact-checked account of what has happened. They want the conversation around it, right? They want to get some sense of how should I feel about this? How are other people feeling about this? What are the arguments that are being made, right? They want to see people. Speculating about what might happen next. There are all these other parts of a story that are just typically not in a mainstream newspaper story for very good reasons, right? The newspaper stories are good at what they do, but they are only part of the solution. So unless the media is able to sort of aggregate all of the conversation in one place, there's just always going to be a market for these social apps.
26:44 Kevin Roose? Yeah, I agree with that. Yeah.
26:46 Casey Newton? I mean, obviously, like, you know, which social app wins is probably the least important question about all of this. At the same time, there is a profound desire to understand important global events in real time and what gets built will just always be of interest to us because it is how the future is shaped, right? Is in the places where the news is being made and distributed and being discussed. And so whatever it winds up being, whether it's a mainstream news site that's figured out a great new thing, or whether it's a social app or it's something that hasn't been built yet, I just think it will always be of interest to us here on this podcast.
27:23 Kevin Roose? Yeah.
27:28 Casey Newton? Coming right up, we're going to switch gears and talk about another way that people are starting to discover what is going on in the world by betting on it.
28:09 Speaker 00 Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.
28:44 Speaker 03 AWS AI is how the world's leading organizations are transforming their industries, not in theory, but in production at scale right now. Like turning race day data into real time insights for Formula One fans, reinventing the journey for every United traveler, and personalizing every trip with booking.com. However big the question for your business or industry, AWS AI is how.
29:09 Speaker 04 AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real time guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.
29:42 Casey Newton? Kevin, you did some great reporting this week about prediction markets, and it all started at something called Manifest, which I read that and I thought, well, is this just a festival for men? Tell us about what Manifest is.
29:55 Kevin Roose? So this is a field trip that I have been planning for a while. This was a very fun and interesting reporting trip to a conference for what they call forecasting nerds. So people who like to predict the future and bet on the future. And this was actually something that came out of an episode that we did several months ago about LK99. Do you remember this episode?
30:17 Casey Newton? Yes, of course.
30:18 Kevin Roose? So this was the room temperature superconductor that a group of scientists in South Korea had claimed to have come up with. And there was this period of maybe a week or two where people were hotly debating whether this was real or not. And we mentioned on the show the existence of something called manifold markets, which is a prediction markets platform where people can go and wager fake money on real world events. And one of the most popular markets was about LK99. And it was sort of a way to track like what the smart money people thought was going to happen and whether this prediction of a room temperature superconductor would pan out. Now, Did not, right? We, LK99 did not turn out to be a room temperature superconductor. But I heard from one of the founders of manifold markets who said,
31:02 Kevin Roose? if you're interested in prediction markets, we're actually having a big conference in a few weeks in Berkeley called Manifest, and you should come report on it. And I thought, well, that sounds like a fun trip.
31:11 Casey Newton? Yeah, I'd actually predicted that you were going to go to that. So that was interesting. So you get there and like sort of describe the scene because what you've described, I'll say it sounds a little bit dull, but then I read your story and it actually seemed like it might be a good time.
31:24 Kevin Roose? Yeah, it was a very strange event. And I say that like I had a good time and I learned a lot, but it was, it was definitely not what I was expecting. I was expecting like, you know, a sort of statistics conference where people in dress shirts and dockers would be going around, like comparing their predictive models of the world. But it was more like a party than I thought. I described in the article as sort of a cross between a math Olympiad and Burning Man. Like there was actually an orgy at this conference. And I know that because there was also a prediction market asking whether or not there was going to be an orgy. I think when I got there, it was like 28% possibility. And by the time I left, someone had had an orgy and closed out the market.
32:05 Casey Newton? Now that just feels like leading the witness to me. Now, I feel like if I go into the prediction market and I say, hmm, I wonder if there will be an orgy at this party? Anybody, anybody? And then just sort of watch the numbers slowly go up. So this is kind of one of the things that I am curious about is, does the creation of these markets wind up influencing the events?
32:25 Kevin Roose? Well, yes, and I want to get there, but I think we should talk about what this idea is.
32:29 Casey Newton? Oh, yeah. Tell us about what this is.
32:31 Kevin Roose? You know, when we talked about it on the podcast in the context of LK99, I believe we made some snarky comments about like, oh, these are just gamblers who like to bet on everything.
32:39 Casey Newton? Yeah.
32:39 Kevin Roose? But I would say after going to this manifest conference, there's also a real movement that I think is worth paying attention to here. Prediction markets, this is not a new idea, right? People have been betting on things like elections for centuries. Actually, in the 19th and early 20th centuries, it was common to like open up the newspaper and see a sort of betting odds breakdown of who people thought was going to win the next election.
33:03 Casey Newton? Like my entire life, I've been hearing about the wisdom of crowds.
33:06 Kevin Roose? Yes, that was a very popular idea. This idea of prediction markets was sort of revived in the 1990s by a group of economists who thought, well, markets collect information. You can bet on, you know, the price of a company's stock, or you can bet on, you know, corn futures. What will the price of corn be a year from now? You can also bet on sports games. Why can't you bet on other things? Why can't you bet about, you know, scientific discoveries? Why can't you bet about policy implementation? Why can't you bet about silly things like whether there's going to be an orgy at a statistics conference? So there's been sort of a real resurgence in the last few years, led by this group of people called the rationalists. Do you know much about the rationalists?
33:47 Casey Newton? A lot of it from reading your reporting, but yeah, tell us a little bit more about the rationalists.
33:51 Kevin Roose? So rationalists are a sort of loose collective of people who are sort of committed to examining their own beliefs. You know, they want to sort of get closer to the truth. Big figures in the movement are people like the guy who runs this blog Astral Codex 10, which used to be known as Slate Star Codex. Eliezer Yudkowsky is sort of an AI safety researcher and a prominent sort of rationalist blogger who started a website called Less Wrong. So there's a crew of people largely based in the Bay Area, but also spread out throughout the world, who are sort of doing what they would describe as rigorous empirical testing of everything that they believe and do. They love attaching probabilities to things. So I want to sketch out the vision for what they believe prediction markets could do.
34:35 Casey Newton? All right.
34:35 Kevin Roose? Because they're not just saying like this could be a way to make money by betting right on things. They're saying like, if you have everyone betting on everything, then you end up with a system where people are incentivized to understand the truth.
34:47 Casey Newton? Okay, so you have a bunch of people making predictions about things. How does that lead us to a better understanding of the truth or what's going to happen in the future?
34:56 Kevin Roose? One example that someone at this conference brought up to me is like, you know, imagine you have someone who's a believer in QAnon and, you know, they say, oh, Democrats are, you know, harvesting the blood of children. And, you know, you are a person who doesn't believe that, who thinks that's a conspiracy theory. And so you say, okay, want to bet. All of a sudden, that Person has to decide like, is this something that I believe in strongly enough to wager money on it? Or is this something that I'm just sort of saying for attention? And so the rationalists and the people who believe in prediction markets think that if you basically had to force people to put their money where their mouth is, it would moderate their views. They would back off some of their crazier beliefs.
35:32 Casey Newton? I would love to believe that. Unfortunately, I think a lot of QAnon believers would happily give you money. And if you showed them conclusive evidence that all of their beliefs are false, they simply would just not accept it. And then they probably wouldn't give you their money. But like, have these people predicted anything interesting? Or like, what is their track record?
35:49 Kevin Roose? So the track record of prediction markets in general really depends on what kind of prediction market you're talking about. So there's been some research that shows that prediction markets have some utility when it comes to predicting things like elections. But prediction markets are also wrong a lot in the same ways that, you know, some polls are wrong a lot. And I talked to a bunch of economists who basically say, if you had a perfect prediction market where everyone was participating, where everyone had sort of diverse sources of information and expertise, and where all of the people with the best information were sharing that information in the form of making predictions, these things would be quite accurate. But there are a number of things that keep that from happening, right? These are very tiny platforms still.
36:32 Kevin Roose? Manifold Markets, who put on this conference, has something in the neighborhood of 50,000 users, which is just very small relative to like, the number of people who bet on even like, sports games or something. And so basically, if a market is small, if the people who are in that market don't have access to good information, and if the question is not something that can be sort of settled with a simple yes or no outcome, prediction markets don't work as well. But interestingly, I was told that this is not a new idea to the world of tech, that actually, Google has run its own prediction markets internally for employees. So if you worked at Google, you could bet-
37:09 Casey Newton? On whether it would become a search monopoly.
37:12 Kevin Roose? Yeah, and everyone who made that bet got paid. No, it was they used a fake currency called Googles. And you could bet on things like, will this project launch in time? Or will Gmail get to this many users by this date? And the company's leaders use this as a way to like, gauge employee sentiment. And sort of when people could bet anonymously, they could actually get people's true opinions.
37:35 Casey Newton? Wait, that's fascinating. Do they not do this anymore?
37:37 Kevin Roose? Well, they were playing around with this as recently as 2021. And there were people I talked to at Manifest who believed that this is ultimately how all companies should run. Like, you get a job at a company and a prediction market opens up that says, in a year, will Casey Newton be more or less successful than we expect him to? And everyone in the company would bet on whether they think you will succeed at your job or not. And over time, you would essentially, you know, see who is the best at forecasting people's performance. And you could put them in charge of your hiring process.
38:10 Casey Newton? That sounds so unbelievably stressful. By the way, if you are really good at the Google predictions market, and so you had more Googles than anyone else, and then they shut down the market and you were stuck with all these Googles you couldn't use, we want to hear from you. I think that would make for a good story. Well, the Google story is really interesting, but it brings up something else I want to ask you about, which is the potential for people to manipulate these markets, right? You set up a market and then you either have some insider information, or you just sort of try really hard to make the thing happen.
38:37 Kevin Roose? Yes, I actually saw insider trading happen because I was interviewing someone at this conference. And he pulled out his phone and he showed me a prediction market that had been placed on Manifold about whether the New York Times would cover Manifold in an article in the year 2023. And as I was talking to him for this article, he was placing a large bet on yes, on that market with his insider information, which is that a reporter from the New York Times was in fact interviewing him for an article.
39:07 Casey Newton? And how does the platform view that? Is it just sort of, well, all's fair in love and prediction markets?
39:11 Kevin Roose? So they actually think that inside information and insider trading is good because people with inside information have the best information and they can bring it to a market. They have some sort of Elaborate theoretical underpinning for not believing that insider trading actually should be illegal. Right now, all of this is play money, right? Because of our gambling laws in the US, there are a couple sort of small real money prediction markets that are very limited, and it's not worth going into why, but most forms of real money prediction markets are not legal in the US.
39:43 Casey Newton? Right, and at manifold, they use something called mana as their virtual currency, which is also the same currency that you use to cast spells in Magic the Gathering and Diablo. So you're gonna want to manage your resources wisely.
39:54 Kevin Roose? Exactly, so this is this play currency called mana that you can use on the platform. They have leaderboards for who's got the most mana. You can also convert it into charity donations. They are not allowed to pay out real money for people who are right on these gambles. And you know, the people at this conference were upset about that. They think this should be legal. I have some concerns about that. I just don't know what it would look like for a society to be gambling on everything all the time. But they are sort of of the mind that the benefits of legalizing this kind of prediction market would outweigh the costs.
40:25 Casey Newton? You know, I gotta say, Kevin, I'm a really mixed mind about this. Because on one hand, the idea of people like betting play money to guess what might happen seems totally innocuous. Have a good time. It seems like you had a great time at this conference. It seemed like all the other people who were there did too. But I start to hear things like, well, these folks think that insider trading should be legal. And I just start to think, keep them away from the real economy. And this whole idea that we make better decisions when we have skin in the game, I just think has been really challenged over the past few years, right? Like, this was one of the big arguments for crypto. And crypto is the place where we used to hear all the time, you gotta have skin in the game. I use, we were told for years, you can make a better social network if you have skin in the game, right?
41:06 Casey Newton? You can develop a better relationship between musicians and fans if the fans have skin in the game. The whole idea of the Bored Ape Yacht Club was, give people skin in the game and they'll be able to make movies. And it all just kind of came to nothing. And one of the reasons was that when you give people skin in the game, and everything just has this like gross economic incentive tied to it, it just changes behavior. And people start kind of behaving in antisocial ways. So what do you think about the value of people having skin in the game? And is it possible that they're overstating the benefits here?
41:40 Kevin Roose? I think it's totally possible, and that's a really good point you brought up. But one application of this that I actually think is kind of interesting would be in our industry, in media. I had a conversation with the guy who runs Astral Codex 10. And one thing that he was saying is like, if the New York Times put little like prediction market things at the bottom of articles, for example, that might give readers a better sense of like what the probabilities behind the news events that they're reading about are. So you could have an article about who will be the next Speaker of the House. And then at the bottom of the article, you could have a little widget that sort of gave you the prediction market for someone specific or sort of an indicator of where the betting odds were on various people. And that might actually help you come to a better conclusion than just reading the article alone.
42:25 Kevin Roose? Do you think that makes any sense?
42:26 Casey Newton? Yeah, I think you subscribe to the New York Times and you're given a certain amount of New York tubos. And then you can sort of bet your tubos on who will be the next House Speaker. And, you know, I mean, I, what that makes me think of is the way that polls will be gamed on Twitter in the heyday, right? People would sort of say like, hey, do you think this thing is going to happen? And then it would get gamed as sort of the most zealous partisans would stuff the ballot box until the poll was over. And I wonder what mechanisms might be put in place to prevent something like that from happening here. But like, on balance, I'm persuaded that this is an interesting technology. And one thing that I have just observed in moving through Silicon Valley is that you do just constantly meet people who are into prediction markets. You know, it's like, along with poker, like, these are the two preferred forms of gambling
43:09 Casey Newton? and increasingly ways of socializing here in our strange little corner of the world.
43:13 Kevin Roose? I mean, the first place I saw this take off was among AI researchers who love to bet on, for example, what year we will get AGI or like when the first AI-generated screenplay will win an Oscar and things like that. And so they really are sort of running prediction models in their heads at all times. There's this sort of like cohort of people who are very into what they call Bayesian analysis or like attaching probabilities to things and living their lives that way. Do I think that is the way most people live their Absolutely not, but it is sort of an interesting idea. And as someone who makes predictions sometimes as part of my job, it's interesting to contemplate a world in which, you know, your, you know, your position as a pundit or a columnist
43:58 Kevin Roose? or a newsletter writer would be quantifiable in some way. Like, readers of Platformer could go in and say, "Okay, Casey's predictions were 75% right last year, so I'm going to trust him more. But if his predictions fall to only 50% right next year, maybe I'll cancel my subscription."
44:15 Casey Newton? I like, by the way, how amazing would it be to like, have the pundit score that sort of said, "This person predicted these like 50 concrete things in the past year and like 12 of them happened." That feels like, that feels like the sort of information that the right person would sue to get taken off of Google.
44:33 Kevin Roose? Totally, but you gotta spend your Googles to get that taken down.
44:37 Casey Newton? So should we set up a prediction market?
44:39 Kevin Roose? Let's do it.
44:39 Casey Newton? Let's do it.
44:41 Kevin Roose? So I'm going to log on to Manifold here.
44:43 Casey Newton? You already have an account? And do you have mana in your account?
44:45 Kevin Roose? I do, I have a thousand mana.
44:46 Casey Newton? Have you won any mana based on your bets so far?
44:48 Kevin Roose? No.
44:48 Casey Newton? Okay.
44:49 Kevin Roose? I lost, I didn't even bet on my own market, like, which was so stupid, because I knew I was gonna write an article.
44:55 Casey Newton? You knew you were gonna write the story.
44:56 Kevin Roose? But I, you know, I thought this is gonna get me investigated by the SEC.
45:00 Casey Newton? That's the last thing you need is Gary Gensler at your doorstep.
45:03 Kevin Roose? Okay, so on the front page, you can see all of the bets that people have going on right now. There are a bunch about Israel and Hamas. There are a bunch about the House Speaker, about the SBF trial, about the 2024 elections. And then you can create your own questions. So let's create a question here.
45:19 Casey Newton? Okay.
45:20 Kevin Roose? So Casey, what market should we create?
45:22 Casey Newton? What, well, something I'm curious about is, "Will Linda Iaccarino be the CEO of Twitter in six months?"
45:29 Kevin Roose? Okay, "Will Linda Iaccarino be the CEO of X on April 13th, 2024?" We can add it to a topic. Let's put this in technology. And then it says we can provide background info and details.
45:46 Casey Newton? Background information. Linda Iaccarino is the CEO of X.
45:50 Kevin Roose? Okay, is the CEO of X. And then we have to sort of create our resolution criteria. If she no longer has that title on April 13th, 2024, this market will resolve to no. Otherwise, it will resolve to yes. So what's your bet?
46:15 Casey Newton? My bet is no.
46:16 Kevin Roose? So we're going to bet 100 mana on this. And that is going to move the probability by 40% down to just 10%.
46:25 Casey Newton? So, oh, because we bet so much.
46:28 Kevin Roose? Yeah.
46:28 Casey Newton? So there's a sort of algorithm in the background that's saying, because we're willing to bet this much, it is therefore less likely that she will get the job.
46:36 Kevin Roose? Exactly. And we are the only participants on this market right now. But as other people bet, the probability will move up or down, depending on whether they bet yes or no.
46:44 Casey Newton? Very interesting.
46:45 Kevin Roose? So now, oh, I accidentally bet twice.
46:48 Casey Newton? Well, now, is that allowed?
46:49 Kevin Roose? Yes.
46:50 Casey Newton? Okay, that seems like a flaw.
46:51 Kevin Roose? So we've bet 200 mana. And now the probability is only 8%. So.
46:59 Casey Newton? So what you're, if you're listening to this, you're listening to market manipulation. This is how markets get manipulated by bad actors.
47:07 Kevin Roose? Yeah, that was what they call a fat finger trade in the financial business. Okay, so now we have our market up, and we can just keep monitoring it. And the idea is, behind prediction markets, that as this date approaches, there will be better and better information. And so the market will actually reflect reality, maybe even more than any individual person's opinion.
47:27 Casey Newton? That's right. We might not be able to like see inside of Elon Musk's mind to know how satisfied he is with his CEO. But the wisdom of the crowds will sort of intuit his vibe.
47:37 Kevin Roose? That is the idea. Yes, that's the pitch. So prediction markets, what do you think?
47:42 Casey Newton? Well, like I said, I think that there is clearly some value here. I think that a lot of this is innocuous fun. And I think it's probably something that we should continue to explore. It seems like at this point, we know that there is wisdom in crowds, that the sort of aggregated opinions of a large group of people are almost always going to be better than one person's opinion. Right? So let's go in that direction. At the same time, do I want a bunch of gamblers running the economy or our politics? Absolutely not.
48:08 Kevin Roose? Okay, well, I'm going to bet against your success on every prediction market I can.
48:12 Casey Newton? Wow.
48:13 Kevin Roose? And that's, that's mean. I won't do that.
48:16 Casey Newton? Okay.
48:18 Kevin Roose? When we come back, we're going to talk to a man who's trying to make AI smell.
48:48 Speaker 00 Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner, Dell. AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.
49:22 Speaker 04 AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.
49:54 Speaker 05 Ripple has been powering modern finance for a decade. It's not our first trip around the blockchain, and it won't be our last. We're leading the charge to bridge the gap between existing financial systems and the future of finance. You can have it all: legacy, stability, and digital innovation. Systems that work with your other systems. Because with the right partner, the question isn't crypto or traditional finance. The question is simply, why choose? A new standard of modern finance. It's happening with Ripple. For informational purposes only, availability of products can vary and may be provided by different Ripple entities.
50:28 Casey Newton? Well, Kevin, this next story stinks to high heaven.
50:33 Kevin Roose? I see what you did there. So this is gonna be a very interesting segment. I'm very excited for this. A couple of weeks ago, I was reading one of my AI newsletters, and I came across this item that I couldn't stop thinking about. And it was all about how AI was being taught to smell.
50:50 Casey Newton? Yeah, I read the same thing. And I came to you and I said, we have to talk about this.
50:53 Kevin Roose? Yes. So this is a project that has been sort of incubated inside of Google, but was spun out into a separate company called Osmo.
51:01 Casey Newton? You might say it was stincubated inside of Google.
51:03 Kevin Roose? Stop. All right. I'm putting a moratorium on smell-related puns and wordplay.
51:08 Casey Newton? Fair enough.
51:10 Kevin Roose? So back in August, researchers at Google and this company, Osmo, put out a paper in Science showing that their AI model was basically indistinguishable from the average human at predicting odors.
51:22 Casey Newton? You know, sometimes I can predict what you had for dinner, just using my sense of smell. It would be interesting to compare that with what the AI can do.
51:30 Kevin Roose? So they created something they're calling the Principle Odor Map. Basically, they are training an AI to be able to identify odors, not by sort of building a replica of the human nose, but by sort of mapping the relationships between different molecules and what they smell like. And this apparently has all kinds of interesting applications, everything from creating new fragrances and perfumes to possibly even being able to like detect disease. So today, we're bringing on the CEO of Osmo, Alex Wiltsho, to talk to us about their smelling AI, or Smell LM, if you...
52:06 Casey Newton? Is this a Smell LM?
52:08 Kevin Roose? It's a Smell LM.
52:23 Speaker 06 Hi there.
52:24 Casey Newton? Hello.
52:25 Kevin Roose? Hi, Alex.
52:25 Speaker 06 Hey, how's it going?
52:27 Casey Newton? Good. How are you?
52:28 Speaker 06 I'm doing great.
52:29 Casey Newton? Well, first of all, you smell great.
52:30 Speaker 06 Oh, thank you very much. Well, the thing is, we haven't invented that yet. We're on the road to that, so you should know that.
52:38 Speaker 06 The voice memo is going. As you could understand, I've got fancy candles that I can stack in order to get the recorder at the same height roughly as my microphone. And what, would you mind saying what scents those candles are? This is called Abd El Kader. So it doesn't give me like the notes of it. And then this one is kind of basic, right? It's the San Tol candle, but I love it. I think it's really, really great.
53:03 Speaker 06 So this is like the La La Bo classic scent.
53:05 Kevin Roose? If you could refer to those by their molecular structures, that would be more helpful for me.
53:10 Speaker 06 So you didn't tell me I needed to be in the laboratory to analyze them live and give you all the details in the readings. Now, what I did do is I got access permissions to our database so that I can check how many molecules are more or less fruity or cinnamon than anyone that we're going to try later. So you can quiz me as I type in queries across, you know, hundreds of millions of molecules and kind of give you the answer of what our models say. So I'm ready to dig in.
53:33 Kevin Roose? So I was looking at your company's website and I found a blog post that you wrote where you talk about sort of your origin story as someone who is interested in teaching computers to smell. You talked about being obsessed with smell from a very young age. So when did your interest in scent start and why?
53:53 Speaker 06 So I can't say why, I mean, because I was just born this way. Some people have a really wide open window to the visual world or to the auditory world. And for whatever reason, my window for the olfactory world has been very open. You say like carton of milk or something like that. I can close my eyes and imagine how it feels to hold it and I can imagine how it smells, but like I don't have any images at all. So I guess I'm just wired this way.
54:20 Kevin Roose? Yeah, so you said that you were a teenage perfume collector, which is, you know, very cool. But you also had an experience you wrote about with your dad when you were in graduate school that made you want to pursue this path.
54:35 Speaker 06 Yeah, so when I was 24, 25, my dad got sick and it ended up being brain cancer. And I later learned after many years having kind of processed his passing, which came about unfortunately very quickly, that cancer has a smell. And the more that I read about this, the more I realized that many kinds of illnesses and ailments and just, you know, wellness and health states of our body. Have a smell. And if you look at the literature, the science, if you read all these little anecdotes that are out there, it's clear, it's clear that we can detect so much more than just, you know, that I put too much hot sauce on my eggs. There's so much signal that's out there and we're just beginning to scratch the surface.
55:18 Speaker 06 And if we're able to give that ability to computers beyond maybe even what we can do as humans, but also what dogs and canines and other animals can do, if we can give that ability to computers so that this ability never rests and it always improves and benefits from the incredible ecosystem of computation, people are going to live healthier, happier, longer lives. So that's the mission that I'm on.
55:39 Kevin Roose? Well, and humans are sort of famously like not the best members of the animal kingdom when it comes to being able to sniff things out, right? We don't have the most developed sense of smell relative to dogs or other animals. And so if we could use AI to improve our perception of smells, that could potentially be a very good thing in all the ways that you're talking about.
56:00 Casey Newton? I could finally track my prey through the forest without having to resort to bringing a dog.
56:05 Speaker 06 Casey, you can do it today. So there's this beautiful study. Yes, beautiful study that if you get down on all fours and get to where the smells actually are, which is on the ground, like we're bipedal, we stand up very tall and we're far away from the good stuff. All the good smells are on the ground. So if you get down on the ground, you can track scents. Now you're slower than a dog. Dogs are amazing at this, but you can do it.
56:27 Kevin Roose? Wow.
56:27 Speaker 06 So I mean, maybe next time we get together, if we do this in person, I'll leave a little scent trail and we can each kind of try our hand at seeing if we can track the scent down.
56:35 Casey Newton? That sounds perfect.
56:36 Kevin Roose? Yeah, we now have an episode for next week. Casey goes crawling through San Francisco on all fours, identifying various scents. So let's talk about this experiment that you all ran as part of this research that was published in Science. You called it the odor touring test. And obviously that's a reference to the Turing test, which is sort of the famous like, can you tell if this thing is an AI or not, just based on chatting with it? But what is the odor touring test that you set up and how did it work?
57:02 Speaker 06 So here's how we set it up. We trained a panel of people to reliably rate what molecules smell like. And so the way that we did that is kind of like being handed a Crayola crayon box and being told to memorize the words on the crayon with the color of the crayon. Turns out even for smell, you can get pretty good at that with like four, five, six hours of training. So what we did is we trained this panel and then we digitally sniffed hundreds of thousands of molecules that were on a large database. And so we predicted the smells of all these hundreds of thousands. We kept our predictions secret. And then we selected this group of 400, sent them out to our panelists and they smelled them and rated them. So that's how
57:43 Speaker 06 all, you know, really big AI systems are trained is you collect lots of data from lots of people. And when you need high quality data, you average people or you have them adjudicate somehow. So the best you can do is the average of the panel here. And every person in the panel has some distance to the panel mean. Some panelists are closer to the panel mean, some are further away. And our model, most of the time, was closer to the panel mean than the median panelist, which is, in my view, an early form of passing the odor turing test.
58:10 Casey Newton? Do you think someday like you'll be able to make an app that's like a Shazam for smells? So if I'm walking through the world, I'm like, hey, what is that exactly? It'll just sort of be able to tell me?
58:17 Speaker 06 A hundred percent. I mean, that's, we know how to do that. And it's something that we're working on the fundamental technology to actually enable. And then if we can take that readout and then replay that smell, maybe with the same molecules that we detected, maybe with totally different ones. And that means we've fully round-tripped a smell. All right. So those are all steps on a staircase that are in front of us and we're starting to climb up them right now. But that's what's ahead of us.
58:43 Kevin Roose? That's fascinating. Well, it's a very interesting project and I will be very interested to see what the next steps in your research and development are. But right now, you have actually given us a quiz. You've mailed us a kit that is similar to the ones that the people who participated in your research got. And Casey and I are ready to take the odor turing test. Are you ready, Casey?
59:06 Speaker 06 Yeah, let's see if we can pass the smell test.
59:08 Kevin Roose? All right. So Alex, you have sent us a Osmo sniff test. Casey, here's your copy. This is a sheet of paper explaining what we're supposed to do, along with some samples of some of these odors. So here's our little kit here. So basically, this is testing how our ability to detect these smells is compared to the AI model, as well as to Osmo's master perfumer. So we are going to, first of all, we have to reset our noses. Have you ever reset your nose before?
59:43 Speaker 06 I don't think I have.
59:44 Kevin Roose? Okay. So it says, we have to smell our elbow pit, our inner elbow. This is a prank.
59:51 Casey Newton? It very much feels like a prank.
59:53 Speaker 06 Nope, that's a pro move. I've worked with perfumers a lot and been in a lot of smelling sessions and people just smell their skin.
1:00:00 Kevin Roose? Wow, okay, so my nose has been reset.
1:00:04 Casey Newton? Thank God I took a shower today.
1:00:09 Kevin Roose? Okay, so now we open the vial labeled 427-3 and we smell this and then we have to write down at least three words to describe it. Okay, so I'm gonna open this up and take a little whiff.
1:00:26 Kevin Roose? Okay.
1:00:27 Casey Newton? All right, I am now also taking a whiff.
1:00:31 Kevin Roose? Now, I have a bad sense of smell, so I'm not gonna do well on this test, I'm afraid.
1:00:37 Casey Newton? It's hard.
1:00:39 Kevin Roose? So I'm gonna write down my words, you write down your words.
1:00:44 Casey Newton? Yeah, what were your words for the first one?
1:00:45 Kevin Roose? Okay, my words were grape, violets, and purple, which is not actually a smell.
1:00:52 Casey Newton? You have synesthesia.
1:00:55 Casey Newton? I wrote down green apple, white wine, flowers.
1:00:58 Kevin Roose? Wow, we are so different. Okay, so here's number two.
1:01:06 Kevin Roose? Okay.
1:01:07 Casey Newton? Now, this one is on a strip rather than a vial.
1:01:09 Kevin Roose? Yes.
1:01:09 Casey Newton? Oh, I've had this cocktail, what is this?
1:01:13 Kevin Roose? Casey, what did you write down for number two?
1:01:15 Casey Newton? I'm so wrong. I've literally had that cocktail and I just don't know what it is, but I wrote down lime, simple syrup, strawberry. I'm just like groping around for like, what is the cocktail that I've had that has whatever that is in it?
1:01:30 Kevin Roose? One of us is really bad at this, cuz I wrote down bourbon, wood, and cayenne pepper.
1:01:36 Casey Newton? How interesting.
1:01:37 Kevin Roose? Number, now, number three.
1:01:39 Casey Newton? Wait, reset your nose.
1:01:40 Speaker 06 Reset nose, okay.
1:01:42 Casey Newton? By the way, if you're listening, feel free to reset your nose along with us at home.
1:01:44 Speaker 06 Everybody, reset your nose, it's time to reset your nose.
1:01:48 Kevin Roose? They call this an olfactory reset.
1:01:51 Casey Newton? That is an olfactory reset.
1:01:52 Kevin Roose? All right, here we go. This is a small vial with some yellow stuff in it called, labeled 400. Okay, I'm getting some, okay, I'm not gonna tell you what I'm getting yet. You have to come up with your words.
1:02:04 Casey Newton? Don't prime me, bro.
1:02:07 Casey Newton? Yeah, that was another, I have another set of wrong answers for you.
1:02:11 Kevin Roose? What did you guess?
1:02:12 Casey Newton? I wrote down sandalwood, bergamot, pepper.
1:02:15 Kevin Roose? Okay, I wrote down meat and tomato sauce.
1:02:20 Kevin Roose? Okay, so we have the last one. This is number 41.
1:02:26 Kevin Roose? Oh, I can get this one. No, I can't.
1:02:33 Casey Newton? This smells like the first one.
1:02:35 Kevin Roose? Okay, so for this one, I got moss, mildew, and forest.
1:02:39 Casey Newton? It's beautiful. And the random tokens I've generated would include fabric softener, lilacs, and the first one.
1:02:50 Kevin Roose? So we have guessed our four. Now we are going to open the envelope containing.
1:02:55 Casey Newton? The envelope, please.
1:02:56 Kevin Roose? The correct answers. Would you like to do the honors?
1:02:58 Casey Newton? Yes, I will. And these answers were actually prepared by Price Water's House Coopers. And they've been kept to the lock briefcase. And the first set goes to, the first one was.
1:03:09 Kevin Roose? This was the one that we guessed. So remind me what you guessed for number one?
1:03:12 Casey Newton? First one, I guessed green apple, white wine flowers.
1:03:15 Kevin Roose? And I guessed grape, violets, and purple.
1:03:17 Casey Newton? Okay, the Osmo AI descriptors were fruit, pineapple, and sweet.
1:03:23 Kevin Roose? So I got close with grape. That is a fruit.
1:03:26 Casey Newton? Yep.
1:03:27 Kevin Roose? Yeah.
1:03:27 Casey Newton? Okay, great. Number two, I had lime, simple syrup, strawberry.
1:03:32 Kevin Roose? And I had bourbon, wood, and cayenne pepper.
1:03:35 Casey Newton? Okay, Osmo AI says floral, spicy, sweet, and green. Green, interestingly, not a smell. So how do you explain that, Alex?
1:03:44 Kevin Roose? Green is definitely a smell. It's like fresh cut grass. It's like all the planty type things.
1:03:49 Casey Newton? Planty type things. Okay, all right, interesting. The master perfumers described it as floral muget? Do you know this word?
1:03:57 Kevin Roose? Muget. It's a little flower with like little bell type flowers. It smells like dryer sheets. Like dryer sheets are the smell of muget now.
1:04:06 Casey Newton? Yes. Now that you say it, number two was definitely giving dryer sheets. So that resonates with me. So other descriptors for that one include anise, apple, and pear. And I think it. Was the pear that was making me think it was a cocktail so much.
1:04:18 Speaker 06 Anis in like licorice can like that's often in cocktails. What's interesting about our prediction is Anis actually wasn't one of the labels that the model knew about. So did the next best thing, which is spicy.
1:04:29 Casey Newton? Huh. Hmm. Interesting.
1:04:31 Kevin Roose? I was getting spicy. Okay.
1:04:32 Casey Newton? Yeah.
1:04:33 Kevin Roose? That's good.
1:04:33 Casey Newton? All right. Number three, I had Sandalwood Bergamot Pepper.
1:04:37 Kevin Roose? And I had meat and tomato sauce.
1:04:39 Casey Newton? Okay. I feel like we're at least in the right zone. The Osmo AI described it as leathery, earthy and tobacco.
1:04:46 Kevin Roose? Okay.
1:04:46 Casey Newton? And the Master Perfumer's described it as saffron, fruity, leather, black tea.
1:04:51 Kevin Roose? Wow.
1:04:52 Casey Newton? So if we'd gotten leather, we would have really.
1:04:54 Speaker 06 You were definitely in the ballpark there. Now, my, my wife used to sell saffron. And so she knows the saffron flower like very well. And when she smelled it first, she's like, holy crap. Not only is it saffron, it's the specific part of the flower.
1:05:07 Kevin Roose? Oh, wow.
1:05:08 Speaker 06 And this molecule is definitely not in the flower. So it's, it's like, it's a totally new molecule, but with smells that exist elsewhere in the world. It's just.
1:05:14 Kevin Roose? So they can make fake saffron now that will only cost like a hundred dollars as opposed to $300 at the grocery store.
1:05:21 Speaker 06 For this molecule, you'll bring along a little leather and tobacco, which maybe doesn't, you know, fit with your risotto or, you know, the dishes you might want to use it in.
1:05:28 Kevin Roose? That's true. That's true. I hate when, when leather taste gets in my risotto. Okay. What's number four?
1:05:34 Casey Newton? Number four, our final one.
1:05:35 Kevin Roose? Where I had a moss, mildew and forest.
1:05:38 Casey Newton? And I famously had fabric, softener, lilacs and the first one. And Osmo described it as woody, herbal, fresh and mint. And the Master Perfumer's described it as woody, patchouli and sage. So if we spent a little bit more time in the lower heat with the grape instead in the sixties, maybe we would have gotten this one.
1:06:00 Kevin Roose? All right. So Alex, how did we do compared to your average panelists in your study and your AI model?
1:06:06 Casey Newton? Well, I actually think I can answer that, Kevin, because we didn't get one right. So I think that gives us a rough sense of how we did. But yeah, Alex, anything to add?
1:06:15 Speaker 06 I'm gonna, you know, step in and you're gonna get very strong partial credit on number one. You got the fruit. Right? And some people do perceive 427.3 as having kind of like that grape or red berry kind of a thing. And then I think you're in the ballpark for 400, right? You're getting like the kind of earthy aspect to it. And then I think Kevin, you get partial credit for being in the woods for 41.
1:06:39 Casey Newton? Yeah.
1:06:40 Kevin Roose? Yeah, Casey.
1:06:41 Casey Newton? Well, congratulations on your partial credit, Kevin.
1:06:43 Speaker 06 And this is with, this is the first time that y'all are doing this. I mean, that's incredible. It's like, this is not easy stuff, right? And I can see the gears turning as well. Like, it actually is kind of a, like, tardy. Like, you have to use your brain to, to, to pull the words out of your mind.
1:06:56 Casey Newton? Yeah, we, we, you know, you know what this is really making me think of, Alex, is Kevin and I need to spend more time stopping and smelling the roses. Because then maybe we do better at your quiz. Yeah.
1:07:07 Kevin Roose? Oh, absolutely. Well, and then ultimately, we can train an AI to smell the roses for us and put ourselves out of a job, so.
1:07:12 Speaker 06 Exactly. What kind of a job is it to stop and smell the flowers? It's the slice of reality that now you get to enjoy. Why would you outsource that?
1:07:19 Casey Newton? I mean, I feel very few people have a job that is closer to stopping and smelling the roses than the job you have, Alex.
1:07:24 Speaker 06 I love it. It's, it's my favorite thing in the world.
1:07:29 Kevin Roose? All right. Alex Wilshco, thank you so much. I really, really appreciate you coming on.
1:07:33 Casey Newton? And Alex, I've always wanted to say this to a guest, smell you later.
1:07:36 Speaker 06 Smell you later. Kevin and Casey, thanks so much.
1:08:31 Speaker 00 Support for this podcast comes from Dell Technologies. It's time to put AI to work. Is your business ready? The challenge isn't getting started with AI. It's scaling beyond pilots while maintaining control of your data, governance, infrastructure, and costs. Dell AI Factory provides the integrated foundation you need to scale AI securely and efficiently with one trusted partner. Dell, AI on your terms. From idea to impact. Learn more at dell.com/yourwaytoAI. That's dell.com/yourwaytoAI.
1:09:06 Speaker 04 AI spend is growing fast, but most businesses still can't see exactly where that money is going. Ramp gives businesses visibility and control across AI providers with real-time guardrails by team and model, helping ensure spend goes to the highest ROI areas. Stop guessing where your AI spend is going. Get started with Ramp. And if you're new to Ramp, you can get $250 off at ramp.com/hardfork. Go to ramp.com/hardfork. Ramp.com/hardfork.
1:09:37 Speaker 05 Ripple has been powering modern finance for a decade. It's not our first trip around the blockchain, and it won't be our last. We're leading the charge to bridge the gap between existing financial systems and the future of finance. You can have it all: legacy, stability, and digital innovation. Systems that work with your other systems. Because with the right partner, the question isn't crypto or traditional finance. The question is simply, why choose? A new standard of modern finance. It's happening with Ripple. For informational purposes only. Availability of products can vary and may be provided by different Ripple entities.
1:10:13 Kevin Roose? Hard Fork is produced by Davis Land and Rachel Cohn. We're edited by Jen Poyant. This episode was fact-checked by Caitlin Love. Today's show was engineered by Chris Wood. Original music by Marian Lozano, Rowan Nemezdo, and Dan Powell. Special thanks to Paula Schuman, Puywing Tam, Nell Galogly, Kate LaPresti, Jeffrey Miranda, Dylan Bergerson, and Ryan Manning. You can email us, as always, at [email protected] .
1:10:39 Casey Newton? But you can't smell us.
1:10:40 Kevin Roose? Not yet.
1:11:59 Speaker 04 You can't undo everything in life, but you can undo prediabetes. More than 2 in 5 adults in the US have prediabetes. Most don't know they have it, but with early diagnosis and action, you can delay or even prevent type 2 diabetes. Visit doihaveprediabetes.org to take the 1-minute risk test and start undoing the things you can. Brought to you by the Ad Council and the Centers for Disease Control and Prevention.
Previous Google’s Trial Heats Up + How to Wear A.I. + It’s Our Birthday! Next Peering Into A.I.’s Black Box + Who’s The Real Techno-Optimist? + Reading Ancient Scrolls With A.I. Page updated October 26, 2026 · transcript and summary are machine-generated
