330 | Petter Törnberg on the Dynamics of (Mis)Information

A characteristic of complex systems is that individual components combine to exhibit large-scale emergent behavior even when the components were not specifically designed for any particular purpose within the collective. Sometimes those individual components are us -- people interacting within societies or online communities. Studying the dynamics of such interactions is interesting both to better understand what is happening, and hopefully to designing better communities. I talk with Petter Törnberg about flows of information, how polarization develops, and how artificial agents can help steer things in better directions. Blog post with transcript: https://www.preposterousuniverse.com/podcast/2025/09/29/330-petter-tornberg-on-the-dynamics-of-misinformation/ Support Mindscape on Patreon https://www.patreon.com/seanmcarroll. Petter Törnberg received a Ph. D. in complex systems from Chalmers University of Technology. He is now an Assistant Professor at the Institute for Language, Logic and Computation at the University of Amsterdam, Associate Professor in Complex Systems at Chalmers University of Technology, NWO VENI laurate, and senior researcher at the University of Neuchâtel. Web site https://www.pettertornberg.com/ Univ. Amsterdam web page https://www.uva.nl/en/profile/t/o/p.tornberg/k.p.tornberg.html Google Scholar publications https://scholar.google.com/citations?user=KuJMuvIAAAAJ&hl=en Amazon author page https://www.amazon.com/stores/Petter-T%C3%B6rnberg/author/B0DRJ8H2G9 Bluesky https://bsky.app/profile/pettertornberg.com

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  From kickoff to the final whistle, trade the biggest moments in football on Kalshi, America's number one prediction market platform. Kansas City takes on defending champions Seattle, currently trading at 41% to win. A $100 trade pays out $244 if they come out on top. Download Kalshi, use code HOOPS to get $25 when you trade $25. K A L S H I. Kalshi, trade what's next. 18 plus only. Restrictions and eligibility requirements apply. Event contract trading involves risk and may not be suitable for all investors. Kalshi products are not available in all jurisdictions. Prices, values, and available markets may differ from those mentioned. For more information, see kalshi.com/ regulatory.

• 0:37Speaker 01

  Still waiting in line? Again? That's time you'll never get back. Save time and money with stamps.com. Over 4 million businesses have skipped the line with stamps.com. Join them to save up to 90% off carrier rates from your computer or phone right now. Print postage for certified mail, registered mail, and packages in seconds. Then schedule a pickup right from your home or office. For a limited time, go to stamps.com and use code PODCAST for a free welcome gift. Taxes and fees apply.

• 1:07Sean Carroll?

  Hello, everyone, and welcome to the Mindscape Podcast. I'm your host, Sean Carroll. There's an idea in social science circles called physics envy. Economics especially is susceptible to this idea. It's not supposed to be a good thing. You're not actually supposed to be envious of physics, but social science is hard. People are messy. There's a lot of variables going on. Physics is able to make enormous progress by simplifying things a great deal. In part, that's because the fundamental ingredients that we study, even though it's like quantum mechanics and cosmology and relativity and other things that sound way out, but there aren't a lot of moving parts. The basic things that we're looking at are sufficiently simple. You can describe them using relatively few variables,

• 1:52Sean Carroll?

  and you can isolate all the interesting things that are going on in these systems with small numbers of variables. As a result of this, you can make tremendous progress. You can prove theorems. You can do experiments that test your theories to many, many decimal places. It's a lot of fun. Of course, people would be envious. But it's a disease, or at least something to be avoided, to therefore try to make your social scientific research too much like physics. When you do social science, you should admit that there are complications there that cannot be abstracted away, in the same way that we abstract away air resistance or friction when we're doing physics. Nevertheless,

• 2:33Sean Carroll?

  I'm sure that everyone who listens to Mindscape on a regular basis knows, I do think that there are contexts in which physics-like reasoning can be helpful, or even interesting, in the social science. There can be contexts in which physics-type of reasoning and concepts borrowed from physics can be really useful, very interesting in the social scientific contexts. Ideas like equilibrium, ideas of emergence in general, ideas of what is collective behavior like when it arises from the sort of mindless, non-directed interaction of many small things. These are things that physicists think about all the time and are

• 3:13Sean Carroll?

  very, very relevant to the social sciences. So today's guest is Peter Turnberg, who is a professor of computational social science. I promise I didn't know this, but he admits on the podcast that he actually has a physics background, so this makes some sense. But he uses models, agent-based models, that we've talked about recently with Don Farmer and others, as ways to study the behavior of social systems. Can you make a little model where the individual pieces are either simple agents that always act in some way, or maybe there's a little bit of stochasticity in there, or maybe they're even very complicated? We'll talk about an example where Peter used LLMs, large language models, to model

• 3:57Sean Carroll?

  human interactions in social media landscapes. And then you can ask, what is the robust behavior? Do you get things that we observe in the real world? Do you get polarization? Do you get sort of an accumulation of influence in certain people, rather than having it be completely uniform? Is this good or bad, if you intervene in certain ways? In a social medium, can you make things better? Is it all the algorithm's fault? Or is it just the preferences of the individual actors? All of these kinds of questions can be addressed with this sort of slightly physicsy attitude, but put into a social science context. I don't want to give away too much about Feder's results, but they're not great.

• 4:40Sean Carroll?

  They're not very encouraging for those of us who want social media to work. There are certain natural dynamics that seem to come up that drive things in a bad direction, that drive polarization and echo chambers and things like that. It doesn't matter whether we like it or not, if that's what happens. Knowing how it happens and that it happens under certain circumstances hopefully will be helpful in making social media or media or information ecosystems in general more functioning for the purposes of democracy, but also for the purposes of just learning fun new things and having a good time and being good human beings and connecting with

• 5:21Sean Carroll?

  our peeps in various different ways and building connections that otherwise wouldn't have been possible. So we want to keep the good aspects of these wonderful technologies without being subject to the bad aspects. And I think this kind of study helps us figure out how to do this. So let's go.

• 5:54Sean Carroll?

  Feder Torenberg, welcome to the Mindscape Podcast.

• 5:57Speaker 03

  Thank you for having me on. It's a real treat as a longtime listener and fan of the podcast. It's really great.

• 6:03Sean Carroll?

  Okay, very good. Well, then you know how it's going to go. Let's just start setting some stage a little bit. In one of your books, you have this provocative line, "Power has an epistemology." Do you remember that one? That's from Seeing Like a Platform. So what does that mean to those people in the audience who might not use words like epistemology every day?

• 6:25Speaker 03

  Yeah, so it's a it's a good starting point, I think. So a lot of my research is kind of informed by this understanding of society being a complex system. So I kind of like, I actually have a kind of physics background myself, but like, long time ago, so don't don't quiz me on that.

• 6:44Speaker 03

  But and come very much from the kind of complex systems perspective. And in my PhD, I was kind of focusing on taking that perspective to try to see how can understand society using the methods of complexity science and using computational methods. And so this this book is basically centered around this idea, okay, we're seeing this notions of complexity becoming used throughout the social sciences, but also in kind of urban planning, you know, and also in kind of when we talk about digital platforms and how they're shaping society. And so we try to kind of understand what does that actually mean? And so, I mean, in in the social sciences, the way that complexity tends to be understood

• 7:28Speaker 03

  is, you know, like this bottom up system. So often you have like, you know, there's complicated systems like a car or spaceship that you can kind of take apart, you can decompose them, you have like an engine, and it's quite easy to figure out how they fit together. You can kind of understand the system by taking it apart. But then, on the other hand, on the other side of the kind of this line, you have complex systems, so like ant colonies or or flux of birds or whatever. And these systems, you if you take them apart, you know, like you take out an individual ant from an ant colony, you can observe its behavior as much as you want, but it won't tell you very much about how an ant colony functions, because the kind of intelligence of the ant colony emerges through the interaction of a lot of ants, right?

• 8:14Speaker 03

  And so the book kind of stems from the observation that we've shifted to more and more talking about society through this complex lens, and that it's kind of intertwined with this notion of new forms of democracy, because obviously, a lot of those ideas come from kind of from, you know, from your field, from physics and from from computer science. But as they're They're changing quite a lot as they're entering into the social world. And they also began to have kind of political implications. And so those are the kind of implications that we follow. And ultimately, in the social sciences, this becomes a question of kind of epistemology. So the question of like, how do we envision what society is?

• 8:53Speaker 03

  And we went from, you know, in the sixties and in what we as social scientists refer to as Fordism or industrial modernity, where we tended to see society as a machine, as a complicated system. So basically, that way of seeing and way of understanding society stemmed, we argue, from a society built on large industry, large mass production, and it led to kind of a mass society. And it led to the kind of ambition of like, we can design, we can plan society, we can build it as if we're building a machine.

• 9:26Sean Carroll?

  So this is Fordism as in Henry Ford and his assembly lines?

• 9:31Speaker 03

  Exactly. So that's, that image of, of the factory that Ford produced, because Ford, he didn't only produce a kind of way of producing, but also a way of consuming. And social scientists have looked at how that kind of idea of the industry, how it kind of leaked in society, leaked into society and shaped how schools function, how companies function, you know, like that it became a kind of way of organizing society, broadly speaking. And so what we are kind of observing in this book is the fact that we've moved from that kind of machine epistemology into an era that's defined by kind of complex systems, where we're talking about society as kind of swarms

• 10:15Speaker 03

  or as self-organizing, you know, we're using these different metaphors that stem from, that are much more organic, you know?

• 10:21Sean Carroll?

  Yeah.

• 10:21Speaker 03

  They're not the kind of machinery, you know? And we've also kind of, in some ways, abandoned the ambition of having the state design society to produce certain outcomes. So this kind of utopia, this hope of improving the world, has, we've kind of given up on that. And instead, and there is this idea that, you know, now that it's implying that the outcomes Of systems, like the bottom out up outcomes of systems somehow will be inherently better. That's a kind of underlying assumption of this, what you might call ideology. And so that's kind of what we're interrogating and questioning. Because we would argue that this isn't just something that's natural.

• 11:03Speaker 03

  There are still forms of power that are shaping society. They're just much less visible and they're operating through by shaping kind of how we interact. By things like algorithms. And so they often become kind of difficult to see, but they can still have very large structural outcomes on society. So for instance, it's by kind of fiddling with the rules of interaction, which is basically what platforms are doing, right? And that can have really important large scale outcomes and there is still power there. It's just power that has a new epistemology.

• 11:36Sean Carroll?

  So that that's very, very helpful. Thank you. I mean, I read the beginning of your book, but I didn't get through the whole thing. So this is, this is why we have the podcast, so I can just ask you questions. So in other words, let me try to rephrase it and see if I'm understanding. You know, we might have had this dream of planning and organization where it was top down. And either Henry Ford, who I guess was very capitalist at heart, but still, you know, he was trying to organize his factories. Or, you know, central planning from a government. And there are various arguments you can make that simply letting things come to an equilibrium. Is more efficient, whether it's a sort of thermodynamic equilibrium or an economic free market equilibrium or whatever.

• 12:21Sean Carroll?

  And you're making the point that maybe so, maybe that's sort of a better way of calculating some optimum. But it still carries with it its structures of domination and power.

• 12:35Speaker 03

  For sure. I mean, there is this, you know, even emergent outcomes, even when we arrive at an equilibrium. It's still like those outcomes are still defined by the conditions that we came in with, right? And so, I mean, it's, it's like these arguments that you sometimes hear, hear from certain, certain parts of academia where it's, they build. Some model of the economy and they find that certain people become very poor or like a large majority become very poor and certain people become very rich. And they say, well, it's an inevitable outcome of this system. We shouldn't try to change it. But that's just not, it's just not a good.

• 13:10Sean Carroll?

  Doesn't follow, right.

• 13:12Speaker 03

  Yeah, it doesn't follow because you could have produced other rules that would produce other outcomes. And if your outcomes that are produced by your system are problematic and are harming a lot of people. Then maybe you should reconsider the rules that you're operating under. And also with the notion that like, that the, this idea that the market somehow be natural. And not, you know, like that it would not be shaped by state power has been, you know, questioned for

• 13:37Sean Carroll?

  Of course.

• 13:37Speaker 03

  The better part of a hundred years that actually the market is very much a construction of state power.

• 13:43Sean Carroll?

  I remember the epiphany I had when I was thinking about different probability distributions. This is purely a math statement, but still it's, it's, you know, important. Like we have a uniform probability distributions, power laws, bell curves, whatever. And you might ask, well, which one maximizes the entropy? And the answer is they all do, subject to different constraints. Like it's, it's all about the constraints you put on the system macroscopically. Nothing is there really inevitably.

• 14:12Speaker 03

  Yeah, no, and I think this, I mean, it's a really, this is very much at the core of what I am interested in to certain degree. And I, I think it's also very much at the core of, of many of, of my papers and much of my research on this, which is this, the fact that as we've moved from a kind of a spatial society. You know, we, we moved from a society that we could, you know, if we were a physicist, you know, as a physicist, one might think of it as a kind of lattice.

• 14:38Sean Carroll?

  Right.

• 14:38Speaker 03

  You know?

• 14:39Sean Carroll?

  Our nearest neighbors.

• 14:40Speaker 03

  Yeah, exactly. And, and then we moved to a digital society, which is characterized by network structures. And those produce, you know, like they are associated with different type of distributions. As social scientists, we, you know, everything is a bell curve, right?

• 14:54Sean Carroll?

  Yeah.

• 14:54Speaker 03

  But, you know, as a computational social scientist, every, everything is power law.

• 14:58Sean Carroll?

  Everything is power laws, right.

• 15:00Speaker 03

  And that's not just like a, a question of how we study these systems and what assumptions we need to make. Is also, you know, that certain people, when we have digital network structures, become very powerful and most people are, you know, powerless and they don't get attention. They don't get resources that are important. And, you know, that's, that is profoundly problematic for society and they are attributes of these networks or the structures.

• 15:26Speaker 00

  From kickoff to the final whistle, trade the biggest moments in football on Kalshi, America's number one prediction market platform. Kansas City takes on defending champions Seattle, currently trading at 41% to win. A $100 trade pays out $244 if they come out on top. Download Kalshi, use code HOOPS to get $25 when you trade $25. K A L S H I. Kalshi, trade what's next. 18 plus only. Restrictions and eligibility requirements apply. Event contract trading involves risk and may not be suitable for all investors. Kalshi products are not available in all jurisdictions. Prices, values and available markets may differ from those mentioned. For more information, see kalshi.com/ regulatory.

• 16:04Speaker 04

  I'm Jake Stow, co-founder and CEO of Serval. We built Serval to automate the IT work that slows companies down. Onboarding, password resets, access to applications, my laptop stopped working. While employees wait for help, their real work is put on hold. IT desperately wants to automate this work and that's why they need Serval. You just tell Serval what you want to automate in plain English and it's built. No drag and drop workflows, no expensive consultants. Employees get unblocked and IT teams go from drowning in tickets to building what actually matters. With Serval, IT becomes the AI engine powering the entire company. This is a new way to run IT.

• 16:45Speaker 04

  We guarantee you'll automate 50% of all tickets and we'll prove it to you in a free four-week pilot.

• 16:52Speaker 04

  Go to serval.com/tickets. That's S E R V A L dot com slash tickets.

• 17:04Sean Carroll?

  Maybe a good inroad here, and I think you just gave us a good segue to it. But there's this classic work on self-organization in society by Thomas Schelling and his segregation model. So you have an updated version of that, but why don't you tell us what Schelling's version was?

• 17:20Speaker 03

  Sure. I think it was published in 1969 already. And it's still kind of pointed to, I mean, I use it all the time when I'm teaching, because I think it's just, it's still somehow the best model. It's not just the first. Also, we peaked, you know, as computational social scientists. And so, basically, Schelling was kind of, he was, I think, the backstories that he was like walking through his cafeteria in the university and was looking around and he saw like, "It's weird that, you know, all the, you know, geographers are sitting in one table and sociologists are at another table." And like, then he was like, "Yeah, and the same thing with the city." And he was trying to figure out like, "Why is segregation such a common outcome across systems?"

• 18:03Speaker 03

  And so, what he did, he actually took a checker's board, so he wasn't actually, you know, he wasn't even a computer simulation back then. And then he had coins on the checker boards. And he was, okay, so let's imagine that the system is a lattice and agents are randomly distributed on this checker's board. And each agent, they follow a simple rule. So if, if more than, like, you know, a very high number of their neighbors, their kind of more neighbors are of a different type than themselves, because there are two types of coins, then they move to a random available space on the lattice.

• 18:47Speaker 03

  And so, and you can have the rule something like that, they're completely satisfied if, you know, 70% are of a different type than themselves. But if it's like 90%, they're like, "Yeah, okay, like, you know, I'm tolerant, but like, come on."

• 19:01Sean Carroll?

  Right. So they want some neighbors to be, they want some neighbors to be like themselves, but they don't insist that most of them be like themselves necessarily?

• 19:08Speaker 03

  Exactly, exactly. And then, so they move to random space. So what happens? You would kind of expect that if they're happy with, you know, like eighty percent being of a different type, you would expect the system to settle on maybe something like eighty percent, you know, like, like, pretty much fifty fifty. You wouldn't expect very high levels of segregation to emerge. But what actually happens is that you get almost complete segregation, even with very kind of high thresholds.

• 19:36Speaker 03

  And why is that? Well, basically, you get this kind of cascade effect. So one person is leaving the neighborhood, the neighborhood is left more segregated. His like neighbors also move and you get this kind of cascade. And so basically, as a, what the model is telling us is that the integrated state is just very unstable. And so the system tends to tip over, like any neighborhood tends to tip over to one,

• 20:01Speaker 03

  to one color or the other. And so to me, looking, I mean, it's always been my favorite model. Yeah. And to me, that seemed to tell us something also about the digital world. So I was interested in like, can we generalize this to other type of interaction structures that are non-spatial? And so I look at kind of forums and different platforms because basically, when looking at platforms, when looking at social media, there's been like for the last twenty years, a debate on, because we often see echo chambers, right? We often see spaces being very homogeneous. And then there's been a longstanding debate about whether that is driven by the algorithms, like the filtering algorithms.

• 20:43Speaker 03

  So there's this notion of, Pariser's notion of, from his two thousand and eleven book, The Filter Bubble. So this idea that the algorithms create this, this cocoon, they only show us content that is same, like that we already agree to. And then especially in recent years, there's been more and more argument that that's not true, that in fact, we want to be segregated. We don't want to, you know, we don't want to be exposed to, to other ideas. Yeah. We don't want to encounter someone that disagrees with us. And so that's been the kind of the debate between those two positions. And to me, it seemed like. That might, you know, maybe it's neither. Right. So basically, what I do in this paper, and it's a very simple and short paper.

• 21:28Speaker 03

  I implement the Shelling Model, but I move it online to a certain degree. So I look instead, instead of having like a lattice, I look at, you know, we have different groups. So that can represent kind of subreddits, it can represent websites. And the agents are randomly located. So it's again, very similar to the Shelling Model. So it's as simple as possible. So the agents are randomly located to these groups. And then in each round, they interact with some random people in their group. And then it's the same rule as in the Shelling Model. So they're happy with, you know, as long as at least some of their interactors are of their same type. And if there is no one of their same type,

• 22:11Speaker 03

  or, you know, maybe a very few percentage, they move to a random other group.

• 22:17Speaker 03

  And I mean, with this background, I can kind of guess what the conclusion is. Yeah. What I find is that actually, the Shelling segregation effect is even stronger in this kind of communities, that they're even more prone to segregate. And of course, that has kind of interesting implications for this debate, because it's not necessarily either that this is driven by filter bubbles, that it's driven by algorithms, nor is it in the interest of anyone. It's just something that follows from having social interaction structured in these ways. And there's also some kind of counterintuitive results from this. So for instance, actually having filtering algorithms, having a filter bubble, it actually reduces segregation.

• 23:02Speaker 03

  Because if you have a filtering algorithm that always shows you someone who agrees with you, like, or, you know, in whatever you're shown, you know, some messages, you always get someone who agrees with you put into those messages, you will become less prone to move. To moving, okay. And so that will, like, for the system level, it will reduce the amount of segregation. And so the system will be much likely to be stable. Under those conditions.

• 23:28Sean Carroll?

  So.

• 23:29Speaker 03

  And so this is, yeah.

• 23:30Sean Carroll?

  In shelling, it's literally a checkerboard. Even in my book, The Big Picture, I talked about the shelling model a little bit. And so it's literally your nearest neighbors, which makes sense if you're talking about, you know, racial segregation in a city or something like that. And so you're saying you're putting it on a network, basically, right? Where there's different nodes that you can hear? Or is it, is it more dynamical than that? Is it, is it just like a different spatial structure or something that it changes with time?

• 23:58Speaker 03

  Focus on, I focus on groups in this case, so more like subreddits. So it's more like you join a community and then you're exposed to random people within that community. You can also run it on networks. But in the, in this sense, the, the network structures are less prone to this emerging as kind of shelling dynamic. Because you would have to, you need a kind of transitivity, you know? You need something like, if you're leaving the community, the community becomes more segregated and that increases the chance of someone else moving. So you get this kind of threshold effect where, you know, one person triggers another person. And in a network, you have to make really strong assumptions for that to be the case.

• 24:35Sean Carroll?

  Oh, okay.

• 24:35Speaker 03

  Because usually if you're, if you get annoyed with someone, you just unfollow them. But that doesn't change your, you know, your friends' networks. And so they're not going to be more likely.

• 24:43Sean Carroll?

  So there's not the positive feedback you get in your, in what you did?

• 24:47Speaker 03

  At least not this kind of shelling feedback.

• 24:49Sean Carroll?

  Good, okay. And, and by the way, my impression is that, you know, shelling was offering an explanation for urban segregation that did not require like, you know, racism handed down from on high via redlining or whatever. It was all just individual preferences. But in fact, when the social scientists have gone to look at it, the reason why real cities are segregated is in fact because of racism from on high, forcing it to happen.

• 25:16Speaker 03

  Yeah, no, I think this is a, it's a really important point. And it's, it's quite funny in some ways. I've, I've been in geography for, not anymore, but I was in, as a postdoc in, in the geography department for about four years and, and quickly realized that the only social scientists that do not really know or engage with the shelling model of segregation are the geographers. Because it's just. Seems fundamentally incompatible with that way of like with that way of thinking and I'm very much, you know, in agreement. And it's it is quite interesting to certain degree and I think it connects to the question of epistemology because it's it's kind of the Thomas Schelling segregation model, it gives a very deep insight into the kind of dynamics of segregation.

• 25:57Speaker 03

  But it's also really hard to kind of bring that insight into dialogue with the existing literature on segregation in cities that, as you say, very much point to kind of structural racism, redlining. But to a certain degree, I mean, both are true, right?

• 26:14Sean Carroll?

  Yeah, exactly.

• 26:15Speaker 03

  It's just it's just difficult to make these theories kind of speak to each other.

• 26:18Sean Carroll?

  Well, my my line has always been that the Schelling model is really good at explaining exactly what you started with, which is where people sit in the cafeteria, right? Like there, there's not rules like, you know, the the jocks and the nerds have to sit on different sides, but they always do because of exactly these preferences.

• 26:35Speaker 03

  Yeah, no, I think that's a that's a good point and it's probably also less of a provocative example than using it to think about our segregation.

• 26:43Sean Carroll?

  And so this idea that, you know, we do change our social social network or social media usage to be just a little bit more within a set of people that we want to hear, like, where does this apply in the real world? Is this, are we thinking of Twitter or YouTube or TikTok or or Facebook or what?

• 27:06Speaker 03

  Yeah, so there's this has basically been a long debate in the social sciences, this kind of question of how pervasive echo chambers are or not. And it's it's still a very, very heated kind of debate, so.

• 27:22Speaker 03

  But what I would say is basically that there are suggestions that there is quite a lot of communities that are relatively segregated. And so looking at, for instance, most subreddits are, if they are political, they tend to be towards one side or the other.

• 27:43Speaker 03

  But I do think it's an interesting, I mean, Twitter is has historically been a good example of the opposite.

• 27:49Speaker 03

  It was for a long time. Quite inclusive in the sense of having both political sides. And it's an interesting because Twitter has kind of functioned as the kind of model organism for social science research for looking at platforms because it's been one of the few platforms where we can actually get a lot of data, where we could. And so, I mean, a lot of computational social science research has looked at Twitter and used it as a kind of way of speaking about social media. I would say that Twitter is like, it was a very different platform from everything else because it actually had all political sides and it was characterized much more by a kind of conflictual debate.

• 28:28Speaker 03

  But if you look at kind of smaller communities, they do tend to be much more segregated in terms of opinion. And that can be a problem in the sense that if, you know, political theorists, when they talk about what conditions need to be fulfilled for us to have a kind of functioning political discourse, functioning deliberation, one of those conditions are that we need to have kind of diversity of opinions. We can't just have like political side, which is, I mean, pretty obvious, I guess.

• 29:00Sean Carroll?

  I guess I was gonna ask a question about that. How bad is it if people on social media interact with people who are like them? Like, I can imagine maybe a utopian political structure wouldn't be like that, but you know, most people on social media are not there to be utopian political actors, they're there to talk to their friends and be reinforced. Is that so terrible?

• 29:25Speaker 03

  So, I mean, it's a, I would say that I, I think in a lot of cases, it can be even very beneficial. I mean, in some way, one of the like ways that social media transformed society, one of the key ways, was this possibility that we couldn't connect with anyone from all over the world. And so, for a lot of communities, especially minorities, you know, if you are LGBT and you grow up in a small village somewhere and you don't, you know, have anyone can connect to, it's been shown that it's very beneficial for your mental health and for your experience, for your lived experience.

• 30:05Speaker 03

  At the same time, the way that it affects politics is not always as beneficial.

• 30:10Sean Carroll?

  Right.

• 30:11Speaker 03

  Because obviously, if the minority that you belong to happen to be kind of, you know, some extremist form of neo-Nazism, it seems to have similar kind of consequences for those communities, because it allows them to come together and form a kind of shared sense of community, and it transforms them from being, you know, someone isolated to a kind of confident political community, and that can be quite dangerous in terms of radicalization.

• 30:43Sean Carroll?

  And is this, I mean, my impression is that it is something that comes from new-fangled technology, social media, things like that, the ability of these smaller groups to come together, like, some of them are just going to be people who like to crochet, and others are going to be neo-Nazis, right? And, but is there, is there data that backs that up in the, in the sense that, have we seen more viability of these small groups than we did in the 1960s or whatever?

• 31:12Speaker 03

  It's, I mean, it's very difficult to kind of look at those kind of changes, right? Because we, unfortunately, we only have one society, so it's hard to compare how society would look different without social media, without digital media. But what we can say is that we've seen a kind of increase in, in political violence, we've seen a kind of democratic backsliding in a lot of countries, and we've seen the kind of political extremist movements entering to the political mainstream.

• 31:47Sean Carroll?

  Right.

• 31:47Speaker 03

  And whether or not that is closely linked to social media is, is very difficult to say, but it is very clear that it is, in our current society, very much entangled with social and digital media.

• 31:59Speaker 03

  And I've looked, so in my previous book, Intimate Communities of Hate, we look at one of these online communities, and basically, we try to, to answer this question by going in depth and looking at the Stormfront community, which is a, it's a very old, kind of Nazi community in the US.

• 32:18Sean Carroll?

  Right. They predate social media, right?

• 32:21Speaker 03

  Yeah, so basically it goes back to like 95. And the nice thing about it is that you like all of the data, all of the conversations over this, you know, long period of time is all available online, you know, if you if you're able to scrape it and bypass their very security from preventing you from scraping it. So we have all of the data and all of the conversations over this, you know, 20 plus year period. And so that allows us to kind of look at how the users are changed by interacting with this community. So we can kind of use natural language processing and various forms of text analysis and kind of see of how individuals, when they interact in this community, how does it change their

• 33:03Speaker 03

  language and different markers of like how they perceive themselves and so on. And we've the the kind of image that we come out with is much more, you know, it's it's a very much the kind of a question of a community formation of changing identity and so on. And so just an example, we can kind of see how when they first come in, they use I and my and speak of themselves. But then over time, they start saying we or SF, you know, for Stormfront, because they start, you know, it's kind of a marker of them starting to think of themselves as part of a collective and as part of something larger.

• 33:39Speaker 00

  From kickoff to the final whistle, trade the biggest moments in football on Kalshi, America's number one prediction market platform. Kansas City takes on defending champions Seattle, currently trading at 41% to win. A $100 trade pays out $244 if they come out on top. Download Kalshi, use code HOOPS to get $25 when you trade $25. K A L S H I. Kalshi, trade what's next. 18 plus only. Restrictions and eligibility requirements apply. Event contract trading involves risk and may not be suitable for all investors. Kalshi products are not available in all jurisdictions. Prices, values, and available markets may differ from those mentioned. For more information, see kalshi.com/ regulatory.

• 34:16Sean Carroll?

  So that's very interesting because it's not just about people in the study we were just talking about, you're talking about associating with people who are like minded. And here you're talking about the feedback acting on yourself and, you know, the individual people are sort of changing their identities in response to those interactions.

• 34:34Speaker 03

  For sure. I mean, I think it's very clearly, you know, a kind of a feedback process, right, where we have a kind of segregation mixed with a kind of changing of identities. And so we can kind of see, you know, in the Stormfront case, especially after the 2008 Obama election, is that the few days after the election, there was just a huge surge of users coming in, so new people joining. And looking at what they're saying, we can see this kind of emotional kind of confusion and anxiety, and they're trying to, you know, they somehow feel confused about this world that they're they're living in, where, you know, a black person who's like articulate can become president.

• 35:15Sean Carroll?

  Yeah.

• 35:15Speaker 03

  And what does that mean for their self-identity? And like, how can they make sense of this? And then the community function is kind of, you know, emotional talk therapy that allows them to find new narratives and kind of resolve this emotional anxiety and turn it into kind of from something passive like anxiety to something active like anger or outrage. And they come out with these narratives about, you know, these absurd narratives about the Jews and, you know, like the actually the whites are the superior race. It was just, you know, whatever that happened. And, you know, so it's very much a kind of process that is on the level of like, identity and emotion and kind of self-narratives.

• 35:58Sean Carroll?

  It's very interesting, because it all cycles back. With the very first podcast episode of Mindscape, I interviewed Carol Tavris, who's a social psychologist. And she has this idea of the pyramid of choice, when if you imagine two people who are basically 50/50 as to how they could make some certain choice, right, you know, what sneakers to wear or whatever. But once they make the choice, if they make it in different directions, they start justifying that choice to themselves. And they end up very far apart, even though they were essentially indistinguishable before they, you know, collapsed their wave function on that particular option.

• 36:33Speaker 03

  Yeah, no, that makes sense. And so, I mean, ultimately, it's kind of the question of the kind of structural context in which people are interacting that can produce this kind of outcomes. And so, yeah.

• 36:44Sean Carroll?

  Can I ask about either your model or the Shelling model? There seems to be, like, as the physicist in me thinks of the Ising model, which is similar but not exactly the same, where you have spins that are interacting on a lattice. And the thing we do there is we introduce probabilities by having a temperature, right? There's some chance that the spin is going to flip and whatever. And in the Shelling model, there's a probability because when the person decides to move, where they move to is random. But the choice about whether to move or not is not random, right? That's just determined by how many neighbors they have of each kind. So have people done that? Have people introduced a probability of moving rather than a certainty and seen if that changes anything?

• 37:28Speaker 03

  Good question. I honestly don't know.

• 37:31Sean Carroll?

  Okay, I don't know either. Someone should do that, someone listening out there should.

• 37:36Speaker 03

  Maybe something for us to do.

• 37:38Sean Carroll?

  Yeah, yeah, absolutely.

• 37:40Speaker 03

  You want a co-authorist.

• 37:41Sean Carroll?

  And then, okay, so you then did a different study, which came out also very recently, using large language models. And here, well, I'll let you tell the story, but the idea is rather than just having these mindless dots on a grid or whatever that are interacting with each other, you literally had little agents talking to each other and making choices in a social media context. So how did that go?

• 38:09Speaker 03

  Yeah, so basically, the aim of this is to address, in part, this like kind of longstanding criticism from social scientists, or at least from a lot of social scientists when it comes to agent-based modeling, which is that, you know, these rule-based agents are just not very good representations of the full spectrum of human behavior, which is, I think, fair enough. And I think, I mean, in a lot of cases, like the Shelling segregation model, this simplicity can be, is very useful, and it allows us to, you know, throw light on some emergent phenomenon that is ultimately structural. But in other contexts, it is also limiting, right? So looking at social media, like politics on social media, for instance, it's an example where

• 38:52Speaker 03

  These richer behaviors, they also can really matter, right?

• 38:58Sean Carroll?

  Right.

• 38:58Speaker 03

  They, we cannot really separate the cultural from the structural. We need to look at them as kind of intertwined and as, as interacting. And so that's kind of the, the background because

• 39:10Speaker 03

  what I'm interested in here, or what we're interested in is this question because we basically, you know, we spent 20 years or something criticizing social media, pointing to the problems and a link to various problematic outcomes. But now there's more and more kind of interest in like, can we be a little bit more constructive? Can we actually like do something about this? Because ultimately, if social media can shape a politics that is like outrage driven and radicalizing, it should also be able to shape a form of politics that is, you know, pro-social, that has like healthier political and social outcomes.

• 39:42Sean Carroll?

  Yeah.

• 39:43Speaker 03

  And so that's the kind of idea. But, and how do we study that? Well, using observational data doesn't really work. So that's a kind of modeling approach can be really beneficial. And so we're using agent-based models, but having instead of these rule followers, we're using large language model. And they work as kind of stand-ins for, for humans.

• 40:03Sean Carroll?

  Maybe if I, if I could interrupt you just quickly, I mean, maybe give a little bit of background on to the concept of an agent-based model, like as opposed to what? What is, what kind of models are not agent-based and what are agent-based and, and what is that used for?

• 40:18Speaker 03

  Sure. I mean, so in the social sciences, the way that we have traditionally approached the social world is in, and to link back to where we started, very much as a kind of complicated system. So we tend to think of society as kind of variables, interacting, which in a lot of cases can work really well. But if you're thinking of this kind of complex aspects of, of, of the social world, where you have interaction between agents and then leading to unexpected outcomes, those traditional kind of variable-based approaches just don't work at all. Like, how would you, like, using used variables? How would you like study the murmuration, you know, like of

• 41:00Sean Carroll?

  Okay.

• 41:01Speaker 03

  of birds? It's, it wouldn't be possible. And so agent-based models is kind of.

• 41:07Speaker 03

  Using this kind of bottom-up modeling approach where we And so one example would just be the shelling model, right? Like, so you have agents that are individuals, they follow simple rules, and then you look at the outcomes. And that allows you to link together the kind of micro behavior of individuals with system-level outcomes that can often be kind of unexpected from the rules that you're coming in with.

• 41:28Sean Carroll?

  So the individual agents need not be very complex themselves?

• 41:32Speaker 03

  Traditionally, they haven't been. So they traditionally have been kind of simple rule followers. So you have like, you know, like the shelling threshold rule, or you have maybe an optimization rule. But basically, building agents that would kind of mimic human behavior in terms of reasoning or language production, it just becomes impossible traditionally, right? Like, it would just be extremely complicated.

• 41:56Sean Carroll?

  Yeah.

• 41:57Speaker 03

  You would have to build a kind of, you know, a reasoning agent, which we just haven't had up until, you know, a few years ago.

• 42:05Speaker 03

  But so basically, when ChatGPT came out and with the kind of rise of large language models, it just became a huge amount of interest in like whether we can use these models to as part of agent-based models to kind of simulate social behavior. And so that's the kind of what we're doing, but we're trying to also use it to contribute some kind of social scientific theory and contribute to our understanding of of social media and its dynamics.

• 42:32Sean Carroll?

  And so what exactly was the experiment you did? I think of it as roughly speaking, letting loose a bunch of LLMs on a fake social network.

• 42:41Speaker 03

  That's that's pretty much it. But basically, our idea was to try to create a social media platform and make it produce the negative outcomes that have been observed on on real social media. And then try out a bunch of suggestions from the literature on how we can address those problems. And so our expectation coming in was kind of that we would have to, you know, fiddle a lot with the system and like try to make it produce problematic outcomes. And then, so we could then see like how kind of how stable those outcomes are and how easy, like what's kind of solutions are best best for addressing the problems.

• 43:21Speaker 03

  And so the problems that we focus on, it's kind of conditions of social media that make public deliberation or public conversation difficult, it makes it difficult to have a kind of a functioning politics playing out on these platforms, drawing on kind of political theory. And so there are three different things that we've already touched on a little bit. But so one of them are echo chambers that you you do need to have if you're going to have a kind of constructive conversation across the political divide. You need to have both sides of the political divide present, otherwise it's going to be really hard. So that's one condition you need. And then the second is this kind of question of attention inequality, we also touched on. If you're going to have functioning political discourse,

• 44:05Speaker 03

  you need to have relative equality among individuals. So you can't just have like two or three individuals dominating the entire conversation because that's not a public discourse, that's just a broadcasting. And then finally, what's been referred to as the kind of social media prism. So this is the idea that you need to have a kind of constructive debate where people are actually trying to come to a solution. And so that speaks to this question of that that social media has kind of tended to benefit loud, polarizing, conflictual voices. And that is very much kind of undermining functioning conversations. And so those are the three outcomes that we were trying to kind of see if we could produce.

• 44:50Speaker 03

  And we were expecting that to be quite hard, to be honest.

• 44:54Sean Carroll?

  That's so sweet that you thought that would be hard.

• 44:57Speaker 03

  Well, I mean, the literature has kind of pointed to or argued that a lot of these are especially the kind of the social media prisms or this kind of the polarizing tendency, that those will be expression of engagement algorithms. And so that they would be like the expression of social media identifying the most outrageous things that are being said and then shove it in your face to to kind of make you upset and increase the probability that you will comment or engage with a post.

• 45:26Sean Carroll?

  So, so, sorry, so the the sort of two alternatives that we're trying to test here are, one is that the, when you get these echo chambers and polarization, things like that, it's the algorithm's fault or the platform's fault versus this is just human nature.

• 45:45Speaker 03

  Well, so I'm not sure if I would put that as the context really, because it's also like in this study, we don't, you know, I was honestly just kind of assuming that it was from the algorithms at least, that it's not just something that would emerge from human behavior.

• 46:04Speaker 03

  But these are kind of structural outcomes from the interaction between people and the rules of the platform. But to me, at least this kind of social media prism, it's such a, it's a rather specific thing. It's kind of odd outcome that the most extreme voices get more attention. And so I was expecting that to be, and I've written about this before, arguing that it's the kind of, what I've called it, the trigger bubble. So, you know, it's not the filter bubble, but the trigger bubble. It's the social media algorithm trying to trigger you, make you upset in order to make you engage, because that's how the platforms ultimately make money. They make you-

• 46:43Sean Carroll?

  Sure, that's what we've all heard.

• 46:44Speaker 03

  Post something, they draw information, they figure out who you are, and they sell ads. And so that was kind of my expectation. But, and basically what we started, we just, was just building the most, you know, bare bones platform we could imagine, which is just the agents. So I should say also that the agents, their personalities are, we take the A N E S, so the American National Election Survey, which has very detailed information about US citizens, including their politics and like, you know, if they, if they like to go fishing,

• 47:20Speaker 03

  we have very detailed information. And basically, we turned that into a description for persona. Because, you know, L L M's, they really love to impersonate people. You know, you can ask it to, you know, explain your microwave and the voice of Shakespeare, and it will do an excellent job.

• 47:35Sean Carroll?

  So, just to be clear, so the L L M's that you let loose on the social media, they weren't all, you know, Adam and Eve? They weren't all Tabula Rasa from the start? You gave them a backstory.

• 47:45Speaker 03

  Exactly.

• 47:46Sean Carroll?

  Each one.

• 47:46Speaker 03

  And the point here isn't necessarily that, you know, we want them to be completely realistic encapsulations of, you know, of this particular individual that they're enacting. We just want to have kind of a diversity and want to capture a little bit of the, a little bit of that cultural richness, you know. And so then they are allowed to interact on this platform. And the platform is very simple. So basically, they can look at the, they see the most recent news, a random selection of news from the specific day that we're simulating. And then they can choose to post about their news, these news, and they can just write a post about whatever, based on it. Or they can, they see the timeline and they can choose to repost what someone else has

• 48:31Speaker 03

  written, someone that they follow. And they can also see, based on the post that they see, they can choose to follow someone that shows up in their feed. And if they follow someone, they kind of go into their timeline, they see their, a little presentation about them, and then they see their, you know, most recent posts.

• 48:48Sean Carroll?

  Wow, okay. And can they unfollow people too?

• 48:50Speaker 03

  No, we don't, we don't have unfollowing. It's really bare bones.

• 48:54Sean Carroll?

  Okay, but there is some ability, they can choose to follow someone or not.

• 48:59Speaker 03

  Exactly. And so what we're looking out at the, at the outcomes here is the network structure that emerges through this interaction.

• 49:08Sean Carroll?

  Yeah.

• 49:09Speaker 03

  Because it's, because we want to have simple measurable things, and we don't want to just look at, you know, how they're talking or something that's like an immediate outcome of, of the large language model. We want to have something that's more an expression of the structure of the platform. And network structures are interesting in the sense that they are very much kind of emergent and they are produced through their structural outcomes. And so that's what we're focusing on. And so we're trying to identify these three attributes. And to our surprise, we didn't actually need to do anything more than provide this bare bone platform, and we got these three features that are, that widely considered the problematic aspects of social media. But I should say that that doesn't mean that engagement algorithms aren't problematic.

• 49:54Speaker 03

  That doesn't mean that, you know, that, you know, it might still be that, that they're making matters worse. But it does imply that removing them will not completely solve the problem.

• 50:03Sean Carroll?

  So how many users did your social network have?

• 50:08Speaker 03

  We ran it with 500 users. 500, okay. Which is, it's a little bit of a limitation with the approach. It's that it is quite expensive compared to running a conventional agent-based model. Right. And conventional agent-based models have always been criticized for being very expensive to run computationally. So it is kind of a weakness of this approach.

• 50:31Speaker 00

  From kickoff to the final whistle, trade the biggest moments in football on Kalshi, America's number one prediction market platform. Kansas City takes on defending champions Seattle. Currently trading at 41% to win. A $100 trade pays out $244 if they come out on top. Download Kalshi, use code HOOPS to get $25 when you trade $25. K-A-L-S-H-I. Kalshi, trade what's next. 18 plus only. Restrictions and eligibility requirements apply. Event contract trading involves risk and may not be suitable for all investors. Kalshi products are not available in all jurisdictions. Prices, values, and available markets may differ from those mentioned. For more information, see kalshi.com/ regulatory.

• 51:08Speaker 04

  I'm Jake Stauk, co-founder and CEO of Serval. We built Serval to automate the IT work that slows companies down. Onboarding, password resets, access to applications, my laptop stopped working. While employees wait for help, their real work is put on hold. IT desperately wants to automate this work, and that's why they need Serval. You just tell Serval what you want to automate in plain English, and it's built. No drag and drop workflows, no expensive consultants. Employees get unblocked and IT teams go from drowning in tickets to building what actually matters. With Serval, IT becomes the AI engine powering the entire company. This is a new way to run IT.

• 51:50Speaker 04

  We guarantee you'll automate 50% of all tickets, and we'll prove it to you in a free four-week pilot.

• 51:57Speaker 04

  Go to serval.com/tickets. That's S-E-R-V-A-L dot com slash tickets.

• 52:08Sean Carroll?

  And you said that the bad outcomes happen. Reminded us what the bad outcomes were. There's a list.

• 52:14Speaker 03

  Yeah, so basically you get echo chambers. So Democrats and Republicans end up not following each other. They're just talking to themselves. You get high levels of inequality, basically power law distributions of attention. So a few users kind of dominate the entire discourse. And then finally, you get what Chris Baal has called the social media prism, which is that the more polarized, more extreme users tend to have more attention.

• 52:43Sean Carroll?

  Okay. Okay. And is there, I mean, I, it's very, very tempting to speak anthropomorphically about LLMs, right, and to ask about their motivations or something like that. But of course, that's illegitimate. They're just faking that. They don't really have motivations. But what is the right way of phrasing the answer to the question, why doesn't LLM want to follow people on its own political side of the spectrum?

• 53:12Speaker 03

  So, I mean, so where these outcomes are kind of stemming from, like, I think it's a little bit different outcomes each one. But so for instance, that we get the power law distribution, that we get very unequal distribution of attention. I mean, that's kind of stems from preferential attachment, right? That the probability of you getting a follower is proportional to how many followers you already have. So that, I wasn't so surprised to a certain degree that that emerged because it is a well-known feature of network, as I already mentioned earlier. But the other kind of dynamic that we identify that we haven't really seen studied before because you do kind of need to have this kind of special large language model

• 53:54Speaker 03

  network combination to be able to study it, which is the fact that Basically, we know that retweets, people repost or sharing is very effective, it's very emotional, it's very much reactive. So we see something that we're upset about or that we're really emotional reacting to, and those are the types of things that we tend to share. And that, you know, that is well known and it's been kind of argued that that shapes the content that we see on social media. But what we're adding to that is that it's not only shaping what culture, what kind of content you see, but it's also shaping the kind of construction, the gradual formation of the network structure. And so you have a kind of feedback effect between what is shared, which is very much

• 54:38Speaker 03

  emotional, and who you follow. And that is what's creating this kind of dynamic where more polarized users tend to have more followers and get more attention, because it is, the kind of sharing is part of this kind of feedback process.

• 54:54Sean Carroll?

  I understand how an LLM, you know, can respond to a query and say something, but did you have to cook up an extra set of instructions for when to follow somebody?

• 55:06Speaker 03

  We only, we don't provide them with any, you know, it's just like, this is your persona, how would you, based on this persona, how would you act in this situation? And then just have them kind of respond on the basis of that.

• 55:18Sean Carroll?

  Okay, and the persona includes their political orientation as well as whether they like fishing or whatever?

• 55:23Speaker 03

  Exactly, so it contains their political affiliation. And so in terms of the echo chambers, it's also not maybe so surprising. And we can also look at the kind of motivations that LLM is giving for why they choose to follow them. And it does, like in terms of the echo chambers, it's kind of like, okay, so I don't want to, like, since I'm Democrat and I feel strongly about it, I don't want to engage with this person who's from the other side. But I think that it is interesting that it goes beyond just that individual choice. It's not like the dynamic and the emergence of the structural echo chambers is not just because individuals are choosing it, but because their choices are impacting others, because it shapes what messages are shared and how the structure of the network is gradually built up.

• 56:06Sean Carroll?

  And is there room for the individual agents, as it were, to be affected by their success on the network? Is their audience captured? Do the LLMs learn to be more provocative so they get more retweets?

• 56:22Speaker 03

  Not in... So this is something that I think is really important and I think is really central to how social media is reshaping politics. Because it's not just that we get more polarized when we go on the platforms, but the platforms are kind of shaping the incentive structures that are shaping society because they are defining who gets attention and who doesn't. And so in that sense, it's reshaping a politics built around being outrageous and so on. But no. Briefly put, no. But this is like a paper that I'm currently working on, actually. So very much what I'm interested in.

• 56:58Sean Carroll?

  And of course, someone is going to say, "Look, LLMs are not people," right? Like, what worries about limitations of mimicking human behavior should we have in the forefronts of our minds when we're using these LLMs?

• 57:14Speaker 03

  Yeah, so this is... It's a really kind of big debate right now because there's a lot of excitement around this generative simulation. I've tended to be on the more skeptical side of this. So me and my co-author, Mike Leroy, we also have another paper coming out where we're specifically trying to answer this question. And like, how much can we trust this? And how much can we trust that they are valid as representations of human behavior? And it is kind of... It's a very interesting question to a certain degree because the models are more realistic as representation of human behavior than just a list of rules. But they're also much harder to validate, right? Because you don't know exactly what is playing out and it's very hard to calibrate their behavior to match human behavior.

• 57:58Speaker 03

  And so it becomes a difficult question. They're like ending up somewhere in between empirical methods that have a high level of validity and formal models that are parsimonious, that are easy to understand. And they're neither. And so it's a little bit unclear how we can use them. And the way that... We think about that in this paper is precisely it's linked to the fact that we're not looking at the text that they're producing. We're trying to look at kind of structural features that are emerging as aspects of the platform, not that, you know, like we're trying to distance ourselves from their behavior and try to look at the kind of structural outcomes of that behavior. And to a certain degree, the what we're interested in is the kind of the robustness of those of those outcomes.

• 58:43Speaker 03

  So in this sense, what do I mean by robustness? Well, basically, similar to the shelling segregation model, you can basically, it's the kind of joke among models that whenever you build a geographical model that has, you know, any type of lattice in it, you always get the shelling segregation effect. Because it's just, it's just such a stable outcome, and you have to really work hard to study any other effect. And so that's the kind of effect that we're interested in. We're interested in seeing how robust these emergent outcomes from the platform are. And so if we were just looking at the kind of toxicity of their conversation, I would feel less confident that these are, you know, something that are actually playing out in real world.

• 59:25Speaker 03

  But given the robustness of these emergent patterns, I have more confidence. But that being said, we definitely also need to, you know, do more studies on this.

• 59:35Sean Carroll?

  Well, it does seem like the robustness from shelling to your L L Ms, I mean, these are sort of different individual, small-scale dynamics, giving rise to similar large-scale behavior. It sounds like there should be a theorem, like the second law of thermodynamics, but about polarization or segregation or something like that being the state with lowest free energy. I don't know, I don't know what to call it. But are there theorems like that in the statistical mechanics of social media?

• 1:00:07Speaker 03

  So there are people doing these much more physicists approaches, kind of opinion dynamics. I have a colleague here who's basically doing icing models wrapped into icing models in order to to understand the social media. My approach tends to be a little bit more based on empirical data and trying to be closer to the system.

• 1:00:33Speaker 03

  So I think it generally is hard to find, unfortunately, this kind of loss in the social world because it is such an open system. And linking back to the question of whether society is a complex system or a complicated system, I would argue that it's neither or it's both to a certain degree.

• 1:00:53Sean Carroll?

  And one of the features of polarization that is a little bit weird to me in the United States is how even it is. Like, we've essentially always in the history of the USA had two political parties with roughly 50% representation each. It never goes to like 70/30 or anything like that. That's not something you can test in your model, right? I mean, you basically baked in into your initial conditions how many Democrats there were and how many Republicans.

• 1:01:24Speaker 03

  Yeah, no, this is not something I would capture in this model. In general, I'm thinking of these models as just trying to somehow capture one mechanism.

• 1:01:34Sean Carroll?

  Yeah.

• 1:01:35Speaker 03

  And as soon as you start having a lot of mechanisms, it gets very, very difficult. So in a certain sense of like how the outcomes of this interaction feed back into society and how it drives polarization, that's to a certain degree outside the bounds of this model.

• 1:01:50Sean Carroll?

  So you said that you were tracking the statistical results, not the individual words that the LLMs are putting into their posts or whatever, but... Do you know that those backstories you gave them, like, you know, this person lives in Boston, you know, they enjoy theater or whatever, did that matter? Did that affect how they sorted or polarized or succeeded in the social media game?

• 1:02:15Speaker 03

  I mean, to a certain degree, yes, because if I wouldn't have given them any sense of if they were Democrats or Republicans, they wouldn't be able to act as that and you wouldn't, you know, trigger this kind of feedback effect. But if it matters that they that they know if they go fishing or not, that probably doesn't matter.

• 1:02:35Sean Carroll?

  Yeah, I don't know. I would be curious to know.

• 1:02:37Speaker 03

  I think of that more as a kind of a little bit of noise or a little bit of perturbation to to make them not only act on the basis of their political personality, but that there's also like, okay, so this, you know, he likes fishing, you know, he's talking about fishing, I like fishing, I'm going to follow him. So it adds a little bit of that kind of noise and the fact that, you know, our our lives are not just politics. It's that's just a small part of of everything that we are. We have much more rich identities than that.

• 1:03:06Sean Carroll?

  And you injected news, basically? Is that like, you know, there was an external source of perturbations that said like, you know, this news event happened or whatever?

• 1:03:15Speaker 03

  Yes, exactly. So so basically, if you're just giving the agents, if you're just letting them talk without, you know, having them something to talk, giving them something to talk about, it just becomes the most generic, uninteresting conversation. And and it also doesn't create this this kind of richness that, you know, also functions as a kind of noise, you know? And so we basically focus on a certain day and we got all the news from that day and then we present them with like a random selection of those news and then have them discuss it.

• 1:03:45Sean Carroll?

  Okay, so you didn't totally make up the news. You actually were inspired by real news.

• 1:03:48Speaker 03

  Exactly, yeah, we we got real news from from a particular day.

• 1:03:51Sean Carroll?

  And so I guess you mentioned the power law distribution of attention. So some of these, they're all L L Ms, but some of them get a lot more followers than others. Is there any sense in which some of them are just better at social media than others? Or is it is it purely statistics and randomness?

• 1:04:13Speaker 03

  So I would say that it's pretty much stochastic. I mean, I wouldn't say that it's just randomly one of them happens to be a, you know, a great influencer as such. But there, I mean, being more political and being more extreme does help to become more influential, but it is, I would say, pretty much stochastic. And I mean, that fits also, there's been various like kind of experimental studies on how, on these power law distributions and how they can emerge on, in systems just through the feedback effects. And it doesn't need to be any difference between the things that are being selected. You still get these power laws and it's just kind of random.

• 1:04:52Sean Carroll?

  So did we learn anything about how to make the world a better place through doing this? Does this help us suggest any ways to make social media better?

• 1:05:01Speaker 03

  I mean, so I guess we didn't really mention the interventions that we tried out, because we built this platform and then, you know, we saw these negative outcomes and that gave us the baseline where we could try, okay, can we fix this problem?

• 1:05:13Sean Carroll?

  Right.

• 1:05:14Speaker 03

  That became the kind of next step. And so we looked at the literature and looked at what has been suggested, what are people kind of optimistic about in terms of trying to solve these problems? And basically, we had the kind of wide variety of more or less sophisticated solutions that have been presented. One of them was kind of the bridging attributes, which Jigsaw has released, which is a sub-company of Google, which is basically, they analyze the content of the, of messages and then you can sort your newsfeed. If you're a social media platform, you can use it to sort your newsfeed and you get the most kind of constructive comments.

• 1:05:58Speaker 03

  Those are the ones that you show. So instead of, you know, like instead of showing the most upsetting comments, you can show the most constructive, the most partisan. So it's called the bridging attributes. So that's one example.

• 1:06:12Speaker 03

  And another example was like is this smaller solutions is like hiding the biography, the little description of the agents when they follow each other. So they don't know if the other person is Democrat or Republican, for instance.

• 1:06:27Speaker 03

  And another is just sorting chronologically instead of, you know, instead of showing the most shared posts.

• 1:06:33Sean Carroll?

  Right.

• 1:06:34Speaker 03

  So basically, we try out a bunch of those solutions that have been suggested. But I should also say that we are doing like fairly extreme versions of it. That, you know, wouldn't necessarily be realistic to implement on a platform. So for instance, we show the, you know, one algorithm where we show the the least liked posts first.

• 1:06:53Sean Carroll?

  Sure, see how it works.

• 1:06:55Speaker 03

  Would lead to a really awful platform if you implemented in the real world.

• 1:07:01Speaker 03

  Because we want to see, you know, like the most extreme solutions, kind of.

• 1:07:05Sean Carroll?

  Yeah.

• 1:07:06Speaker 03

  But unfortunately, none of these solutions really fix the problems that we're observing. And some of them actually make matters worse. So for instance, the chronological timeline actually leads to more of a social media prism where where you get more extreme users get even more attention. And so it seems what we take from this is that these this kind of emergent phenomena seems to be very kind of rigorous to perturbations. That is, it basically, you know, a little bit like the Shelling segregation effect. It's a very robust emergent phenomenon.

• 1:07:45Sean Carroll?

  Yeah, and you don't want to have a social media network that just tells every user who to follow. Right? You need to give them some agency there.

• 1:07:53Speaker 03

  Yeah, I mean, it's also the question of if people are going to use the platform or not. But basically, I mean, to me, what this suggests is that this basic structure that we see across social media platforms where you have a network, you follow people and you repost things, that that tends to be linked to these problematic outcomes.

• 1:08:14Sean Carroll?

  Well, I guess that's where I was gonna go. You already sort of said this is hard to study, but is it something truly new, these social media things? We used to be happy just getting the conventional news on one of the three network stations on TV. And now we have a lot more variety in what we can listen in on. But has this had a big effect? Has it really, like, you can see that it has an effect, but I guess how much of our current political mess can we be tracing to this? I know it's a hard question to answer.

• 1:08:50Speaker 03

  Yeah, I mean, it's ultimately, you know, impossible to know. And to a certain degree, it's also a bit tricky treating social media as something that's like external to society and that happened to society. Because to me, that social media structure, the way it is, is very much an expression of, coming back to Fordism, the kind of transition from an industrial society to a post-Fordist society, where the focus of capital is to, is advertising and figuring out information about you. That was not, you know, catering to a mass market where, you know, everyone, you're selling the same product to everyone, but that you're really trying to not only identify consumer niches, but even create consumer niches.

• 1:09:33Speaker 03

  And that, like, that basic fact of how the companies make money, the business model underlying social media and the internet, that has very much shaped what social media has become. Of course, it's also feeding back, you know, but it's very difficult to say how social media would be different if it wasn't that context.

• 1:09:54Sean Carroll?

  Well, I guess it's the feeding back I was gonna mention very briefly. Like, it's not just that you have social media in addition to mainstream media, but the social media affects the mainstream media. They want those clicks too.

• 1:10:07Speaker 03

  Yeah, exactly. I mean, I think this is something that people often mention as like, "Okay, but we can just, you know, I can just stop using social media and I won't be affected by these negative consequences." But of course, that's not the case, right? Because social media is, as I mentioned, it's reshaping our politics and it's very much reshaping also mainstream media. So a student of mine in a student project in my course, last year, for instance, he looked at the New York Times headlines over time and then measured how clickbaity they are.

• 1:10:41Speaker 03

  And basically, what he saw was that when social media entered on the scene in like 2010, and so you saw a kind of jump and you saw that the New York Times also changed how they wrote their headlines. And I mean, that's just the kind of, there's one expression of it, but of course, it's reshaping, you know, the incentives of attention produced by these platforms are reshaping our politics, our media and our culture overall.

• 1:11:05Sean Carroll?

  And what does clickbaity mean? Is it a function of sort of giving less information and saying like, "You won't believe what happened next"?

• 1:11:13Speaker 03

  Yeah, there is like actually a bunch of kind of features of text that make them more or less clickbaity. But basically, the way he did it was to just look at the databases of clickbait news articles and then non-clickbait news articles and then train a classifier on it.

• 1:11:30Sean Carroll?

  So I guess a last thing to talk about is there's polarization. So you had these LLMs, they sorted themselves in a shelling-like way, et cetera. Can we say anything about the quality of the information, like the truthfulness versus misinformation? Or social media helping us not just only talk to people like ourselves, but to get it wrong by sharing miss and disinformation?

• 1:11:56Speaker 03

  So I would say, I mean, this is not something I'm looking at in this specific model, right? Because the, in part because the LLMs are under, you know, OpenAI doesn't let them produce misinformation, so you can't really use them to study that. But I have looked at this in the context of using actual social media. Social media data. And I mean, what I would say broadly around it is that social media is by removing the kind of gatekeepers that we used to have from mainstream conventional media. They're also and by creating kind of really strong incentives for gaining attention and shaping the kind of conditions, but that they're really producing conditions where

• 1:12:40Speaker 03

  not anchoring what you're saying to, to, you know, truth becomes a kind of beneficial strategy for gaining attention, right? Because you're both, it allows you to be kind of outrageous. It allows you to trigger people and you're not really, you know, constrained by, by reality in the same way.

• 1:12:58Speaker 03

  And then, and of course, that also becomes interconnected with politics. So I had a paper coming out with my co-author Julian Schweri earlier this year where we look at politicians across countries. And we look at their Twitter posts over a five, six year period. And we look at all the examples of when they're shared links. And we identify misinformation through that. And so we, we can link each politician to their likelihood of sharing misinformation. And then we can basically use that for a kind of comparative model, so basically a statistical model to identify the conditions when politicians are spreading misinformation.

• 1:13:42Speaker 03

  And so this links to this broader question of the link between social media and the spread of misinformation, which has, there's been a big debate around this, especially in the last few years about

• 1:13:54Speaker 03

  whether it's just social media reducing the quality of information overall. Right. And what we argue is basically, it's not just that, but that social media becomes intertwined with politics, it becomes intertwined with different political movements. And the result is that certain political movements are emerging shaped by the interests and incentives of social media in such a way that they use misinformation as a political strategy to gain advantages in, in political competition. And what we find in that study is basically that it's specifically the radical right populist parties that are driving this rise of misinformation.

• 1:14:33Speaker 03

  So it's not just a social media phenomenon in itself, but it's social media intertwined with politics and political systems.

• 1:14:41Sean Carroll?

  So I'll let you give like a last big picture kind of thought here like, am I getting the impression that social media are just bad? That it was a mistake? That their net effect is negative? Or can we have some shred of optimism to hold on to?

• 1:14:57Speaker 03

  I mean, I think that there are also kind of to some degree positive outcomes from it. And like for certain communities can be beneficial. And I mean, it's a certain degree, like growing up, I love the internet. It was great. I grew up on this, you know, in the countryside on this little island in Sweden, in the middle of nowhere and like the internet kind of provided a social world for me and allowed me to, you know, connect to ideas and everything. So and to a certain degree, what I would hope for is also kind of going back to that, you know, innocent era of the 90s, of the internet of the 90s. Like, you know, I was using ICQ and you had this button where you could click and talk to like a random person anywhere in the world.

• 1:15:41Speaker 03

  And I just like love that. I spend my days like talking to some, you know, some random person in Arizona. And it's a certain degree, you know, I mean, of course, that was like a more innocent time. And maybe if we would try to bring that back, it would, you know, lead to something horrible these days. But I do think that, you know, we could create structures, like we could create platforms and spaces that would actually be, you know, beneficial for us and that would actually be positive. It's just we might need to rethink it in more fundamental ways than just this cosmetics changes to algorithms or designs.

• 1:16:17Sean Carroll?

  All right. That is something this homework out there for all the young people. Think about, you know, fundamental changes we can make because they're not going away. You know, even if social media are net bad, they also are good. And we're going to have to live with both of them. So Better Torenberg, thanks very much for being on the Mindscape Podcast.

• 1:16:33Speaker 03

  Thank you so much. This was really great.

• 1:17:00Speaker 00

  From kickoff to the final whistle, trade the biggest moments in football on Kalshi, America's number one prediction market platform. Kansas City takes on defending champions Seattle, currently trading at 41% to win. A $100 trade pays out $244 if they come out on top. Download Kalshi, use code HOOPS to get $25 when you trade $25. K A L S H I. Kalshi, trade what's next. 18 plus only. Restrictions and eligibility requirements apply. Event contract trading involves risk and may not be suitable for all investors. Kalshi products are not available in all jurisdictions. Prices, values, and available markets may differ from those mentioned. For more information, see kalshi.com/ regulatory.

• 1:17:36Speaker 05

  John Sealy here. Some people say I got range. Comedy, action, drama. You know what else says range? Chime. You get fee-free banking and 5% cash back on everyday purchases. Easy credit building, high yield savings, premium travel perks with Chime Card. Oh, did I say fee-free banking? Open a Chime account in minutes to switch today. America's number one choice for banking. Chime.

• 1:17:58Speaker 06

  Chime is a fintech, not a bank. Banking services and Chime Card provided by Chime's bank partners. Cash back on eligible Chime Card purchases. Terms and limits apply. Go to chime.com/disclosures for more details.

• 1:18:07Speaker 07

  Most people overpay for car insurance, not because they're careless, but because switching feels like too much hassle. That's why there's Jerry, your proactive insurance assistant. Jerry compares rates side by side from over 100 top insurers and helps you switch with ease. Jerry even tracks market rates and alerts you when it's best to shop. No spam calls, no hidden fees. Drivers who save with Jerry could save over $1,400 a year. Switch with confidence. Download the Jerry app and see what you could save today.