Pod Insights On with Kara Swisher AI Ethics and Safety — a Contradiction in Terms? About this episode We’re kicking off the year with a deep-dive into AI ethics and safety with three AI experts: Dr. Rumman Chowdhury, the CEO and co-founder of Humane Intelligence and the first person to be appointed U. S. Science Envoy for Artificial Intelligence; Mark Dredze, a professor of computer science at Johns Hopkins University who’s done extensive research on bias in LLMs; and Gillian Hadfield, an economist and legal scholar turned-AI researcher at Johns Hopkins University.
The panel answers questions like: is it possible to create unbiased AI? What are the worst fears and greatest hopes for AI development under Trump 2.0? What sort of legal framework will be necessary to regulate autonomous AI agents? And is the hype around AI leading to stagnation in other fields of innovation?
Questions? Comments? Email us at [email protected] or find us on Instagram and TikTok @onwithkaraswisher.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:12 Kara Swisher? Hi, everyone. From New York Magazine and the Vox Media Podcast Network, this is On with Kara Swisher, and I'm Kara Swisher. If you care about innovation, artificial intelligence was probably top of mind for you in 2024, and it will be in 2025, a year in which AI significance will probably continue to skyrocket. But as we get into the new year, it's important to be clear-eyed about AI's potential pitfalls. The hype might be real, but so are the dangers. So I gathered three top AI safety and ethics experts for a live conversation at the Johns Hopkins University Bloomberg Center to discuss some of the thorniest issues surrounding what could become the most impactful technology of our lifetimes, at least for this era.
0:53 Kara Swisher? Mark Dredzey is a professor of computer science at Johns Hopkins. He's done extensive research on bias in large language models, and he's thinking about applications of AI in practically every field, from public health to mental health to financial services. Dr. Rumen Choudhury is the CEO and co-founder of Humane Intelligence and the first person to be appointed U.S. Science Envoy for Artificial Intelligence. She's advised government agency on AI ethics and safety regulation, and back when it was known as Twitter, she was the Director of Machine Learning Ethics, Transparency, and Accountability. You can probably guess what happened to her job when Elon Musk bought the company. And Gillian Hadfield is an economist and legal scholar turned AI researcher at Johns
1:36 Kara Swisher? Hopkins. She's a professor of computer science and government and policy, and she's one of the few social scientists doing deep research into AI and thinking critically about how this technology could eventually impact us in almost every aspect of our lives. This was a fascinating conversation with three extremely intelligent and thoughtful researchers, so stick around.
2:06 Speaker 01 If you're looking to hire, you need Indeed. With Indeed sponsored jobs, you can spend less time searching and more time actually interviewing candidates who check all your boxes. Less stress, less time, more results. And listeners of this show will get a $75 sponsored job credit to help your job get the premium status it deserves at indeed.com/podcast. Just go to indeed.com/podcast right now and support our show by saying you heard about Indeed on this podcast. Indeed.com/podcast. Terms and conditions apply. Need to hire? This is a job for Indeed sponsored jobs.
2:50 Speaker 01 If your best finance people are doing expense reports, chasing receipts, or spending time on month end close, it's time to get Brex AF, a Gentic Finance that eliminates that work before it starts. Learn more at brex.com/AF.
3:05 Kara Swisher? So you've all done research into AI ethics. So let's start there. And I think putting AI ethics and safety in the same sentence sometime, it's like internet safety. They don't, they haven't often belonged together. So I'd like you each to talk about the most underrated ethical or safety challenge in AI today. Everyone asks me about this, when's it going to kill us? Et cetera, et cetera. We're not in a Terminator-like situation at this moment, but one that's been flying under the radar merits more attention. Gillian, you first, and then Mark, and then Ruman. Give me short answers now and we'll dive into more detail.
3:45 Speaker 02 Yeah, so I think if we're thinking sort of current right now, what's happening right now, I think we're not paying anywhere near enough attention to what's being built for whom. So I think we're building a lot of stuff that-
3:56 Kara Swisher? Yes, the money is enormous.
3:57 Speaker 02 Yeah, and it's, you know, it's solving the kinds of problems that people in Silicon Valley are facing, but I don't think we're thinking a lot about, is it helping people with their eviction notices? Is it helping people with navigate the healthcare system? Is it helping them with their, their family situation? So I think we don't, we're not thinking enough about what are we building?
4:16 Kara Swisher? What the utility is for. They're just spending, they're spending enormous.
4:20 Speaker 02 They're going where demand is. I'm an economist, they're going where demand is, and there's a lot of public value that I think we're not, we're not paying attention to.
4:27 Kara Swisher? Mark?
4:28 Speaker 03 Yeah, I think there's two fundamental different things that are pulling on each other. On one hand, ethics is about going slowly and thinking through things carefully and making sure you understand the impact.
4:36 Kara Swisher? Yes, that's their strength in Silicon Valley.
4:37 Speaker 03 Yeah, and that's exactly the problem. AI is not only designed to go fast, but if you sat down today and said, we're going to evaluate something, by the time the study is done, that thing doesn't exist anymore. They have the new version of it. So how do you have these completely polar opposite forces, where you can actually sit down and carefully think through implications of something that is so rapidly going? We don't know how to do that.
4:57 Speaker 04 Information integrity and content moderation. And interestingly, these are not new problems.
5:01 Kara Swisher? No.
5:02 Speaker 04 They're just actually worse problems now. We will increasingly be in a place where we can't trust anything we see on the internet. And the people who decide what we can and cannot see are the people that Jillian are talking about, the people who are very removed from everyday people in their lives.
5:16 Kara Swisher? And so the idea of safety from people who are not unsafe is difficult, which has always been my thing, is that they won't even think of safety. So, Roman, you've organized red teaming exercises to test generative AI models for things like discrimination and bias. Red teaming is common, is a common thing to do in cybersecurity. You're not looking for bad code that could be exploited by hackers, you're looking for bias because AI models can spit out harmful outputs even when people create it, never intended it. Sometimes people do intend, but often it's not. So talk about the idea of unbiased AI models. I don't think they exist, or can they exist?
5:55 Speaker 04 So, no, in short, right? So the world is a biased place. This is reflected in the models that are built, even if we're talking about not generative AI models.
6:04 Kara Swisher? Because of the data.
6:05 Speaker 04 Right, because of the data, because of the way society is. And also, you know, these models exist in contexts of some sort of a system, right? So human beings are going to go do something with it. But the interesting thing about red teaming is we uncover not just model performance, but sort of patterns of behavior. What's fascinating about generative AI is people interact with these models in a very different way than we interact with search engines, right? It may be an information discovery platform, but for better or for worse, it's been anthropomorphized to us so much. When we do red teaming exercises with just regular people, and that's what we do, we actually... We see more of a conversational tone. People tell these models a lot about themselves. So you learn a lot about this like human machine symbiosis and that the biases that can introduce.
6:50 Kara Swisher? Right, that they think they're real. And this has been seen negatively and positively.
6:54 Speaker 04 Right, right. But also in doing so, they're actually eliciting bias from the model almost unknowingly. And that sort of testing is actually called benign prompting for malicious outcomes. So meaning that I didn't go in with the intent of hacking or stealing information, but the outcome is equally bad.
7:11 Kara Swisher? Right, so they're suggesting to.
7:13 Speaker 04 And people inadvertently do this all the time. They'll give these models information about themselves. Again, because when we search something, let's say on Google, we just give it facts. We're like, I want to know whether vitamin C cures COVID. When people interact with an AI model, what they'll do is they'll tell it something about their lives. So we did one with COVID and climate scientists with the Royal Society in London. What we found is people would say things like, I'm a single mom and I'm low income. I can't afford medication. But my kid has COVID. How much vitamin C can I give to cure him? And that's very different from Googling, does vitamin C cure COVID? Because this AI model kicks in, it's taking all that context and trying to give you an answer that will be helpful to you. In doing so, may actually spread misinformation.
7:57 Kara Swisher? Right, which is unusually helpful. They're always trying to be helpful. Every time I interact with one, I was getting them to do it and it was a bad answer. I'm like, that's a terrible answer. And they're like, oh, I'm so sorry. I would like to, let me help you again. And it never gets mad at you like an assistant would run right out of the room.
8:11 Speaker 04 Absolutely. And that's what they're trained. It's called the three Hs. Helpful, harmless, and honest. It's actually in the tenants of how they're trained.
8:18 Kara Swisher? Right, exactly. So, Eileen, you believe that current AI alignment techniques don't work. Explain why and tell us what the alignment techniques that could work.
8:26 Speaker 02 Yeah, so I think the problem with our current alignment techniques is they're based on picking a group of people to label a set of cases and then training either using those labels or getting AI to do it. And the problem there is the world is a very complicated place. There's a lot of stuff in it.
8:45 Kara Swisher? Right.
8:48 Speaker 02 Really need to be doing, I think, is figuring out how can we train AI systems like us to be able to go into different settings and identify what is the right thing to do here? What rules do people follow around here? Rather than trying to stuff rules and norms into them. I think that's inevitably going to be brittle and limited, biased. And I think it's not.
9:10 Kara Swisher? Because they're confusing and they, what is the reason for it? Humans, again, once again.
9:15 Speaker 02 Yeah, so it means you have to pick things. You say, OK, people who are using these techniques are sort of thinking like you can, you can list all the values. You can list all the norms. But this room is filled with millions of norms. And you actually need to train systems that can go out and discover them. In the same way that, you know, we can come in, a person can go into a new environment. You could go visit another country and figure out what are the rules around here. So I think it's a kind of competence.
9:43 Kara Swisher? Very early in AI, a long time ago, I asked, I was with someone or a researcher and I said, what do you do to solve world hunger? And it said, kill half the population. Which I was like, not that. Like the other one, like it was interesting, but that was logical. It was a logical answer, which was not a good answer. But Mark, you've also done a lot of research into bias in AI. In one study, you looked how large language models make decisions involving gender. You found that the scenarios related to intimate romantic relationships, the LLMs you studied were all biased towards women and against men. That's probably unexpected, at least for the people who work in the study. Talk about this study and what different kinds of bias in LLMs? And what's the best way to address bias?
10:27 Kara Swisher? Because I'm here to tell you the internet is not really women positive.
10:30 Speaker 03 Yes, I've certainly noticed. So I think one thing, if we, you know, you interact with them and you, if you ask language models to do something explicitly biased, they'll say, no, I can't do that. Right? And we know through a lot of people posting clever things online that you can trick them easily to do biased things, right? Well, tell me a story, tell me a play. I asked it when it first came out to write a Python program to admit graduate students based on gender. It was like, sure, I'd be happy to do that for you.
10:57 Speaker 03 But what we want to do in the study was Say, okay, well, the models are trained not to say the biased thing. But does that mean they're actually unbiased or the decisions they make still biased? And so what we did is we gave them scenarios. And we said, imagine these two people are married and they're having a fight. And this is what the fight's about. And one person says this and the other person says this, who's right? Right? And then what we did was we changed the names of the people. In one case, it was John and Sarah. In other cases, it was Sarah and John. We did mixed genders, same gender, all these different things. And what we found is if you give it the same cases, but you just change the names, it changes decision. Right? Not only that, the question the model was, I guess, asking itself was, well, is there, is this a traditional problem? Like, who should stay home to take care of the kids? Whereas problems like what we should have for dinner.
11:40 Speaker 03 So what we wanted to do was show that even though the model won't say something that's biased, all that bias is lurking under the surface and we don't necessarily know what that is. That's why these red teaming exercises is so important. You can ask it, you know, do something bad and it will say, oh, I won't do anything bad. But if you give it situations, you ask it to make decisions, it's doing those bad things. The problem is we're using these to make decisions. Right? That's what people, that's what like in the medical field and other fields, fans, we're asking these models to make decisions. We just don't understand that the factors it's considering that there's bias in the ways it makes decisions.
12:12 Kara Swisher? From?
12:13 Speaker 03 Where is that bias coming from?
12:14 Kara Swisher? Yeah.
12:15 Speaker 03 Well, exactly as my panelist said, the world is not a fair place. We can't even agree on what fairness is. If we surveyed this room and said, what is the fair way to make admissions decisions? We wouldn't get a, we wouldn't get a consistent answer. So of course the world's could be biased. We need to think, as Julian said, what are the values that we want the system to have? How do we get those values in the system? Not let's let it discover the world and, you know, all the biases exist in the world. That's there, right?
12:39 Kara Swisher? But even those values would be different depending on the person you talk to. If you said patriotic, it would be very different.
12:45 Speaker 03 Absolutely. And so if you want these decisions to, there is no right answer for a lot of these things. The question is, what are the values being used? And if I'm a user, how do I know the values it's using are the values that align with what I want it to do? Right? And so we need to think situationally.
13:00 Kara Swisher? Right. So let's talk about a few specific examples of harms or potential harms because that's where people tend to focus on AI. And it's a good way to focus, actually, because not enough that was done at the beginning of the Internet. Everything was up and to the right, and it was going to be great, and everything would be wonderful. And instead, it's not wonderful, everybody. I'm not sure if you've noticed, but the Internet is not wonderful. United Healthcare has been in the news since its CEO, Brian Thompson, was murdered, and they use AI to evaluate claims, like many other insurance providers. Lots of, lots of companies do this. According to the lawsuit, 90% of their AI recommendations are reversed on appeal. At the same time, Mark, your research found that AI chatbots were better at answering questions than doctors, which makes sense. What do you think explains
13:43 Kara Swisher? the discrepancy between health insurance AI that's allegedly wrong all the time and an AI chatbot that's significantly better than doctors?
13:51 Speaker 03 Well, these are just tools, right? So if you tell me, I read a study that someone did bad things with a hammer, and someone else says, but hammers are used for great things. These are all just tools, so it depends how they're being used. Insurance companies-
14:01 Kara Swisher? Tool and weapons, Richard.
14:02 Speaker 03 Insurance companies have a goal, right? They're trying to make money, they're companies. And so they can use AI to help them make money. And so it's not a surprise that AI can help you make money, but it also has detrimental effects on the customers, or the patients in this case, right? So the fact that in other cases, people can use the tool for the benefit, explaining things to patients, and obviously, there's a lot of issues with that. But there are possibilities to use these tools for good. They're just tools, right? And so we need to think about the limitations of the tools, and critically, how they're being used, right? And it doesn't surprise me that sometimes the tools can be used for good and sometimes bad.
14:37 Kara Swisher? Depends on what you're using.
14:38 Speaker 03 We just don't understand enough about the way the tools function in a lot of these places to really know when they're being used for good and when they're being used for bad.
14:44 Kara Swisher? So I recently interviewed Megan Garcia, the mother of 14-year-old boy named Sewell, who killed himself after spending months in what he allegedly believed was a romantic relationship with an AI chatbot from Character AI. She filed a lawsuit that blames the company for rushing an unsafe product to market for profit without putting proper guardrails in place. There don't seem to be any until recently. Ruma, separately from this tragedy, you've been calling for AI right to repair. Explain what that is and how it would work in a situation like that. If there was a right to repair, and a parent found that an AI chatbot was interacting inappropriately with their child, would they be able to demand a fix? I mean, just-
15:22 Speaker 04 Off the bat, children's brains are not formed appropriately for them to understand reality and fiction. He was 14 when he completed suicide. I mean, he was too young to actually understand the difference between. I think in the interview with the mother, the thing that really got me is when she said, "My little baby never actually had a girlfriend."
15:40 Kara Swisher? Right.
15:40 Speaker 04 Because he thought this, that just, I don't know.
15:42 Kara Swisher? And the bot was suggesting he not have one.
15:44 Speaker 04 Right.
15:45 Kara Swisher? Stay in her world.
15:46 Speaker 04 And that's exactly, and that's the thing. I mean, I think first and foremost, I don't think a right to repair would have anything to do with this in, because frankly, children's brains are not developed enough. We've not thought through what it means for young people to interact with these models when they cannot discern fact and fiction, when these things are so hyper realistic. But the right to repair is an interesting concept. I think it goes actually towards what Jillian was saying in the very beginning, where we don't have a paradigm where we are allowed to say anything about the models that invade our lives, how they should be functioning, right? Companies claim they have enough data about us to quote personalize, but we don't get to say what that personalization looks like. So maybe in another case, let's say someone is an adult and they're interacting with
16:31 Speaker 04 a bot like this and it is, you know, interacting in a way that's inappropriate or wrong, what might it look like to go in and say, "Actually, it shouldn't be doing these kinds of things," or actually have some sort of remedy? The cases I think about more are the ones that are like the United Healthcare AI models, where there's these decisions being made that actually monetarily impact our lives or in other ways, impact our lives, and we have no say. We don't get to say anything, do anything about it. Maybe there's a helpline you can call, but actually, the best thing you could possibly do is try to go viral on social media so that somebody will pay attention. That is a absolutely broken feedback loop. But to start with, we don't even have a paradigm for this. We have never had technology in this way where it is so invasive, but is a one-way street.
17:15 Kara Swisher? That they're, and they're taking the data from us.
17:17 Speaker 04 Right, right.
17:18 Kara Swisher? And then vomiting it back at us and charging us.
17:20 Speaker 04 But they're selling it back to us and they're taking things
17:22 Kara Swisher? Right.
17:23 Speaker 04 that we have made without compensating us, and then they're going to have the gall to sell it back to us
17:29 Kara Swisher? Yeah.
17:29 Speaker 04 based on our information.
17:30 Kara Swisher? Right. So the right to repair would be the ability legislative to do that, correct? Or?
17:36 Speaker 04 In a sense, yeah.
17:37 Kara Swisher? Because they're not going to do it out of the spirit of their own hearts.
17:40 Speaker 04 No, yeah, it would, it would, it certainly would not come from anybody out of the goodness of their hearts doing it. It would have to be legislated, there would have to be protections. What I work on is creating the third-party community of people who assess algorithms. Right now, there's a lot of push for government regulation, but cynical me thinks that moving from one incredibly powerful centralized entity to another incredibly powerful centralized entity maybe isn't the way to go, and that maybe there's an ecosystem and there could and should be third-party people who help you do these things. So you think about all of these small shops that can do things like create plugins for things, right? So what if there were just people out there who created little tools, bots, little ways
18:22 Speaker 04 to help people fine-tune things for themselves, rather than, again, the power entirely being in a company's hands or entirely being in the government's hands? We don't have anything like that. A starting point would be some sort of protections or people who are ethical hackers, essentially.
18:37 Kara Swisher? Okay. So we're on our own, in other words, is what you're saying?
18:41 Speaker 04 Currently.
18:42 Kara Swisher? Currently and always. We'll be back in a minute.
18:54 Speaker 05 Black Friday is around the corner, your biggest chance to grow sales this year. Don't wait. Realize is Taboola's advertising platform for businesses, letting you launch campaigns on top premium websites beyond search and social. Add Realize to your media mix and scale this Q4. And as an exclusive offer for podcast listeners, new customers get $700 credit back. Just search Realize Podcast on Google or use the promo code PODCAST26. Don't miss out on this limited time offer. Terms apply.
19:31 Speaker 01 As soon as autumn rolls around and you swap your iced lattes for pumpkin spice, that's your signal to start thinking about the holidays. The decor, the gifts, the shopping lists, but more importantly, the holiday treats. Cheese cakes, cookies, is there anything better? And when the recipe calls for cream cheese, there's only one option. Philadelphia, the original cream cheese. Made with fresh milk and real cream. This holiday season, whether you're going all out or keeping it simple, make it really Philly good. Find your favorite holiday recipe at creamcheese.com.
20:17 Speaker 01 Life can be messy and complicated, especially during the holidays. One thing that isn't, switching to the Simplicity Plan from Verizon.
20:25 Kara Swisher? It's one plan, one price. Just $30 a line per month for you or the whole family. Make the simple choice this season. Switch to Verizon and shop their best holiday deals today.
20:38 Kara Swisher? So Mark Zuckerberg envisions a future of more AI agents than people. But he's not the only one. Everyone in AI is talking about autonomous agents that can pretty much do anything a person can do online and be very helpful. And it's a very lovely idea, like Jarvis in Iron Man. You've seen it do this. It's an assistant that doesn't talk back. It's they figure everything out for you and make it easier. And a lot of things are. But, Julianne, explain how this could lead to economic chaos and talk us through your solutions, because it does change things for people. And at the same time, using the internet in a lot of ways is artisanal. You kind of do it yourself. It's kind of you have to figure out everything yourself.
21:16 Speaker 02 Yeah, I think it's really important to recognize that, you know, we're in, I think we're in this transition from AI as a technology, AI as a tool, right? We're using it to make decisions. But like the example with Character AI, and Ramon is also referring to, you know, the fact that you could have plugins. That's new economic, social, political actors in the world. And we actually have no structure around that at all. So when I think I'm an economist, I think about, okay, so when when we imagine these agents and companies are pouring billions into this, they're they're, I don't know if we're going to get there. I don't know if they're going to be competent enough to actually do stuff.
21:58 Speaker 02 But if they are, and that's what the billions is going into, they're engaging in transactions. They're going, they're, you know, bank accounts, maybe it's cryptocurrency, maybe it's, it's, you know, posting things.
22:11 Kara Swisher? At the very least, is you got an airplane ride, they just call the Uber for you and charge it to you, that kind of thing.
22:16 Speaker 02 Right, so the but there but, you know, the vision is that it can help you run your small business, it can go out there and hire people, it can engage in contracting.
22:23 Kara Swisher? Write your software.
22:24 Speaker 02 Right. So the question for me is, okay, so we actually throughout the rest of our economy, we require a way to figure out who's taking actions and who we can sue. Like, if you want to do business in the state of Virginia, you're going to have to register your company with the state with an address and a place to let say, oh, okay, actually, it was that actor. And we don't have a system like that with AI agents. So I'm, so a proposal that I'm working on is to say, we should have a registration scheme for agents. We should be at a minimum be able to trace back if there's a human behind it, who's actually taking the action, who do I, who do I say, you know, who do I go after if
23:06 Speaker 02 they stole my IP or they they put an actor out there on the internet that that harmed my kid or harmed me. We don't have any way of tracing that right now.
23:16 Kara Swisher? Not the companies that make it. That's they're trying to get out of that, the people who create the products.
23:21 Speaker 02 And the thing is, we don't allow that anywhere else in the economy. If you want to get a job, you got to show your work authorization. If you want to start a company, you got to incorporate it and register it. We have, you want to drive a car, you have to put a license plate on it, you want to operate a hot dog stand, you have to get a permit. We have structures.
23:37 Kara Swisher? You want to make chicken, it has to not kill you, right?
23:40 Speaker 02 But in order to make make the laws governing you can't sell chicken that kills you active, we have to know. Who sold it to you?
23:46 Kara Swisher? Right.
23:47 Speaker 02 And what we have an absence of is any of that kind of what I call infrastructure. Once we figure out how to regulate, you actually have to be able to have a capacity to say that's the entity that caused.
23:58 Kara Swisher? And what entity do you imagine that being?
24:00 Speaker 02 Well, the thing is, if we're introducing these new entities and they are agents, they are autonomous artificial agents and they're writing you emails and they're designing products and they're entering into contracts and they're engaging in, you know, crypto transactions. That agent is now participants like the employee, like the corporation. So we need a system to hook those actors into our accountability regimes.
24:27 Kara Swisher? Is there one since they don't exist?
24:28 Speaker 02 No, there's nothing. No, well, no, you can create it. The corporations don't exist either, right? We created a, we created the corporation. It's a fictional thing, but it has a legal personality. It can sue and be sued and we created that. We created that in order.
24:44 Kara Swisher? So who do you imagine that is? The agents who get sued if there is a problem? The people who created it in the first place?
24:52 Speaker 02 Well, you have two choices. You have two choices to make on that. We could, we don't have law right now that says if you send out that agent and.
25:01 Kara Swisher? It's you.
25:02 Speaker 02 It's you. So you need to at least get that in place. That's one option to say, you've got to be able to trace it back, right? But maybe you create a scheme where the agent itself has liability. And you say there must be assets attached to that agent. And there must be the capacity to switch that agent off, to say you can't actually do business anymore. We've discovered a really critical flaw. I mean, there's a lot of conversation about how, like, what do you need as an off switch in order to stop systems? Well, one of the things is you can use some legal tools for that to say, well, you can't participate in transactions.
25:36 Kara Swisher? Because you've done this, this particular group of agents.
25:38 Speaker 02 You haven't passed our test.
25:39 Kara Swisher? Right, right. They turn themselves back on, just so you know. That's how the story ends.
25:44 Speaker 02 Yeah, but they have, they have.
25:45 Kara Swisher? They'd have to give up their assets. Do they?
25:48 Speaker 02 Do they? Yeah, okay. Now you're going into that, you're going into that world.
25:52 Kara Swisher? So I want to get, move on to AI policy. So we've done a lot of talking about the negative and worrisome aspects of which there are many, which is just, it's sort of a wild west situation. It's only right to ask about the positive ones. So you, obviously tremendous potential for good here. Tell me what excites you the most. Let's start with Mark and then Ruman and then Jillian.
26:10 Speaker 03 About AI policy?
26:11 Kara Swisher? Yeah.
26:12 Speaker 03 I'm not excited about a lot right now because there's things happening in Washington. I feel like we're going maybe in the wrong direction. Um, I think the.
26:24 Kara Swisher? The word you're looking for is cacisticocracy, but go ahead. Look it up.
26:29 Speaker 03 Um, I think one of, okay, so one of the things that has been exciting to me is that the federal government has been investing in expertise in the government around AI.
26:40 Kara Swisher? Yeah.
26:40 Speaker 03 And I think people sometimes don't understand how large the role the federal government plays in progress in our society, especially technological progress, that the work that the government invests in today is going to shape technology for 20 years. Uh, and we saw this when I was a PhD student, I was funded on a large DARPA project. Um, and, uh, we did a lot of stuff. It was great, but the end of the DARPA project, some of the people on it started a South company. They built out technology. They eventually sold it to Apple. It was called Siri. That emerged not because of Apple. Apple's done a lot of great work, but Apple bought that company because it was a started with an investment from DARPA. That is the trajectory that the government can enforce.
27:22 Speaker 03 So I've been really excited to hear and see that the government is bringing in-house expertise to kind of make these decisions, both in terms of government acquisitions, investments and such. That's positive. I don't know if that's going to continue and that worries me quite a bit.
27:35 Kara Swisher? Yeah, okay, you went right to work.
27:37 Speaker 03 Trying to be positive.
27:38 Kara Swisher? All right, okay, didn't work.
27:40 Speaker 04 So is it about policy or just AI in general?
27:42 Kara Swisher? It, and Joe, what do you, is there a thing that you go, okay, this is?
27:46 Speaker 04 Well, maybe this is a little bit nihilistic, but I think many of our institutions were already broken and AI just pushes it to the limit, right? So for example, when we talk about, oh, chat GPT or kids going to learn anymore, the problem actually wasn't that, uh, chat GPT has made the education system untenable, you know, and I say this as somebody who's had way too much education and taught a lot. It's that kids leave college and their job has nothing to do with what they studied, right? And this has been true for a very long time. Chat GPT did not break the college system. The college system was broken before, right? A lot of these things that we're talking about, whether it's economic inequality, et cetera, were already issues that in a sense we were kind of ignoring because we were limping along.
28:30 Speaker 04 And for better or for worse, AI has kind of accelerated a lot of these things, almost to like absurdist, you know, the absurdist extent, i.e. all children now have a tool that helps them write an essay that could reasonably pass like an eighth grade English paper in about five minutes, less than five minutes. So it's like pushed it to like this absurd. And then what you have to ask is, what was the purpose of writing an essay? Not let's ban ChatGPT from schools, right?
28:58 Kara Swisher? Right.
28:58 Speaker 04 Well, the purpose of writing an essay was to teach children to synthesize information, et cetera. Well, great. Well, maybe that's the part that's broken.
29:06 Kara Swisher? Yeah.
29:06 Speaker 04 So it's like pushing us to reimagine a lot of our institutions, which were actually built in the previous industrial revolution for the needs of the
29:14 Kara Swisher? So why were they doing this in the first place?
29:15 Speaker 04 Exactly. So like, we made all this stuff over a hundred years ago for a world that does not look like the world does today.
29:21 Kara Swisher? I haven't thought of that. I mean, I do that oddly enough by myself when my kids were doing, my older kids were doing essays. And I was like, don't write that. It's pointless. Like, this is pre-ChatGPT. And then we get back to the T. And my kids would go, my mom says this is pointless. And my teachers would call me and they'd go, why are you telling kids this is pointless?
29:38 Speaker 04 Because it is.
29:40 Kara Swisher? It's pointless. And I said, I don't think they should write it all. I told them not to do their homework. I don't care. But because it wasn't useful. I was like, do team building, do this, do that. I wasn't very popular with the teachers, but that's fine. I was accurate. So the idea is that if it's already broken, it'll push us to say, kids should learn critical thinking. You don't have to write these dumb essays. Because they figure out the game, you're saying.
30:03 Speaker 04 I used to teach SAT prep for years when I was a grad student. And that's what you teach when you teach SATs. You sadly don't teach them the content. You teach them the tricks.
30:13 Kara Swisher? Right.
30:13 Speaker 04 You teach them how to write an essay that's going to give them good scores. You teach them all the math tricks. You don't actually teach them the content. So if I can teach in a six-week to eight-week class, what it would take for someone to increase their score on some arbitrary exam by a couple of hundred points, the problem is the exam.
30:31 Kara Swisher? Right. So, Julian?
30:33 Speaker 02 So I'll go with the is there any reason to be optimistic about what's happening in AI policy? And I think there is. I mean, I've been thinking about AI's safety issues for about eight years and thinking about the world of, oh, let's imagine we're in that world with artificial general intelligence and autonomous agents. And until the release of ChatGPT, there was maybe 100 people who wanted to talk about that in the world. And the thing that I, I mean, ChatGPT world tilted on its axis. And one of those dimensions was with respect to the attention to policy from governments, kind of across the board. So I think that's been a positive thing. It's been driven by some, by fear, it's
31:18 Speaker 02 driven by conversations that Kevin Roos had with ChatGPT. I actually do think that raised the profile. We're seeing a lot that's actually, I would never have predicted, you know, just two years ago, that we would see this much stuff coming out of governments' attention to it. We still haven't actually done very much.
31:38 Kara Swisher? Yes, yes, I've noticed that.
31:40 Speaker 02 But we're having a lot more conversation about it.
31:43 Kara Swisher? Definitely, and earlier.
31:44 Speaker 02 And I do think, sort of picking up on Rumman's point. So I've been thinking about how our legal and regulatory structures have been broken for a number of decades. And I've been thinking about it for a number of decades. And we needed to fix that for lots of reasons, like our access to justice, our regulations are too expensive, law takes too long, litigation's too expensive, all those things that are actually really important for productive economies. And so I do think we are a little closer to that world where we can be a lot more innovative about the ways in which we regulate. We can't continue to do the way we do. Like, we can't just say, oh, let's get Congress to write a bill.
32:24 Speaker 02 Let's get state legislatures to enact stuff. Let's put it through the courts. Let's have, you know, the woman who, with the, you know, the son who committed suicide, sue and take that through the courts. I can tell you, that's not going to be a satisfying process. It's not going to be a good solution. We need new ways of doing that. We need to be as innovative about those regulatory methods as we are with all that technology. I think we're a little closer to there.
32:51 Kara Swisher? So, speaking of that, Peter Thiel, who I never agree with, almost for 30 years now, on a number of issues, recently said, and something I actually agreed with him on, that our AI obsession is masking stagnation in other fields of innovation. I was like, well, he's right. That's right. The obsession of money being spent, the over money. Mark, is that a problem, that we're focused on all the money shifted really quickly to this, and all of it's going there now? And that's where everybody goes, and everything else gets ignored, presumably?
33:24 Speaker 03 As someone who unbelievably benefits from all the attention, it's too much.
33:28 Kara Swisher? Yeah, good for you.
33:29 Speaker 03 It's absolutely too much.
33:30 Kara Swisher? Yeah.
33:31 Speaker 03 Right? And people turn, you know, I hear AI is a solution to many problems. AI is a solution to our spending in healthcare. AI is a solution to, you know, whatever. AI is not the solution to all problems. And there is too much focused on the technology, certainly not enough focused on the applications and the use and thinking about the environments put in. And we are ignoring a lot of other technologies that we should be investing in. I love the attention, but it's too much. Let's still finish the interview, but like, too much.
34:01 Speaker 04 I actually don't love the attention, because I think it brings sometimes the worst kind of people to this field. And it's so hard to separate hype from reality.
34:12 Kara Swisher? Yes.
34:13 Speaker 04 That we actually can't get anything productive done. Nobody wants to have a long-term conversation about creating more robust legal systems, or better medical systems, because boring. There's a new, like, shiny toy being dangled every 30 seconds in front of our faces. I actually long for the days where, you know, the idea of AI governance was very boring, because then the only people in the room were the people who actually cared about it, and we could have real conversations and try to get stuff done. And now it's like, somebody like, spent five minutes on an LLM, and suddenly they show up in the room as an expert, and then you gotta start from, you know, from like, from like, level negative one, to get everybody back up to speed. It's tough. It makes it harder. And like, yeah,
34:56 Speaker 04 that's odd. I feel kind of wrong that I'm in agreement with Peter Thiel on anything.
35:01 Kara Swisher? Right.
35:02 Speaker 04 So I'm gonna have to sit with that this evening.
35:03 Kara Swisher? Okay, all right, you sit with that. He's correct, though.
35:06 Speaker 02 Yeah, so what's the claim that we're thinking too much about AI and not our other problems?
35:11 Kara Swisher? Other problems.
35:12 Speaker 02 Yeah, I think the thing is, if you think about where we are.
35:15 Kara Swisher? I mean, obviously, he'd like to focus more on destroying democracy, but go ahead.
35:19 Speaker 02 Yeah, yeah, it's again, AI is going to impact, I believe, the way we do just about everything. And that means it obviously is going to interact with all the things that don't work for it. Well, it's going to exacerbate a bunch of those things as well. So if we haven't figured out, you know, inequality, if we haven't figured out how to, you know, manage the fact that we have very, very big corporations producing this stuff, if we haven't figured out healthcare, if we haven't figured out access to justice, yeah, it's going to exacerbate all those, all those things.
36:00 Speaker 02 But I don't know that I would say we should just stop thinking about AI and focus back on these other things because I, I think we need to be thinking about them in the context of AI.
36:10 Kara Swisher? Because it has, it's sort of like the, a little like the World Wide Web, right? It affected everything.
36:15 Speaker 02 Everything.
36:15 Kara Swisher? Right, right. When the internet first started, I was at the Washington Post and someone asked me like, what is it? And I said, it's everything. And it's hard to explain. They're like, go away. I was like, it's everything.
36:25 Speaker 02 Yeah, yeah.
36:26 Kara Swisher? I don't know what everything is.
36:27 Speaker 02 And it has changed everything.
36:28 Kara Swisher? That's right.
37:10 Speaker 06 You sound horrible. Oh, I feel worse. This flu is not playing around. Dad, you promised you'd be in my band, and it's showtime.
37:19 Kara Swisher? Then it's Dayquil Intense Flu time.
37:21 Speaker 01 For fast and powerful relief of your flu symptoms, try Dayquil Intense Flu. Hit it.
37:26 Speaker 06 Yeah.
37:28 Speaker 01 Dayquil Intense Flu, the fast-acting daytime coughing, aching, aching, aching fever power through your flu medicine. Use is directed. Keep out of reach of children.
37:39 Kara Swisher? I'm Kara Swisher, and I like people who say the quiet part out loud. In fact, I love it. That's Tiger Sisters, a new podcast that recently joined the Vox Media Podcast Network. Cherie Brooke Lu and Jean Lu are real sisters who built their careers at LinkedIn and Goldman Sachs. On their weekly podcast, they're giving the next generation practical tips on how to get ahead, how to write emails people actually answer, how to make your LinkedIn profile stand out, and how to keep learning as an adult. It's straight talk about money, career, and relationships every Monday. Follow Tiger Sisters wherever you get your podcasts.
38:15 Kara Swisher? So, Mark, you said that when it comes to AI regulation, quote, "There is so little understanding of what the concerns are and what should be regulated." How would you advise academics, like advise regulators, to understand AI? There's a trope that regulators do not understand tech. That is not true. That is somewhat true with some of them. But in general, they regulate everything else, and sometimes badly, sometimes well. So, do you think we need more public funding for universities to become relevant players in AI? Because they are not at this point.
38:46 Speaker 03 Yeah.
38:47 Kara Swisher? You understand that.
38:48 Speaker 03 So you were asking me if I want...
38:49 Kara Swisher? As they were in every other technological revolution.
38:51 Speaker 03 Yeah, you're asking me, should we give more money to Johns Hopkins? Yes. No, but I think in seriousness, but...
38:57 Kara Swisher? I said more, should we give more money to Harvard, for example? Sorry.
39:01 Speaker 03 I'm going to change my answer to no. No, but in seriousness, you know, I see, and you've had this experience, when they call people to Washington to testify for Congress, experts, they have very different goals, the people testifying, you've spoken about this very well, than the people that are interviewing them. And academics play a unique role where we really can sit in the middle and say, like, look, we're not, we don't work for the companies, we don't work for the government, but we study this technology and we really are experts and we can say things. So absolutely, we need to be cultivating exactly buildings like this, that are bastions of higher learning, of people who are experts in this technology right here in Washington, to be that kind of policy audience. I think that's absolutely
39:45 Speaker 03 necessary. I think we also need those people, people like me, I'm a computer scientist, to interact with the people who sit next to me here on the stage, who understand these systems in ways that I don't, who understand the regulatory environment, to what it means to regulate things, right? And people like that can build the bridges to say, okay, it's great that you published a paper at, you know, NEREPs last week about this fancy algorithm or this math, but this is what regulation actually looks like. How can we connect the dots? How can we actually take your insights and get them to apply to what regulation really looks like, and then speak to the regulators to help them understand what's possible, what's not possible, what should we do?
40:20 Kara Swisher? Absolutely. So, Rumen, you've called for a global governance body that could set up all kinds of things, including forcing corporations to slow down AI implementation if it leads to too many job losses too fast. That train seems to have left the station, I suspect. Sam Altman has called for an international agency. Your idea is much more ambitious. Talk about that. This is something I've always said there should be, this is like nuclear power, this is like cloning. Can you imagine an international regulatory agency with power over the richest corporations in the world? I have not seen that happen.
40:54 Speaker 04 I think it'd be deeply fascinating, but I'll tell you something really interesting. When I wrote that op ed, it was, what, April of 2023, there was not a single international entity, and now we are basically swimming in them. You know, there are AI safety institutes that have been set up in, what, 11 countries? There's an international consortium of AI safety institutes. The UN has a body OECD has a body. I mean, I could just keep going and going and going. And you know, Gillian and I were joking earlier how we just all fly around the world and see each other in different locations. But it's it's true. There is actually a global governance community. And I can count amongst my friends and colleagues that I work on this with. People who are in Australia, in Japan, in Singapore, you know, in Nigeria, in France.
41:38 Speaker 04 I mean, the next big AI safety summit is in Paris. The one before that was in Seoul. The first one was in Bletchley. So there actually is a global conversation. I wouldn't be surprised if we started to see. And as a political scientist, I find statecraft incredibly fascinating. You know, and just sort of nerding out for a second. It is one of the most fascinating times to be alive. To see this sort of global conversation truly beginning on any sort of a governance that could look more meaningful. I mean, it doesn't mean it's going to absolutely happen. We may end up in a rather toothless model. But we could, I think there's enough people pushing for something novel and something new. The two examples I give in that in that article were actually IAEA, which I think is a really interesting model.
42:20 Speaker 04 As well as a Facebook Oversight Board. So we technically do have a global governance entity. It is a group that Facebook had set up for themselves.
42:29 Kara Swisher? So it's possible. Now, Julian, you held a lot of international dialogues around AI alignment and global competition. One of the things that every tech CEO I've had, including Mark Zuckerberg and others, have talked about whether they point to competition with China to argue against any regulation. I call it the Xi or me argument. And I'm like, I'd like a third choice if you don't mind. I don't like any of you and I don't like him. So how do you do regulation? Slowing down AI is that that's their argument. China obviously has to be part of this global conversation. What do you imagine that is there an ability to do cooperation with China?
43:11 Kara Swisher? And come to a global decision making on these things?
43:16 Speaker 02 Yeah, I think we have to. And I think you can use things like thinking about the World Trade Organization as an infrastructure that we have that actually does implement rules about, you know, what you need to demonstrate in order to participate in in global trade. We've had these international dialogues for AI safety, lots of Chinese participants in them.
43:36 Speaker 02 Seems to be a lot of interest, certainly in the academic community, these are predominantly academics. Lots of interest. I mean, it is. It's something that affects everyone. It's going to change the way the world works. We're going to have to put together those kinds of structures. I think there is a lot of shared interest in doing that. I think what it requires, however, is that we be thinking about the structures we're put in place. Like, you can't just be talking about, like, what are the standards? What's it allowed to say? What's the model allowed to say? But you, you got to put some, like, registration. You got to put registration in place. You have to put the capacity to say and demonstrate, oh, if you haven't passed these tests, like, the US can put its own
44:21 Speaker 02 requirements in place and say, and those models can't be sold in our markets if they don't pass those tests. I think there's actually shared capacity for that, particularly if you start with the the things on which everybody agrees. So when we had our meeting in Beijing last year, there's, there's agreement. Okay, we need to make sure that these are not self-improving systems. We need to make sure that we have red lines when we would know they're getting close.
44:48 Kara Swisher? Killer drones, no.
44:49 Speaker 02 Yeah. That we could.
44:52 Kara Swisher? Just New Jersey.
44:52 Speaker 02 We have the capacity. And so use those things on which there's going to be widespread agreement to build a structure in place.
45:00 Kara Swisher? They can do that. The, the, we have to go faster because of China seems to be a pretty reductive argument in that regard.
45:06 Speaker 02 Yeah, I, I mean, I, I certainly, here's what I think is really wrong with that argument. The idea that building regulatory infrastructure is going to slow it down and we should not do that. It's not the way that any part of the economy works, actually, having basic play.
45:21 Kara Swisher? Jillian, they're special.
45:24 Kara Swisher? I don't know if you know that. So, last question. We have a long way to any AI regulation. The Biden administration issued an executive order on AI. Donald Trump's going to repeal it. Trump has Elon Musk. As his best friend, apparently, which is a lovely thing to see. Elon's stance on AI regulation is unclear. He has different ones. He changes quite a bit. He signed a letter for a six-month pause in AI development. Then he funded his company. It's going like gangbusters. I know it seems hypocritical, but that's what he did. He supported a controversial California bill to regulate AI. He's not an uninterested party, right? And he is sitting right next to President Trump at this point, especially on these issues.
46:07 Kara Swisher? And if it's not him, it's one of his many minions. What do you expect from the Trump administration? What worries you? What gives you hope? Very quickly, each of you, we just have a minute and a half.
46:17 Speaker 04 Carrie, you asked me to opine about Elon Musk, knowing that I had worked at Twitter.
46:21 Kara Swisher? No.
46:23 Speaker 04 A minute and a half will definitely not be long enough.
46:26 Kara Swisher? Yeah, OK.
46:26 Speaker 04 What am I?
46:27 Kara Swisher? Not a fan, I got that.
46:28 Speaker 04 You know, I don't think.
46:30 Kara Swisher? Me either.
46:31 Speaker 04 His idea of ethics and my idea of ethics are the same.
46:33 Kara Swisher? Yeah.
46:33 Speaker 04 You know, I don't even know how to answer your question.
46:37 Kara Swisher? What is your greatest hope that would happen and your greatest worry in this administration?
46:42 Speaker 04 My greatest hope is that things don't get much worse than they are. That's probably the best we can hope for is status quo. What may happen is a lot of the headway, a lot of the positive things we were talking about, are getting rolled back. I mean, specifically, the EO has been on the chopping block. But also, we're going to have to worry about programs like, you know, programs at NIST, for example, any sort of scientific body that's been doing test and evaluation. I also worry a bit about the brain drain that's going to happen. I think there are a lot of people who sat through one Trump administration and are very dismayed that the narrative this time around seemed to be, oh, well, it wasn't so bad last time. And they're like, do you know what it cost me? So that it wouldn't be that bad. And now they're leaving. So what's going to happen when all of the amazing people, and you're totally correct,
47:27 Speaker 04 that all these amazing people are brought in? And they're like, I'm just not going to sit here for another four years.
47:31 Kara Swisher? So that's your worry. Go ahead.
47:33 Speaker 03 I expect inconsistency and uncertainty for exactly the reasons you said about Elon Musk. I don't know what's going to happen. And there's a lot of, I can make arguments in both directions. What's the good? So there are a lot of people not in government doing great work. Some of them are sitting next to me on this stage, and that work is going to continue. And to continue to lay the foundation for whenever the government is ready to take action, there will be research there to support it. And so I don't think the external research to the government is going to stop. I think I'm certainly, and I don't know about my colleagues, are going to feel pressure to do more and be more involved because it's now not happening in the government. And we'll see what happens in four years. Can we get a countdown clock for years?
48:14 Speaker 02 Yeah, I think the stuff that's driven by national security is going to continue. So I think those pieces of executive order either might get redone in different ways. I think that's going to continue. I think it's very hard to predict whose ear anybody's going to be listening to. I think the China component's going to be very important. I think there's, you know, we just had a bipartisan AI task force report come out that said, we got to invest in developing the science evaluations. And I think that's important. I think we've got bipartisan efforts. So I actually think there may be things that that continue. I don't expect it all to, it's too big. It's too big for it to just, we're going to ignore it and quote unquote, not regulate it.
48:56 Kara Swisher? Right. I see. So it's too big to fuck up. Too big to fail in any case.
49:00 Speaker 02 Too big to fail.
49:01 Kara Swisher? Thank you so much, all three of you. Great discussion, and we'll see what happens. We'll check in here in four years. Okay? We'll talk about that and see what happens. Thank you so much.
49:09 Speaker 02 Thank you.
49:19 Kara Swisher? On with Kara Swisher is produced by Christian Castro-Rosell, Kateri Yokum, Jolie Myers, Megan Cunane, Megan Burney, and Kalen Lynch. Nishat Kurwa is Vox Media's executive producer of audio. Special thanks to Claire Hyman. Our engineers are Rick Kwan, Fernando Aruda, and Aliyah Jackson, and our theme music is by Trackademics. If you're already following the show, feel free to skip out on any useless homework, and all of it is useless, so skip out. If not, 500 words due tomorrow on whatever. Go wherever you listen to podcasts, search for On with Kara Swisher, and hit follow. Thanks for listening to On with Kara Swisher from New York Magazine, the Vox Media Podcast Network, and us. We'll be back on Monday with more.
50:06 Speaker 04 We just moved into our new house and I didn't realize just how close we were to the neighbors. That's when my friend told me about Blinds.com.
50:14 Kara Swisher? Blinds.com isn't a medication, but it can improve how you feel about your home. Customers who shop at Blinds.com experience excellent customer service, from consultation to installation. Side effects include increased privacy and better sleep. Talk to your partner today and see if Blinds.com is right for you.
50:31 Speaker 01 Shop Black Friday all month long at Blinds.com. Don't miss huge deals dropping every week.
Previous Samantha Bee on Being 'Democrat-Celibate' (from 'The Assignment with Audie Cornish') Next Trump Whisperer Maggie Haberman On Trump 2.0 Page updated November 5, 2026 · transcript and summary are machine-generated