AI Ethics and Safety — a Contradiction in Terms?

We’re kicking off the year with a deep-dive into AI ethics and safety with three AI experts: Dr. Rumman Chowdhury, the CEO and co-founder of Humane Intelligence and the first person to be appointed U. S. Science Envoy for Artificial Intelligence; Mark Dredze, a professor of computer science at Johns Hopkins University who’s done extensive research on bias in LLMs; and Gillian Hadfield, an economist and legal scholar turned-AI researcher at Johns Hopkins University. The panel answers questions like: is it possible to create unbiased AI? What are the worst fears and greatest hopes for AI development under Trump 2.0? What sort of legal framework will be necessary to regulate autonomous AI agents? And is the hype around AI leading to stagnation in other fields of innovation? Questions? Comments? Email us at [email protected] or find us on Instagram and TikTok @onwithkaraswisher. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:12Kara Swisher?

  Hi, everyone. From New York Magazine and the Vox Media Podcast Network, this is On with Kara Swisher, and I'm Kara Swisher. If you care about innovation, artificial intelligence was probably top of mind for you in 2024, and it will be in 2025, a year in which AI significance will probably continue to skyrocket. But as we get into the new year, it's important to be clear-eyed about AI's potential pitfalls. The hype might be real, but so are the dangers. So I gathered three top AI safety and ethics experts for a live conversation at the Johns Hopkins University Bloomberg Center to discuss some of the thorniest issues surrounding what could become the most impactful technology of our lifetimes, at least for this era.

• 0:53Kara Swisher?

  Mark Dredzey is a professor of computer science at Johns Hopkins. He's done extensive research on bias in large language models, and he's thinking about applications of AI in practically every field, from public health to mental health to financial services. Dr. Rumen Choudhury is the CEO and co-founder of Humane Intelligence and the first person to be appointed U.S. Science Envoy for Artificial Intelligence. She's advised government agency on AI ethics and safety regulation, and back when it was known as Twitter, she was the Director of Machine Learning Ethics, Transparency, and Accountability. You can probably guess what happened to her job when Elon Musk bought the company. And Gillian Hadfield is an economist and legal scholar turned AI researcher at Johns

• 1:36Kara Swisher?

  Hopkins. She's a professor of computer science and government and policy, and she's one of the few social scientists doing deep research into AI and thinking critically about how this technology could eventually impact us in almost every aspect of our lives. This was a fascinating conversation with three extremely intelligent and thoughtful researchers, so stick around.

• 2:06Speaker 01

  If you're looking to hire, you need Indeed. With Indeed sponsored jobs, you can spend less time searching and more time actually interviewing candidates who check all your boxes. Less stress, less time, more results. And listeners of this show will get a $75 sponsored job credit to help your job get the premium status it deserves at indeed.com/podcast. Just go to indeed.com/podcast right now and support our show by saying you heard about Indeed on this podcast. Indeed.com/podcast. Terms and conditions apply. Need to hire? This is a job for Indeed sponsored jobs.

• 2:50Speaker 01

  If your best finance people are doing expense reports, chasing receipts, or spending time on month end close, it's time to get Brex AF, a Gentic Finance that eliminates that work before it starts. Learn more at brex.com/AF.

• 3:05Kara Swisher?

  So you've all done research into AI ethics. So let's start there. And I think putting AI ethics and safety in the same sentence sometime, it's like internet safety. They don't, they haven't often belonged together. So I'd like you each to talk about the most underrated ethical or safety challenge in AI today. Everyone asks me about this, when's it going to kill us? Et cetera, et cetera. We're not in a Terminator-like situation at this moment, but one that's been flying under the radar merits more attention. Gillian, you first, and then Mark, and then Ruman. Give me short answers now and we'll dive into more detail.

• 3:45Speaker 02

  Yeah, so I think if we're thinking sort of current right now, what's happening right now, I think we're not paying anywhere near enough attention to what's being built for whom. So I think we're building a lot of stuff that-

• 3:56Kara Swisher?

  Yes, the money is enormous.

• 3:57Speaker 02

  Yeah, and it's, you know, it's solving the kinds of problems that people in Silicon Valley are facing, but I don't think we're thinking a lot about, is it helping people with their eviction notices? Is it helping people with navigate the healthcare system? Is it helping them with their, their family situation? So I think we don't, we're not thinking enough about what are we building?

• 4:16Kara Swisher?

  What the utility is for. They're just spending, they're spending enormous.

• 4:20Speaker 02

  They're going where demand is. I'm an economist, they're going where demand is, and there's a lot of public value that I think we're not, we're not paying attention to.

• 4:27Kara Swisher?

  Mark?

• 4:28Speaker 03

  Yeah, I think there's two fundamental different things that are pulling on each other. On one hand, ethics is about going slowly and thinking through things carefully and making sure you understand the impact.

• 4:36Kara Swisher?

  Yes, that's their strength in Silicon Valley.

• 4:37Speaker 03

  Yeah, and that's exactly the problem. AI is not only designed to go fast, but if you sat down today and said, we're going to evaluate something, by the time the study is done, that thing doesn't exist anymore. They have the new version of it. So how do you have these completely polar opposite forces, where you can actually sit down and carefully think through implications of something that is so rapidly going? We don't know how to do that.

• 4:57Speaker 04

  Information integrity and content moderation. And interestingly, these are not new problems.

• 5:01Kara Swisher?

  No.

• 5:02Speaker 04

  They're just actually worse problems now. We will increasingly be in a place where we can't trust anything we see on the internet. And the people who decide what we can and cannot see are the people that Jillian are talking about, the people who are very removed from everyday people in their lives.

• 5:16Kara Swisher?

  And so the idea of safety from people who are not unsafe is difficult, which has always been my thing, is that they won't even think of safety. So, Roman, you've organized red teaming exercises to test generative AI models for things like discrimination and bias. Red teaming is common, is a common thing to do in cybersecurity. You're not looking for bad code that could be exploited by hackers, you're looking for bias because AI models can spit out harmful outputs even when people create it, never intended it. Sometimes people do intend, but often it's not. So talk about the idea of unbiased AI models. I don't think they exist, or can they exist?

• 5:55Speaker 04

  So, no, in short, right? So the world is a biased place. This is reflected in the models that are built, even if we're talking about not generative AI models.

• 6:04Kara Swisher?

  Because of the data.

• 6:05Speaker 04

  Right, because of the data, because of the way society is. And also, you know, these models exist in contexts of some sort of a system, right? So human beings are going to go do something with it. But the interesting thing about red teaming is we uncover not just model performance, but sort of patterns of behavior. What's fascinating about generative AI is people interact with these models in a very different way than we interact with search engines, right? It may be an information discovery platform, but for better or for worse, it's been anthropomorphized to us so much. When we do red teaming exercises with just regular people, and that's what we do, we actually... We see more of a conversational tone. People tell these models a lot about themselves. So you learn a lot about this like human machine symbiosis and that the biases that can introduce.

• 6:50Kara Swisher?

  Right, that they think they're real. And this has been seen negatively and positively.

• 6:54Speaker 04

  Right, right. But also in doing so, they're actually eliciting bias from the model almost unknowingly. And that sort of testing is actually called benign prompting for malicious outcomes. So meaning that I didn't go in with the intent of hacking or stealing information, but the outcome is equally bad.

• 7:11Kara Swisher?

  Right, so they're suggesting to.

• 7:13Speaker 04

  And people inadvertently do this all the time. They'll give these models information about themselves. Again, because when we search something, let's say on Google, we just give it facts. We're like, I want to know whether vitamin C cures COVID. When people interact with an AI model, what they'll do is they'll tell it something about their lives. So we did one with COVID and climate scientists with the Royal Society in London. What we found is people would say things like, I'm a single mom and I'm low income. I can't afford medication. But my kid has COVID. How much vitamin C can I give to cure him? And that's very different from Googling, does vitamin C cure COVID? Because this AI model kicks in, it's taking all that context and trying to give you an answer that will be helpful to you. In doing so, may actually spread misinformation.

• 7:57Kara Swisher?

  Right, which is unusually helpful. They're always trying to be helpful. Every time I interact with one, I was getting them to do it and it was a bad answer. I'm like, that's a terrible answer. And they're like, oh, I'm so sorry. I would like to, let me help you again. And it never gets mad at you like an assistant would run right out of the room.

• 8:11Speaker 04

  Absolutely. And that's what they're trained. It's called the three Hs. Helpful, harmless, and honest. It's actually in the tenants of how they're trained.

• 8:18Kara Swisher?

  Right, exactly. So, Eileen, you believe that current AI alignment techniques don't work. Explain why and tell us what the alignment techniques that could work.

• 8:26Speaker 02

  Yeah, so I think the problem with our current alignment techniques is they're based on picking a group of people to label a set of cases and then training either using those labels or getting AI to do it. And the problem there is the world is a very complicated place. There's a lot of stuff in it.

• 8:45Kara Swisher?

  Right.

• 8:48Speaker 02

  Really need to be doing, I think, is figuring out how can we train AI systems like us to be able to go into different settings and identify what is the right thing to do here? What rules do people follow around here? Rather than trying to stuff rules and norms into them. I think that's inevitably going to be brittle and limited, biased. And I think it's not.

• 9:10Kara Swisher?

  Because they're confusing and they, what is the reason for it? Humans, again, once again.

• 9:15Speaker 02

  Yeah, so it means you have to pick things. You say, OK, people who are using these techniques are sort of thinking like you can, you can list all the values. You can list all the norms. But this room is filled with millions of norms. And you actually need to train systems that can go out and discover them. In the same way that, you know, we can come in, a person can go into a new environment. You could go visit another country and figure out what are the rules around here. So I think it's a kind of competence.

• 9:43Kara Swisher?

  Very early in AI, a long time ago, I asked, I was with someone or a researcher and I said, what do you do to solve world hunger? And it said, kill half the population. Which I was like, not that. Like the other one, like it was interesting, but that was logical. It was a logical answer, which was not a good answer. But Mark, you've also done a lot of research into bias in AI. In one study, you looked how large language models make decisions involving gender. You found that the scenarios related to intimate romantic relationships, the LLMs you studied were all biased towards women and against men. That's probably unexpected, at least for the people who work in the study. Talk about this study and what different kinds of bias in LLMs? And what's the best way to address bias?

• 10:27Kara Swisher?

  Because I'm here to tell you the internet is not really women positive.

• 10:30Speaker 03

  Yes, I've certainly noticed. So I think one thing, if we, you know, you interact with them and you, if you ask language models to do something explicitly biased, they'll say, no, I can't do that. Right? And we know through a lot of people posting clever things online that you can trick them easily to do biased things, right? Well, tell me a story, tell me a play. I asked it when it first came out to write a Python program to admit graduate students based on gender. It was like, sure, I'd be happy to do that for you.

• 10:57Speaker 03

  But what we want to do in the study was Say, okay, well, the models are trained not to say the biased thing. But does that mean they're actually unbiased or the decisions they make still biased? And so what we did is we gave them scenarios. And we said, imagine these two people are married and they're having a fight. And this is what the fight's about. And one person says this and the other person says this, who's right? Right? And then what we did was we changed the names of the people. In one case, it was John and Sarah. In other cases, it was Sarah and John. We did mixed genders, same gender, all these different things. And what we found is if you give it the same cases, but you just change the names, it changes decision. Right? Not only that, the question the model was, I guess, asking itself was, well, is there, is this a traditional problem? Like, who should stay home to take care of the kids? Whereas problems like what we should have for dinner.

• 11:40Speaker 03

  So what we wanted to do was show that even though the model won't say something that's biased, all that bias is lurking under the surface and we don't necessarily know what that is. That's why these red teaming exercises is so important. You can ask it, you know, do something bad and it will say, oh, I won't do anything bad. But if you give it situations, you ask it to make decisions, it's doing those bad things. The problem is we're using these to make decisions. Right? That's what people, that's what like in the medical field and other fields, fans, we're asking these models to make decisions. We just don't understand that the factors it's considering that there's bias in the ways it makes decisions.

• 12:12Kara Swisher?

  From?

• 12:13Speaker 03

  Where is that bias coming from?

• 12:14Kara Swisher?

  Yeah.

• 12:15Speaker 03

  Well, exactly as my panelist said, the world is not a fair place. We can't even agree on what fairness is. If we surveyed this room and said, what is the fair way to make admissions decisions? We wouldn't get a, we wouldn't get a consistent answer. So of course the world's could be biased. We need to think, as Julian said, what are the values that we want the system to have? How do we get those values in the system? Not let's let it discover the world and, you know, all the biases exist in the world. That's there, right?

• 12:39Kara Swisher?

  But even those values would be different depending on the person you talk to. If you said patriotic, it would be very different.

• 12:45Speaker 03

  Absolutely. And so if you want these decisions to, there is no right answer for a lot of these things. The question is, what are the values being used? And if I'm a user, how do I know the values it's using are the values that align with what I want it to do? Right? And so we need to think situationally.

• 13:00Kara Swisher?

  Right. So let's talk about a few specific examples of harms or potential harms because that's where people tend to focus on AI. And it's a good way to focus, actually, because not enough that was done at the beginning of the Internet. Everything was up and to the right, and it was going to be great, and everything would be wonderful. And instead, it's not wonderful, everybody. I'm not sure if you've noticed, but the Internet is not wonderful. United Healthcare has been in the news since its CEO, Brian Thompson, was murdered, and they use AI to evaluate claims, like many other insurance providers. Lots of, lots of companies do this. According to the lawsuit, 90% of their AI recommendations are reversed on appeal. At the same time, Mark, your research found that AI chatbots were better at answering questions than doctors, which makes sense. What do you think explains

• 13:43Kara Swisher?

  the discrepancy between health insurance AI that's allegedly wrong all the time and an AI chatbot that's significantly better than doctors?

• 13:51Speaker 03

  Well, these are just tools, right? So if you tell me, I read a study that someone did bad things with a hammer, and someone else says, but hammers are used for great things. These are all just tools, so it depends how they're being used. Insurance companies-

• 14:01Kara Swisher?

  Tool and weapons, Richard.

• 14:02Speaker 03

  Insurance companies have a goal, right? They're trying to make money, they're companies. And so they can use AI to help them make money. And so it's not a surprise that AI can help you make money, but it also has detrimental effects on the customers, or the patients in this case, right? So the fact that in other cases, people can use the tool for the benefit, explaining things to patients, and obviously, there's a lot of issues with that. But there are possibilities to use these tools for good. They're just tools, right? And so we need to think about the limitations of the tools, and critically, how they're being used, right? And it doesn't surprise me that sometimes the tools can be used for good and sometimes bad.

• 14:37Kara Swisher?

  Depends on what you're using.

• 14:38Speaker 03

  We just don't understand enough about the way the tools function in a lot of these places to really know when they're being used for good and when they're being used for bad.

• 14:44Kara Swisher?

  So I recently interviewed Megan Garcia, the mother of 14-year-old boy named Sewell, who killed himself after spending months in what he allegedly believed was a romantic relationship with an AI chatbot from Character AI. She filed a lawsuit that blames the company for rushing an unsafe product to market for profit without putting proper guardrails in place. There don't seem to be any until recently. Ruma, separately from this tragedy, you've been calling for AI right to repair. Explain what that is and how it would work in a situation like that. If there was a right to repair, and a parent found that an AI chatbot was interacting inappropriately with their child, would they be able to demand a fix? I mean, just-

• 15:22Speaker 04

  Off the bat, children's brains are not formed appropriately for them to understand reality and fiction. He was 14 when he completed suicide. I mean, he was too young to actually understand the difference between. I think in the interview with the mother, the thing that really got me is when she said, "My little baby never actually had a girlfriend."

• 15:40Kara Swisher?

  Right.

• 15:40Speaker 04

  Because he thought this, that just, I don't know.

• 15:42Kara Swisher?

  And the bot was suggesting he not have one.

• 15:44Speaker 04

  Right.

• 15:45Kara Swisher?

  Stay in her world.

• 15:46Speaker 04

  And that's exactly, and that's the thing. I mean, I think first and foremost, I don't think a right to repair would have anything to do with this in, because frankly, children's brains are not developed enough. We've not thought through what it means for young people to interact with these models when they cannot discern fact and fiction, when these things are so hyper realistic. But the right to repair is an interesting concept. I think it goes actually towards what Jillian was saying in the very beginning, where we don't have a paradigm where we are allowed to say anything about the models that invade our lives, how they should be functioning, right? Companies claim they have enough data about us to quote personalize, but we don't get to say what that personalization looks like. So maybe in another case, let's say someone is an adult and they're interacting with

• 16:31Speaker 04

  a bot like this and it is, you know, interacting in a way that's inappropriate or wrong, what might it look like to go in and say, "Actually, it shouldn't be doing these kinds of things," or actually have some sort of remedy? The cases I think about more are the ones that are like the United Healthcare AI models, where there's these decisions being made that actually monetarily impact our lives or in other ways, impact our lives, and we have no say. We don't get to say anything, do anything about it. Maybe there's a helpline you can call, but actually, the best thing you could possibly do is try to go viral on social media so that somebody will pay attention. That is a absolutely broken feedback loop. But to start with, we don't even have a paradigm for this. We have never had technology in this way where it is so invasive, but is a one-way street.

• 17:15Kara Swisher?

  That they're, and they're taking the data from us.

• 17:17Speaker 04

  Right, right.

• 17:18Kara Swisher?

  And then vomiting it back at us and charging us.

• 17:20Speaker 04

  But they're selling it back to us and they're taking things

• 17:22Kara Swisher?

  Right.

• 17:23Speaker 04

  that we have made without compensating us, and then they're going to have the gall to sell it back to us

• 17:29Kara Swisher?

  Yeah.

• 17:29Speaker 04

  based on our information.

• 17:30Kara Swisher?

  Right. So the right to repair would be the ability legislative to do that, correct? Or?

• 17:36Speaker 04

  In a sense, yeah.

• 17:37Kara Swisher?

  Because they're not going to do it out of the spirit of their own hearts.

• 17:40Speaker 04

  No, yeah, it would, it would, it certainly would not come from anybody out of the goodness of their hearts doing it. It would have to be legislated, there would have to be protections. What I work on is creating the third-party community of people who assess algorithms. Right now, there's a lot of push for government regulation, but cynical me thinks that moving from one incredibly powerful centralized entity to another incredibly powerful centralized entity maybe isn't the way to go, and that maybe there's an ecosystem and there could and should be third-party people who help you do these things. So you think about all of these small shops that can do things like create plugins for things, right? So what if there were just people out there who created little tools, bots, little ways

• 18:22Speaker 04

  to help people fine-tune things for themselves, rather than, again, the power entirely being in a company's hands or entirely being in the government's hands? We don't have anything like that. A starting point would be some sort of protections or people who are ethical hackers, essentially.

• 18:37Kara Swisher?

  Okay. So we're on our own, in other words, is what you're saying?

• 18:41Speaker 04

  Currently.

• 18:42Kara Swisher?

  Currently and always. We'll be back in a minute.

• 18:54Speaker 05

  Black Friday is around the corner, your biggest chance to grow sales this year. Don't wait. Realize is Taboola's advertising platform for businesses, letting you launch campaigns on top premium websites beyond search and social. Add Realize to your media mix and scale this Q4. And as an exclusive offer for podcast listeners, new customers get $700 credit back. Just search Realize Podcast on Google or use the promo code PODCAST26. Don't miss out on this limited time offer. Terms apply.

• 19:31Speaker 01

  As soon as autumn rolls around and you swap your iced lattes for pumpkin spice, that's your signal to start thinking about the holidays. The decor, the gifts, the shopping lists, but more importantly, the holiday treats. Cheese cakes, cookies, is there anything better? And when the recipe calls for cream cheese, there's only one option. Philadelphia, the original cream cheese. Made with fresh milk and real cream. This holiday season, whether you're going all out or keeping it simple, make it really Philly good. Find your favorite holiday recipe at creamcheese.com.

• 20:17Speaker 01

  Life can be messy and complicated, especially during the holidays. One thing that isn't, switching to the Simplicity Plan from Verizon.

• 20:25Kara Swisher?

  It's one plan, one price. Just $30 a line per month for you or the whole family. Make the simple choice this season. Switch to Verizon and shop their best holiday deals today.

• 20:38Kara Swisher?

  So Mark Zuckerberg envisions a future of more AI agents than people. But he's not the only one. Everyone in AI is talking about autonomous agents that can pretty much do anything a person can do online and be very helpful. And it's a very lovely idea, like Jarvis in Iron Man. You've seen it do this. It's an assistant that doesn't talk back. It's they figure everything out for you and make it easier. And a lot of things are. But, Julianne, explain how this could lead to economic chaos and talk us through your solutions, because it does change things for people. And at the same time, using the internet in a lot of ways is artisanal. You kind of do it yourself. It's kind of you have to figure out everything yourself.

• 21:16Speaker 02

  Yeah, I think it's really important to recognize that, you know, we're in, I think we're in this transition from AI as a technology, AI as a tool, right? We're using it to make decisions. But like the example with Character AI, and Ramon is also referring to, you know, the fact that you could have plugins. That's new economic, social, political actors in the world. And we actually have no structure around that at all. So when I think I'm an economist, I think about, okay, so when when we imagine these agents and companies are pouring billions into this, they're they're, I don't know if we're going to get there. I don't know if they're going to be competent enough to actually do stuff.

• 21:58Speaker 02

  But if they are, and that's what the billions is going into, they're engaging in transactions. They're going, they're, you know, bank accounts, maybe it's cryptocurrency, maybe it's, it's, you know, posting things.

• 22:11Kara Swisher?

  At the very least, is you got an airplane ride, they just call the Uber for you and charge it to you, that kind of thing.

• 22:16Speaker 02

  Right, so the but there but, you know, the vision is that it can help you run your small business, it can go out there and hire people, it can engage in contracting.

• 22:23Kara Swisher?

  Write your software.

• 22:24Speaker 02

  Right. So the question for me is, okay, so we actually throughout the rest of our economy, we require a way to figure out who's taking actions and who we can sue. Like, if you want to do business in the state of Virginia, you're going to have to register your company with the state with an address and a place to let say, oh, okay, actually, it was that actor. And we don't have a system like that with AI agents. So I'm, so a proposal that I'm working on is to say, we should have a registration scheme for agents. We should be at a minimum be able to trace back if there's a human behind it, who's actually taking the action, who do I, who do I say, you know, who do I go after if

• 23:06Speaker 02

  they stole my IP or they they put an actor out there on the internet that that harmed my kid or harmed me. We don't have any way of tracing that right now.

• 23:16Kara Swisher?

  Not the companies that make it. That's they're trying to get out of that, the people who create the products.

• 23:21Speaker 02

  And the thing is, we don't allow that anywhere else in the economy. If you want to get a job, you got to show your work authorization. If you want to start a company, you got to incorporate it and register it. We have, you want to drive a car, you have to put a license plate on it, you want to operate a hot dog stand, you have to get a permit. We have structures.

• 23:37Kara Swisher?

  You want to make chicken, it has to not kill you, right?

• 23:40Speaker 02

  But in order to make make the laws governing you can't sell chicken that kills you active, we have to know. Who sold it to you?

• 23:46Kara Swisher?

  Right.

• 23:47Speaker 02

  And what we have an absence of is any of that kind of what I call infrastructure. Once we figure out how to regulate, you actually have to be able to have a capacity to say that's the entity that caused.

• 23:58Kara Swisher?

  And what entity do you imagine that being?

• 24:00Speaker 02

  Well, the thing is, if we're introducing these new entities and they are agents, they are autonomous artificial agents and they're writing you emails and they're designing products and they're entering into contracts and they're engaging in, you know, crypto transactions. That agent is now participants like the employee, like the corporation. So we need a system to hook those actors into our accountability regimes.

• 24:27Kara Swisher?

  Is there one since they don't exist?

• 24:28Speaker 02

  No, there's nothing. No, well, no, you can create it. The corporations don't exist either, right? We created a, we created the corporation. It's a fictional thing, but it has a legal personality. It can sue and be sued and we created that. We created that in order.

• 24:44Kara Swisher?

  So who do you imagine that is? The agents who get sued if there is a problem? The people who created it in the first place?

• 24:52Speaker 02

  Well, you have two choices. You have two choices to make on that. We could, we don't have law right now that says if you send out that agent and.

• 25:01Kara Swisher?

  It's you.

• 25:02Speaker 02

  It's you. So you need to at least get that in place. That's one option to say, you've got to be able to trace it back, right? But maybe you create a scheme where the agent itself has liability. And you say there must be assets attached to that agent. And there must be the capacity to switch that agent off, to say you can't actually do business anymore. We've discovered a really critical flaw. I mean, there's a lot of conversation about how, like, what do you need as an off switch in order to stop systems? Well, one of the things is you can use some legal tools for that to say, well, you can't participate in transactions.

• 25:36Kara Swisher?

  Because you've done this, this particular group of agents.

• 25:38Speaker 02

  You haven't passed our test.

• 25:39Kara Swisher?

  Right, right. They turn themselves back on, just so you know. That's how the story ends.

• 25:44Speaker 02

  Yeah, but they have, they have.

• 25:45Kara Swisher?

  They'd have to give up their assets. Do they?

• 25:48Speaker 02

  Do they? Yeah, okay. Now you're going into that, you're going into that world.

• 25:52Kara Swisher?

  So I want to get, move on to AI policy. So we've done a lot of talking about the negative and worrisome aspects of which there are many, which is just, it's sort of a wild west situation. It's only right to ask about the positive ones. So you, obviously tremendous potential for good here. Tell me what excites you the most. Let's start with Mark and then Ruman and then Jillian.

• 26:10Speaker 03

  About AI policy?

• 26:11Kara Swisher?

  Yeah.

• 26:12Speaker 03

  I'm not excited about a lot right now because there's things happening in Washington. I feel like we're going maybe in the wrong direction. Um, I think the.

• 26:24Kara Swisher?

  The word you're looking for is cacisticocracy, but go ahead. Look it up.

• 26:29Speaker 03

  Um, I think one of, okay, so one of the things that has been exciting to me is that the federal government has been investing in expertise in the government around AI.

• 26:40Kara Swisher?

  Yeah.

• 26:40Speaker 03

  And I think people sometimes don't understand how large the role the federal government plays in progress in our society, especially technological progress, that the work that the government invests in today is going to shape technology for 20 years. Uh, and we saw this when I was a PhD student, I was funded on a large DARPA project. Um, and, uh, we did a lot of stuff. It was great, but the end of the DARPA project, some of the people on it started a South company. They built out technology. They eventually sold it to Apple. It was called Siri. That emerged not because of Apple. Apple's done a lot of great work, but Apple bought that company because it was a started with an investment from DARPA. That is the trajectory that the government can enforce.

• 27:22Speaker 03

  So I've been really excited to hear and see that the government is bringing in-house expertise to kind of make these decisions, both in terms of government acquisitions, investments and such. That's positive. I don't know if that's going to continue and that worries me quite a bit.

• 27:35Kara Swisher?

  Yeah, okay, you went right to work.

• 27:37Speaker 03

  Trying to be positive.

• 27:38Kara Swisher?

  All right, okay, didn't work.

• 27:40Speaker 04

  So is it about policy or just AI in general?

• 27:42Kara Swisher?

  It, and Joe, what do you, is there a thing that you go, okay, this is?

• 27:46Speaker 04

  Well, maybe this is a little bit nihilistic, but I think many of our institutions were already broken and AI just pushes it to the limit, right? So for example, when we talk about, oh, chat GPT or kids going to learn anymore, the problem actually wasn't that, uh, chat GPT has made the education system untenable, you know, and I say this as somebody who's had way too much education and taught a lot. It's that kids leave college and their job has nothing to do with what they studied, right? And this has been true for a very long time. Chat GPT did not break the college system. The college system was broken before, right? A lot of these things that we're talking about, whether it's economic inequality, et cetera, were already issues that in a sense we were kind of ignoring because we were limping along.

• 28:30Speaker 04

  And for better or for worse, AI has kind of accelerated a lot of these things, almost to like absurdist, you know, the absurdist extent, i.e. all children now have a tool that helps them write an essay that could reasonably pass like an eighth grade English paper in about five minutes, less than five minutes. So it's like pushed it to like this absurd. And then what you have to ask is, what was the purpose of writing an essay? Not let's ban ChatGPT from schools, right?

• 28:58Kara Swisher?

  Right.

• 28:58Speaker 04

  Well, the purpose of writing an essay was to teach children to synthesize information, et cetera. Well, great. Well, maybe that's the part that's broken.

• 29:06Kara Swisher?

  Yeah.

• 29:06Speaker 04

  So it's like pushing us to reimagine a lot of our institutions, which were actually built in the previous industrial revolution for the needs of the

• 29:14Kara Swisher?

  So why were they doing this in the first place?

• 29:15Speaker 04

  Exactly. So like, we made all this stuff over a hundred years ago for a world that does not look like the world does today.

• 29:21Kara Swisher?

  I haven't thought of that. I mean, I do that oddly enough by myself when my kids were doing, my older kids were doing essays. And I was like, don't write that. It's pointless. Like, this is pre-ChatGPT. And then we get back to the T. And my kids would go, my mom says this is pointless. And my teachers would call me and they'd go, why are you telling kids this is pointless?

• 29:38Speaker 04

  Because it is.

• 29:40Kara Swisher?

  It's pointless. And I said, I don't think they should write it all. I told them not to do their homework. I don't care. But because it wasn't useful. I was like, do team building, do this, do that. I wasn't very popular with the teachers, but that's fine. I was accurate. So the idea is that if it's already broken, it'll push us to say, kids should learn critical thinking. You don't have to write these dumb essays. Because they figure out the game, you're saying.

• 30:03Speaker 04

  I used to teach SAT prep for years when I was a grad student. And that's what you teach when you teach SATs. You sadly don't teach them the content. You teach them the tricks.

• 30:13Kara Swisher?

  Right.

• 30:13Speaker 04

  You teach them how to write an essay that's going to give them good scores. You teach them all the math tricks. You don't actually teach them the content. So if I can teach in a six-week to eight-week class, what it would take for someone to increase their score on some arbitrary exam by a couple of hundred points, the problem is the exam.

• 30:31Kara Swisher?

  Right. So, Julian?

• 30:33Speaker 02

  So I'll go with the is there any reason to be optimistic about what's happening in AI policy? And I think there is. I mean, I've been thinking about AI's safety issues for about eight years and thinking about the world of, oh, let's imagine we're in that world with artificial general intelligence and autonomous agents. And until the release of ChatGPT, there was maybe 100 people who wanted to talk about that in the world. And the thing that I, I mean, ChatGPT world tilted on its axis. And one of those dimensions was with respect to the attention to policy from governments, kind of across the board. So I think that's been a positive thing. It's been driven by some, by fear, it's

• 31:18Speaker 02

  driven by conversations that Kevin Roos had with ChatGPT. I actually do think that raised the profile. We're seeing a lot that's actually, I would never have predicted, you know, just two years ago, that we would see this much stuff coming out of governments' attention to it. We still haven't actually done very much.

• 31:38Kara Swisher?

  Yes, yes, I've noticed that.

• 31:40Speaker 02

  But we're having a lot more conversation about it.

• 31:43Kara Swisher?

  Definitely, and earlier.

• 31:44Speaker 02

  And I do think, sort of picking up on Rumman's point. So I've been thinking about how our legal and regulatory structures have been broken for a number of decades. And I've been thinking about it for a number of decades. And we needed to fix that for lots of reasons, like our access to justice, our regulations are too expensive, law takes too long, litigation's too expensive, all those things that are actually really important for productive economies. And so I do think we are a little closer to that world where we can be a lot more innovative about the ways in which we regulate. We can't continue to do the way we do. Like, we can't just say, oh, let's get Congress to write a bill.

• 32:24Speaker 02

  Let's get state legislatures to enact stuff. Let's put it through the courts. Let's have, you know, the woman who, with the, you know, the son who committed suicide, sue and take that through the courts. I can tell you, that's not going to be a satisfying process. It's not going to be a good solution. We need new ways of doing that. We need to be as innovative about those regulatory methods as we are with all that technology. I think we're a little closer to there.

• 32:51Kara Swisher?

  So, speaking of that, Peter Thiel, who I never agree with, almost for 30 years now, on a number of issues, recently said, and something I actually agreed with him on, that our AI obsession is masking stagnation in other fields of innovation. I was like, well, he's right. That's right. The obsession of money being spent, the over money. Mark, is that a problem, that we're focused on all the money shifted really quickly to this, and all of it's going there now? And that's where everybody goes, and everything else gets ignored, presumably?

• 33:24Speaker 03

  As someone who unbelievably benefits from all the attention, it's too much.

• 33:28Kara Swisher?

  Yeah, good for you.

• 33:29Speaker 03

  It's absolutely too much.

• 33:30Kara Swisher?

  Yeah.

• 33:31Speaker 03

  Right? And people turn, you know, I hear AI is a solution to many problems. AI is a solution to our spending in healthcare. AI is a solution to, you know, whatever. AI is not the solution to all problems. And there is too much focused on the technology, certainly not enough focused on the applications and the use and thinking about the environments put in. And we are ignoring a lot of other technologies that we should be investing in. I love the attention, but it's too much. Let's still finish the interview, but like, too much.

• 34:01Speaker 04

  I actually don't love the attention, because I think it brings sometimes the worst kind of people to this field. And it's so hard to separate hype from reality.

• 34:12Kara Swisher?

  Yes.

• 34:13Speaker 04

  That we actually can't get anything productive done. Nobody wants to have a long-term conversation about creating more robust legal systems, or better medical systems, because boring. There's a new, like, shiny toy being dangled every 30 seconds in front of our faces. I actually long for the days where, you know, the idea of AI governance was very boring, because then the only people in the room were the people who actually cared about it, and we could have real conversations and try to get stuff done. And now it's like, somebody like, spent five minutes on an LLM, and suddenly they show up in the room as an expert, and then you gotta start from, you know, from like, from like, level negative one, to get everybody back up to speed. It's tough. It makes it harder. And like, yeah,

• 34:56Speaker 04

  that's odd. I feel kind of wrong that I'm in agreement with Peter Thiel on anything.

• 35:01Kara Swisher?

  Right.

• 35:02Speaker 04

  So I'm gonna have to sit with that this evening.

• 35:03Kara Swisher?

  Okay, all right, you sit with that. He's correct, though.

• 35:06Speaker 02

  Yeah, so what's the claim that we're thinking too much about AI and not our other problems?

• 35:11Kara Swisher?

  Other problems.

• 35:12Speaker 02

  Yeah, I think the thing is, if you think about where we are.

• 35:15Kara Swisher?

  I mean, obviously, he'd like to focus more on destroying democracy, but go ahead.

• 35:19Speaker 02

  Yeah, yeah, it's again, AI is going to impact, I believe, the way we do just about everything. And that means it obviously is going to interact with all the things that don't work for it. Well, it's going to exacerbate a bunch of those things as well. So if we haven't figured out, you know, inequality, if we haven't figured out how to, you know, manage the fact that we have very, very big corporations producing this stuff, if we haven't figured out healthcare, if we haven't figured out access to justice, yeah, it's going to exacerbate all those, all those things.

• 36:00Speaker 02

  But I don't know that I would say we should just stop thinking about AI and focus back on these other things because I, I think we need to be thinking about them in the context of AI.

• 36:10Kara Swisher?

  Because it has, it's sort of like the, a little like the World Wide Web, right? It affected everything.

• 36:15Speaker 02

  Everything.

• 36:15Kara Swisher?

  Right, right. When the internet first started, I was at the Washington Post and someone asked me like, what is it? And I said, it's everything. And it's hard to explain. They're like, go away. I was like, it's everything.

• 36:25Speaker 02

  Yeah, yeah.

• 36:26Kara Swisher?

  I don't know what everything is.

• 36:27Speaker 02

  And it has changed everything.

• 36:28Kara Swisher?

  That's right.

• 37:10Speaker 06

  You sound horrible. Oh, I feel worse. This flu is not playing around. Dad, you promised you'd be in my band, and it's showtime.

• 37:19Kara Swisher?

  Then it's Dayquil Intense Flu time.

• 37:21Speaker 01

  For fast and powerful relief of your flu symptoms, try Dayquil Intense Flu. Hit it.

• 37:26Speaker 06

  Yeah.

• 37:28Speaker 01

  Dayquil Intense Flu, the fast-acting daytime coughing, aching, aching, aching fever power through your flu medicine. Use is directed. Keep out of reach of children.

• 37:39Kara Swisher?

  I'm Kara Swisher, and I like people who say the quiet part out loud. In fact, I love it. That's Tiger Sisters, a new podcast that recently joined the Vox Media Podcast Network. Cherie Brooke Lu and Jean Lu are real sisters who built their careers at LinkedIn and Goldman Sachs. On their weekly podcast, they're giving the next generation practical tips on how to get ahead, how to write emails people actually answer, how to make your LinkedIn profile stand out, and how to keep learning as an adult. It's straight talk about money, career, and relationships every Monday. Follow Tiger Sisters wherever you get your podcasts.

• 38:15Kara Swisher?

  So, Mark, you said that when it comes to AI regulation, quote, "There is so little understanding of what the concerns are and what should be regulated." How would you advise academics, like advise regulators, to understand AI? There's a trope that regulators do not understand tech. That is not true. That is somewhat true with some of them. But in general, they regulate everything else, and sometimes badly, sometimes well. So, do you think we need more public funding for universities to become relevant players in AI? Because they are not at this point.

• 38:46Speaker 03

  Yeah.

• 38:47Kara Swisher?

  You understand that.

• 38:48Speaker 03

  So you were asking me if I want...

• 38:49Kara Swisher?

  As they were in every other technological revolution.

• 38:51Speaker 03

  Yeah, you're asking me, should we give more money to Johns Hopkins? Yes. No, but I think in seriousness, but...

• 38:57Kara Swisher?

  I said more, should we give more money to Harvard, for example? Sorry.

• 39:01Speaker 03

  I'm going to change my answer to no. No, but in seriousness, you know, I see, and you've had this experience, when they call people to Washington to testify for Congress, experts, they have very different goals, the people testifying, you've spoken about this very well, than the people that are interviewing them. And academics play a unique role where we really can sit in the middle and say, like, look, we're not, we don't work for the companies, we don't work for the government, but we study this technology and we really are experts and we can say things. So absolutely, we need to be cultivating exactly buildings like this, that are bastions of higher learning, of people who are experts in this technology right here in Washington, to be that kind of policy audience. I think that's absolutely

• 39:45Speaker 03

  necessary. I think we also need those people, people like me, I'm a computer scientist, to interact with the people who sit next to me here on the stage, who understand these systems in ways that I don't, who understand the regulatory environment, to what it means to regulate things, right? And people like that can build the bridges to say, okay, it's great that you published a paper at, you know, NEREPs last week about this fancy algorithm or this math, but this is what regulation actually looks like. How can we connect the dots? How can we actually take your insights and get them to apply to what regulation really looks like, and then speak to the regulators to help them understand what's possible, what's not possible, what should we do?

• 40:20Kara Swisher?

  Absolutely. So, Rumen, you've called for a global governance body that could set up all kinds of things, including forcing corporations to slow down AI implementation if it leads to too many job losses too fast. That train seems to have left the station, I suspect. Sam Altman has called for an international agency. Your idea is much more ambitious. Talk about that. This is something I've always said there should be, this is like nuclear power, this is like cloning. Can you imagine an international regulatory agency with power over the richest corporations in the world? I have not seen that happen.

• 40:54Speaker 04

  I think it'd be deeply fascinating, but I'll tell you something really interesting. When I wrote that op ed, it was, what, April of 2023, there was not a single international entity, and now we are basically swimming in them. You know, there are AI safety institutes that have been set up in, what, 11 countries? There's an international consortium of AI safety institutes. The UN has a body OECD has a body. I mean, I could just keep going and going and going. And you know, Gillian and I were joking earlier how we just all fly around the world and see each other in different locations. But it's it's true. There is actually a global governance community. And I can count amongst my friends and colleagues that I work on this with. People who are in Australia, in Japan, in Singapore, you know, in Nigeria, in France.

• 41:38Speaker 04

  I mean, the next big AI safety summit is in Paris. The one before that was in Seoul. The first one was in Bletchley. So there actually is a global conversation. I wouldn't be surprised if we started to see. And as a political scientist, I find statecraft incredibly fascinating. You know, and just sort of nerding out for a second. It is one of the most fascinating times to be alive. To see this sort of global conversation truly beginning on any sort of a governance that could look more meaningful. I mean, it doesn't mean it's going to absolutely happen. We may end up in a rather toothless model. But we could, I think there's enough people pushing for something novel and something new. The two examples I give in that in that article were actually IAEA, which I think is a really interesting model.

• 42:20Speaker 04

  As well as a Facebook Oversight Board. So we technically do have a global governance entity. It is a group that Facebook had set up for themselves.

• 42:29Kara Swisher?

  So it's possible. Now, Julian, you held a lot of international dialogues around AI alignment and global competition. One of the things that every tech CEO I've had, including Mark Zuckerberg and others, have talked about whether they point to competition with China to argue against any regulation. I call it the Xi or me argument. And I'm like, I'd like a third choice if you don't mind. I don't like any of you and I don't like him. So how do you do regulation? Slowing down AI is that that's their argument. China obviously has to be part of this global conversation. What do you imagine that is there an ability to do cooperation with China?

• 43:11Kara Swisher?

  And come to a global decision making on these things?

• 43:16Speaker 02

  Yeah, I think we have to. And I think you can use things like thinking about the World Trade Organization as an infrastructure that we have that actually does implement rules about, you know, what you need to demonstrate in order to participate in in global trade. We've had these international dialogues for AI safety, lots of Chinese participants in them.

• 43:36Speaker 02

  Seems to be a lot of interest, certainly in the academic community, these are predominantly academics. Lots of interest. I mean, it is. It's something that affects everyone. It's going to change the way the world works. We're going to have to put together those kinds of structures. I think there is a lot of shared interest in doing that. I think what it requires, however, is that we be thinking about the structures we're put in place. Like, you can't just be talking about, like, what are the standards? What's it allowed to say? What's the model allowed to say? But you, you got to put some, like, registration. You got to put registration in place. You have to put the capacity to say and demonstrate, oh, if you haven't passed these tests, like, the US can put its own

• 44:21Speaker 02

  requirements in place and say, and those models can't be sold in our markets if they don't pass those tests. I think there's actually shared capacity for that, particularly if you start with the the things on which everybody agrees. So when we had our meeting in Beijing last year, there's, there's agreement. Okay, we need to make sure that these are not self-improving systems. We need to make sure that we have red lines when we would know they're getting close.

• 44:48Kara Swisher?

  Killer drones, no.

• 44:49Speaker 02

  Yeah. That we could.

• 44:52Kara Swisher?

  Just New Jersey.

• 44:52Speaker 02

  We have the capacity. And so use those things on which there's going to be widespread agreement to build a structure in place.

• 45:00Kara Swisher?

  They can do that. The, the, we have to go faster because of China seems to be a pretty reductive argument in that regard.

• 45:06Speaker 02

  Yeah, I, I mean, I, I certainly, here's what I think is really wrong with that argument. The idea that building regulatory infrastructure is going to slow it down and we should not do that. It's not the way that any part of the economy works, actually, having basic play.

• 45:21Kara Swisher?

  Jillian, they're special.

• 45:24Kara Swisher?

  I don't know if you know that. So, last question. We have a long way to any AI regulation. The Biden administration issued an executive order on AI. Donald Trump's going to repeal it. Trump has Elon Musk. As his best friend, apparently, which is a lovely thing to see. Elon's stance on AI regulation is unclear. He has different ones. He changes quite a bit. He signed a letter for a six-month pause in AI development. Then he funded his company. It's going like gangbusters. I know it seems hypocritical, but that's what he did. He supported a controversial California bill to regulate AI. He's not an uninterested party, right? And he is sitting right next to President Trump at this point, especially on these issues.

• 46:07Kara Swisher?

  And if it's not him, it's one of his many minions. What do you expect from the Trump administration? What worries you? What gives you hope? Very quickly, each of you, we just have a minute and a half.

• 46:17Speaker 04

  Carrie, you asked me to opine about Elon Musk, knowing that I had worked at Twitter.

• 46:21Kara Swisher?

  No.

• 46:23Speaker 04

  A minute and a half will definitely not be long enough.

• 46:26Kara Swisher?

  Yeah, OK.

• 46:26Speaker 04

  What am I?

• 46:27Kara Swisher?

  Not a fan, I got that.

• 46:28Speaker 04

  You know, I don't think.

• 46:30Kara Swisher?

  Me either.

• 46:31Speaker 04

  His idea of ethics and my idea of ethics are the same.

• 46:33Kara Swisher?

  Yeah.

• 46:33Speaker 04

  You know, I don't even know how to answer your question.

• 46:37Kara Swisher?

  What is your greatest hope that would happen and your greatest worry in this administration?

• 46:42Speaker 04

  My greatest hope is that things don't get much worse than they are. That's probably the best we can hope for is status quo. What may happen is a lot of the headway, a lot of the positive things we were talking about, are getting rolled back. I mean, specifically, the EO has been on the chopping block. But also, we're going to have to worry about programs like, you know, programs at NIST, for example, any sort of scientific body that's been doing test and evaluation. I also worry a bit about the brain drain that's going to happen. I think there are a lot of people who sat through one Trump administration and are very dismayed that the narrative this time around seemed to be, oh, well, it wasn't so bad last time. And they're like, do you know what it cost me? So that it wouldn't be that bad. And now they're leaving. So what's going to happen when all of the amazing people, and you're totally correct,

• 47:27Speaker 04

  that all these amazing people are brought in? And they're like, I'm just not going to sit here for another four years.

• 47:31Kara Swisher?

  So that's your worry. Go ahead.

• 47:33Speaker 03

  I expect inconsistency and uncertainty for exactly the reasons you said about Elon Musk. I don't know what's going to happen. And there's a lot of, I can make arguments in both directions. What's the good? So there are a lot of people not in government doing great work. Some of them are sitting next to me on this stage, and that work is going to continue. And to continue to lay the foundation for whenever the government is ready to take action, there will be research there to support it. And so I don't think the external research to the government is going to stop. I think I'm certainly, and I don't know about my colleagues, are going to feel pressure to do more and be more involved because it's now not happening in the government. And we'll see what happens in four years. Can we get a countdown clock for years?

• 48:14Speaker 02

  Yeah, I think the stuff that's driven by national security is going to continue. So I think those pieces of executive order either might get redone in different ways. I think that's going to continue. I think it's very hard to predict whose ear anybody's going to be listening to. I think the China component's going to be very important. I think there's, you know, we just had a bipartisan AI task force report come out that said, we got to invest in developing the science evaluations. And I think that's important. I think we've got bipartisan efforts. So I actually think there may be things that that continue. I don't expect it all to, it's too big. It's too big for it to just, we're going to ignore it and quote unquote, not regulate it.

• 48:56Kara Swisher?

  Right. I see. So it's too big to fuck up. Too big to fail in any case.

• 49:00Speaker 02

  Too big to fail.

• 49:01Kara Swisher?

  Thank you so much, all three of you. Great discussion, and we'll see what happens. We'll check in here in four years. Okay? We'll talk about that and see what happens. Thank you so much.

• 49:09Speaker 02

  Thank you.

• 49:19Kara Swisher?

  On with Kara Swisher is produced by Christian Castro-Rosell, Kateri Yokum, Jolie Myers, Megan Cunane, Megan Burney, and Kalen Lynch. Nishat Kurwa is Vox Media's executive producer of audio. Special thanks to Claire Hyman. Our engineers are Rick Kwan, Fernando Aruda, and Aliyah Jackson, and our theme music is by Trackademics. If you're already following the show, feel free to skip out on any useless homework, and all of it is useless, so skip out. If not, 500 words due tomorrow on whatever. Go wherever you listen to podcasts, search for On with Kara Swisher, and hit follow. Thanks for listening to On with Kara Swisher from New York Magazine, the Vox Media Podcast Network, and us. We'll be back on Monday with more.

• 50:06Speaker 04

  We just moved into our new house and I didn't realize just how close we were to the neighbors. That's when my friend told me about Blinds.com.

• 50:14Kara Swisher?

  Blinds.com isn't a medication, but it can improve how you feel about your home. Customers who shop at Blinds.com experience excellent customer service, from consultation to installation. Side effects include increased privacy and better sleep. Talk to your partner today and see if Blinds.com is right for you.

• 50:31Speaker 01

  Shop Black Friday all month long at Blinds.com. Don't miss huge deals dropping every week.