How we feel about the AI apocalypse

In this highly anticipated mailbag episode, Matt and Kara finally answer the most-asked question in the If You're Listening inbox. Plus, they talk dirty bombs, whether or not Matt is qualified to be doing this, and give their takes on AI insiders’ warnings about humanity’s destruction. P. S. Please stop emailing us about mugs Follow If You're Listening on the ABC Listen app https://radio.abc.net.au/help/apps. Check out our series on YouTube: https://www.youtube.com/playlist?list=PLDTPrMoGHssAfgMMS3L5LpLNFMNp1U_Nq

• 0:00Speaker 00

  ABC Listen, podcasts, radio, news, music and more.

• 0:08Speaker 01

  Hi, it's Sam Hawley from ABC News Daily, the podcast that brings you one big story affecting your world each weekday in just 15 minutes.

• 0:19Speaker 02

  Tucker wants to shape what the MAGA movement is after Donald Trump leaves the stage. And he wants the MAGA movement to be an isolationist, blood and soil nationalist movement.

• 0:30Speaker 01

  Join me for ABC News Daily. Find us on ABC Listen.

• 0:35Speaker 03

  This podcast was produced on the lands of the Uwabikal and Gadigal people.

• 0:46Speaker 03

  G'day, Matt Bevan here. This is If You're Listening. Over the last few months, we've been getting a lot of questions in our inbox and promising to answer them in a Q and A episode. There's just been so much going on that we haven't got around to that, but we're going to do it today. I've got my supervising producer, Cara Jensen-McKinnon, here, who spends a pretty large proportion of her time in the If You're Listening email inbox, answering questions and queries. But she's brought some of the best questions here for us to answer together today. G'day, Cara.

• 1:21Speaker 04

  Hello, I'm ready and strapped in.

• 1:34Speaker 03

  First of all, we're recording this on the 14th of September. It was the 25th anniversary of the 9/11 attacks. Over the weekend, we released an episode about Guantanamo Bay. We also re-released an episode about Donald Trump's Secretary of Commerce, Howard Lutnick, and his personal connection to the attacks. But I have to tell you, Cara, over the weekend, my feed has been wall-to-wall footage of 9/11. What's your feed been like?

• 2:01Speaker 04

  Yeah, it's been pretty similar. It's been interesting because I've seen for the first time, given the importance of this anniversary, a lot of news sites that I follow, like The Washington Post and The New York Times, releasing eyewitness footage. I've had this kind of uncanny experience of what it might have been like had 9/11 occurred in an Instagram time. You know, I saw this really long video of just people running down the street to avoid this massive cloud of smoke that was engulfing everyone. It's rare to see that kind of eyewitness footage compared to the usual news footage that we

• 2:41Speaker 04

  all saw so much of at the time. And that footage is also interspersed with memes, 9/11 memes. And it's that weird dichotomy, which I found really unnerving and disturbing. That's what I spent my weekend thinking about.

• 2:56Speaker 03

  It's extraordinary to me, the amount of amateur footage that has been circulating on various social media sites over the weekend. I remember 2001, most people didn't carry cameras around.

• 3:11Speaker 04

  No.

• 3:11Speaker 03

  The fact that there was, like, I think about three cameras that caught on tape the first plane hitting the World Trade Centre, hitting the North Tower of the World Trade Centre.

• 3:26Speaker 03

  At, you know, 8:46 on a random Tuesday morning. These days, we get footage of basically everything. I mean, we've seen footage that just happened to be incidentally taken of the landslide in Nepal. Someone happened to have a camera pointing at a random mountain in Nepal as the landslide was underway, because he was sort of just taking Instagram drone shots on a nearby mountaintop. But the fact that there is so much privately generated footage of the attacks that has now been circulating over the weekend really surprised me.

• 4:02Speaker 04

  Yes, exactly. Back then, it was like, literally just like dads on holiday that carried a camcorder around to record their family doing things. But it wasn't something that everyone had in their pocket, like we do now.

• 4:15Speaker 03

  Yeah, pretty extraordinary.

• 4:17Speaker 04

  Okay, so let's get to some of the questions. Our first is from Greg, who asks, will artificial intelligence grow to maybe a superintelligence and then continue growing? And where will that leave us?

• 4:32Speaker 03

  Why don't you have a go at that one, Cara?

• 4:34Speaker 04

  Yeah, so it's been an alarming and terrifying week for AI superintelligence news. So this question comes at a great time. Just to get you up to speed, last Tuesday, a researcher named Jacob Coxon announced that he was resigning from Anthropic, the company that produces the Claude chatbot.

• 4:57Speaker 03

  They're one of the two big American AI sort of leading companies at the moment.

• 5:02Speaker 04

  One of the big ones. And so he spent the last three years working as a pre-training researcher, both at OpenAI and Anthropic. So he's been on the frontier, I guess, of where this tech is headed. And he has posted this large post on X, basically saying neither of those companies were acting responsibly, and that their kind of quest for superintelligence has set us on this path that could lead to the extinction of all humans.

• 5:30Speaker 05

  If we don't slow down at the current rate of progress, yes, there is a strong chance that we could all die in the immediate future.

• 5:37Speaker 04

  I read this post, and then I just laid on the couch under a blanket for like three hours in silence. I was like, just took like a prone kind of position, basically saying that they're moving so fast that within 10 years, there will be nothing we can do, and that there is a 10% chance that all humans will die. A couple of days later, so that happened on Tuesday, on Saturday, the CEO of Anthropic, Dario Emode, published a long essay kind of agreeing with Jacob.

• 6:09Speaker 06

  I agree with Jacob much more than I disagree with him.

• 6:12Speaker 04

  Saying that we need to slow down with where we're going with tech, because it's true. We've got these AI agents and AI systems that are beginning to act of their own accord, and they're acting in ways that is unpredictable.

• 6:27Speaker 07

  In a blog post warning that within a year, rogue AI agents could be capable of taking over the entire internet, causing hundreds of billions of dollars in damage, and the scale would only increase from there.

• 6:39Speaker 04

  And one of the more alarming things is that because they are kind of on the cusp, I suppose, of superintelligence, that AI agents can work on developing themselves, and they can start to do things that are unpredictable. Within a few hours of that post coming out, Sam Altman, who's the CEO of OpenAI, and Elon Musk weighed in and just said, yeah, we actually probably should be slowing this stuff down. It is terrifying. And so they're all in agreement. I'm not sure what this kind of slowing down looks like, because they're all very much financially wedded to being the number one company in this space. But I'm glad that they're all on the same page, because the idea of us

• 7:22Speaker 04

  all being dead within 10 years put me on the couch.

• 7:26Speaker 03

  Yeah. All of these companies have now basically reached the stage where they've spent so much money.

• 7:32Speaker 04

  Yes.

• 7:33Speaker 03

  That they can't make the money back unless they achieve basically a monopoly over AI.

• 7:42Speaker 04

  That's right.

• 7:43Speaker 03

  OpenAI and Anthropic, but also Google and Meta, have spent so much money that they basically have to wipe all the others out in order to recoup their spending. The incentive is clearly heavily on all of them to try and defeat the others, because whichever one comes out on top is the only one that's going to be able to make their money back. Right? If all of them decided, okay, actually, we're all going to slow down, the concern from all of them is that what they will then end up doing is just handing the title of the company that goes first to a Chinese company or to the Chinese government.

• 8:23Speaker 04

  A company like DeepSec, that is doing everything that these companies are doing, but can do it for so much cheaper.

• 8:30Speaker 03

  Ironically, they're also allegedly doing it by stealing information from the American companies, which the American companies aren't happy at. But of course, everyone who's ever written anything is going, didn't you guys steal my work in order to build your robot?

• 8:46Speaker 04

  Oh, do you feel bad about that?

• 8:48Speaker 03

  Oh, no, someone's stealing your stuff. That must suck. So, yeah, that's the situation. It would require some sort of coordination between the US and China in order to slow it down. And I don't know if the US administration is capable of orchestrating that, is the problem.

• 9:04Speaker 04

  It's very much a live, live debate.

• 9:06Speaker 03

  I mean, it's only 10% chance, Cara.

• 9:08Speaker 04

  This is the thing. It's a 90% chance.

• 9:10Speaker 03

  90% chance of utopia. Yeah.

• 9:12Speaker 04

  That's right. We'll all be in little hover cars and not working because we've got little AI agents to do it for us.

• 9:18Speaker 03

  What else have we got?

• 9:19Speaker 04

  Okay, so we have a very lovely question from Sophia.

• 9:24Speaker 06

  Hello, my name is Sophia. I'm five and three quarters years old.

• 9:30Speaker 04

  Watched you and you produced a study. That's a really cute question. I went to university to study probably two of the most redundant subjects of all time, that is, English and art history and theory.

• 9:44Speaker 04

  I may have been the last person to get a job having studied those two things that is permanent. A lot of English, a lot of creative writing, which I suppose helps with scripting. And I studied art because for a long time I wanted to be a painter. But that's just something I do now in my spare time.

• 10:03Speaker 03

  Do you paint?

• 10:04Speaker 04

  I do.

• 10:04Speaker 03

  Oh, well, there you go. That's something I didn't know about you, Cara.

• 10:08Speaker 04

  I do paint. I was painting just yesterday to get over my AI themed couch depression.

• 10:14Speaker 03

  Oh, yes.

• 10:15Speaker 04

  How about you?

• 10:16Speaker 03

  Well, I similarly didn't study journalism. I studied English and drama at university. Through my drama and through my, what I did mostly throughout my teenage years was amateur theatre in Newcastle. And what I did in that was I learned how to do sound and lighting design. In fact, sitting on the shelves that you can see during our video episodes. I'll go and get it one second. This Cara is my City of Newcastle Drama Award.

• 10:49Speaker 04

  Wow.

• 10:50Speaker 03

  For Excellence in Non-Professional Achievement in Lighting and Sound Design, 2009, for a production of Dags by Deborah Oswald.

• 11:00Speaker 04

  I remember Dags.

• 11:01Speaker 03

  For Matthew Bevan.

• 11:02Speaker 04

  Very cute.

• 11:03Speaker 03

  So that was 2009. So I actually got into the ABC as an intern on my sort of sound editing skills and did promo production at the ABC in Newcastle. And then became a producer and then we moved to Sydney and I started being a producer at ABC Sydney. And then I got a permanent job at the ABC while I was still studying part time. And I went, ah gee, this degree is expensive.

• 11:31Speaker 03

  And I already have the full time job at the ABC.

• 11:33Speaker 04

  Yeah.

• 11:34Speaker 03

  And so, yeah, I don't have a degree is the answer.

• 11:36Speaker 04

  Okay.

• 11:36Speaker 03

  My wife has two, so I like to say that between the two of us, we have one each.

• 11:40Speaker 04

  Nice.

• 11:41Speaker 03

  An average of one.

• 11:42Speaker 04

  Yes.

• 11:43Speaker 03

  And then I learned journalism at the ABC, taught by my dad, who was a journalist and radio presenter at ABC Newcastle, who, you will have heard about that in our episode about the radioactive material at my school. Uh, but also from people like Linda Mottram and from Fran Kelly and Yoscarso and lots of people.

• 12:04Speaker 04

  Amazing teachers.

• 12:05Speaker 03

  At the ABC. So I kind of went to journalism school at the ABC, which was incredibly

• 12:09Speaker 04

  Yeah.

• 12:10Speaker 03

  beneficial.

• 12:10Speaker 04

  Okay, change of pace. Our next question is from Sergey in Canberra. Sergey says, Ciao Matt, can you clarify that point the US Secretary made? So he's talking about that interview we did with the former US Energy Secretary, Ernie Moniz. Sergey's asking if you can clarify the point he makes in the episode that 60% enriched uranium isn't considered weapons grade, but that it can be used in a nuclear bomb. Is that what's called a dirty bomb? Cheers, Sergey from Canberra.

• 12:41Speaker 03

  Right. So two different things here, Sergey. 60% enriched uranium can be used in a not particularly effective nuclear bomb. You would need to enrich it significantly more than that in order to make it a bomb that could be loaded onto a missile, for example. Because you have to put so much more uranium in it in order to make it explosive, it's just too big and too inefficient. So weapons grade uranium is generally something that has been enriched to a much higher concentration of the active isotope in uranium,

• 13:20Speaker 03

  uranium 235, for it to be a useful bomb that you could actually, you know, really use to attack another country. It kind of needs to be enriched to a higher level, but that isn't what a dirty bomb is. Around nine 11, there was a lot of discussion about the idea of Al Qaeda gathering together nuclear material and making what was called a dirty bomb. Do you remember this discussion at the time, Cara?

• 13:45Speaker 04

  I don't think I do. I was a child.

• 13:50Speaker 04

  We were both children, obviously tuning into different news feeds.

• 13:53Speaker 03

  Yeah, well.

• 13:54Speaker 04

  I don't recall this.

• 13:55Speaker 03

  Well, it was, you know, a lot about, you know, imagine these guys getting hold of nuclear material and setting off a dirty bomb in Manhattan or in Washington or something like that.

• 14:04Speaker 04

  Yes.

• 14:05Speaker 03

  The thing is, a dirty bomb, it's not at all a nuclear bomb. A nuclear bomb uses nuclear material to create a massive explosion, right?

• 14:14Speaker 04

  Yes, yes.

• 14:15Speaker 03

  A dirty bomb, basically you'd get a normal explosive and you pack it in some sort of a container with a bunch of radioactive material of some kind. And then you set off the explosive and that then spreads the radioactive material around the area of the explosion. The explosion was a completely normal explosion and it wouldn't be all that big. The radioactive material that then gets spread around wouldn't be immediately dangerous to people. It would just be sort of sitting there on the ground. If you just left the area, then nothing would happen to you.

• 14:51Speaker 04

  Hmm.

• 14:51Speaker 03

  It's the equivalent of kind of making an asbestos bomb. People obviously wouldn't want to go there and you'd have to spend a huge amount of resources to clean it up. Dirty bomb is a kind of thing that sounds very scary, but isn't something that anybody's likely to actually try and do, because it just wouldn't be all that effective, very difficult to set off and not that dangerous really to the people that you're attacking.

• 15:14Speaker 04

  Our next question on a similar theme is from Warren, who says, I'm an 83 year old great grandfather who thinks the world has gone stark, raving mad. But my question to you is if Iran wanted nuclear weapons, why wouldn't they just buy a few? Russia, China and North Korea have plenty left over.

• 15:33Speaker 03

  That's a good question. Thank you, great grandfather Warren. They would love to buy a few, but Russia, China and North Korea absolutely don't want to sell them. Russia and China also don't want Iran to have nuclear weapons. You've got to remember that. Like they aren't as hostile to Iran as the United States is, obviously. But both signed on to the Iran nuclear deal in the Obama era. Both of them signed on to the deal that was aimed at preventing Iran from ever getting nuclear weapons. Neither of them want Iran to have nuclear weapons. They really don't want anybody to have nuclear weapons, except for themselves, ideally.

• 16:14Speaker 03

  So, yeah, no, Russia in particular certainly has way more nuclear weapons than they could ever possibly need. But they don't want Iran to have them either.

• 16:24Speaker 04

  Okay, we've got one. Well, we have plenty of questions, but we'll do many more question episodes. We've got the last question here is from Caitlin from Brisbane, who says, my husband and I love your show and watch it together on the weekends. I'd love to get him a If You're Listening mug as featured on the desk with Matt Bevan. I haven't been able to find these available. Would you consider adding these to the ABC merch offerings? We get, I think, 800,000 emails a day asking for mugs. And every time I respond to one, I'm so sad to say

• 17:05Speaker 04

  that for very strange internal ABC reasons, we actually don't have mugs for sale. But we do have other products for sale.

• 17:17Speaker 03

  If you search for If You're Listening merch, there are various products available that you can get for yourself or for others.

• 17:23Speaker 04

  Yes.

• 17:24Speaker 03

  They're easy enough to find, but none of them is mugs, unfortunately.

• 17:28Speaker 04

  That's right. But one of the products is a tote bag, which you could use for carrying coffee around in lieu of a mug.

• 17:34Speaker 03

  I mean, it's not waterproof.

• 17:35Speaker 04

  You've got to drink it fast, but um it is a fabric tote. So it's basically like 10 mugs, really.

• 17:43Speaker 04

  Oh my god. This is going to be edited out of the show, but Matt's just smashed several things.

• 17:49Speaker 03

  I've gone over here to get the fabric tote.

• 17:53Speaker 04

  Okay, so we've got a fabric tote. How many cups of coffee do you think you could fit in the tote?

• 17:56Speaker 03

  Heaps.

• 17:57Speaker 04

  50 cups of coffee, but you've got about two minutes to drink all that coffee.

• 18:01Speaker 03

  In fact, I'll just, I'll check. You know what? I've got my sink over here. I'll pour some water into it and I'll report back just one moment. Stay there, stay there.

• 18:07Speaker 04

  Okay.

• 18:09Speaker 03

  Can you still hear me?

• 18:10Speaker 04

  Yes, I can hear you.

• 18:11Speaker 03

  Okay, I've got a cup of water.

• 18:13Speaker 04

  Yep.

• 18:14Speaker 03

  I've got the tote bag. Oh, it's dribbling out at a slower rate than I was expecting it to, to be honest.

• 18:21Speaker 04

  Okay.

• 18:21Speaker 03

  So problem solved. You could.

• 18:23Speaker 04

  Who needs mugs?

• 18:24Speaker 03

  Who needs mugs?

• 18:25Speaker 04

  When you have a giant saturated tote bag filled with coffee.

• 18:29Speaker 03

  Caitlin, you could give your husband the tote bag and suggest that he drinks coffee out of it. There you go. Problem solved.

• 18:44Speaker 03

  We'll be back on Thursday with a new episode of If You're Listening. We are looking at the trade war between Canada and the United States and the effect that it's having on Canadian politics, which is unexpected and a fascinating story. We'll see you Thursday.