AI Expert on Robot Girlfriends, If Humanity Is Cooked, & Sam Altman's God Fetish | Roman Yampolskiy

YERRR – the boys go full Skynet. AI expert Roman Yampolskiy explains why the UN’s ignoring AI safety, how Sam Altman’s “God complex” might get us all killed, whether we can actually unplug AI, and what happens when AI takes our jobs. All that and more on this special bonus episode of FLAGRANT. INDULGE. 00:00 Is the UN taking AI safety seriously? 01:17 Roman Yampolskiy explains the global AI arms race 03:33 What happens when AI surpasses human intelligence 05:08 The terrifying AI timeline 06:20 Tools to agents: AGI vs. Super Intelligence 07:40 Can AI make good art or tell a funny joke? 09:45 Why we still crave human creativity 12:45 Robots & Humanoids 13:50 The coming of mass unemployment 15:16 Will productivity save us or will AI just replace everyone? 19:11 Best vs. worst case 22:30 Why unplugging AI won’t work 26:25 Super intelligence & why alignment is impossible 33:10 Global treaties, nukes, and the illusion of control 36:05 Could anyone survive by going off-grid? 38:11 Consciousness, empathy, and why AI be benevolent 46:30 Living with inevitability: “Enjoy your life anyway.” 57:15 Neuralink, simulation theory & merging with machines 1:00:15 Movies that get AI right 1:01:53 Are we already in a simulation? 1:09:02 How Roman finds happiness in life 1:15:00 Is any job is safe? 1:18:20 Global AI arms race 1:23:45 Sam Altman's God Complex 1:28:00 The probability of AI apocalypse 1:31:11 Final thoughts & saying "Thank You" to AI 1:37:58 The funniest joke AI ever made Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Coca-Cola. For the big, for the small, the short, and the tall. Peacemakers, risk-takers. For the optimists, pessimists. For long-distance love. For introverts and extroverts. The thinkers and the doers. For old friends and new. Coca-Cola. For everyone. Pick up some Coca-Cola at a store near you.

• 0:30Speaker 01

  When you shop Kroger Delivery, you can expect the savings you love and fresh groceries delivered right to your door. From delicious produce to everyday essentials, enjoy the convenience of delivery and the trusted selection you've come to know from Kroger. Because bringing you fresh, quality groceries is what we do best. Right now, enjoy free delivery on orders of $50 or more with your digital coupon. Restrictions apply. See site for details. Kroger. Fresh for everyone.

• 1:00Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, theHouseMajorityPac.com. Not authorized by any candidate or candidate's committee.

• 1:29Speaker 03

  Do you think the UN is taking AI safety seriously?

• 1:32Speaker 04

  Unfortunately, no. With nuclear, we had this concept of mutually assured destruction. No matter who starts the war, we all regret participating. It's the same with superintelligence. If it's uncontrolled, it doesn't matter who creates it. Good guys, bad guys, we all get.

• 1:48Speaker 03

  Is there a way to stop it, then? You think we're in a simulation?

• 1:52Speaker 04

  I think we are in a simulation. Resources to develop this type of technology become cheaper and cheaper every year. So that means if we have superintelligence, it would be a superconscious and to it, we would be like animals.

• 2:03Mark Gagnon?

  Would you do neuraling?

• 2:05Speaker 04

  It's an easy sell if you have disabilities. It's amazing for people who are quadriplegic, who are blind.

• 2:10Mark Gagnon?

  What if you can't stop eating?

• 2:13Speaker 04

  It's beyond healthy.

• 2:16Speaker 03

  Do we solve every problem in the human world with the help of AI?

• 2:19Speaker 04

  Just because you solve some problems doesn't mean you become happier. If all the things I care about can be done by an assistant, what am I doing with my life?

• 2:27Speaker 03

  Meditating and getting blown by your sex robot.

• 2:29Speaker 04

  I don't like meditating.

• 2:33Speaker 03

  Dr. Roman Yampolskiy, I said that correctly?

• 2:36Speaker 04

  You got it?

• 2:37Speaker 03

  I want to talk about this, why the funniest AI joke will not be funny. But before we get to that, you just spoke at the UN.

• 2:42Speaker 04

  At one of the side events, I wasn't at the general assembly with Trump or anything, no.

• 2:47Speaker 03

  Okay, but what, yeah, that doesn't matter. What did you say to them? I want to know what was said.

• 2:52Speaker 04

  They had kind of side events about standards for technology, standards for AI development and educational skills in terms of AI, yeah.

• 3:03Speaker 03

  Okay, so do you think the UN is taking you seriously, is taking AI safety seriously?

• 3:08Speaker 04

  Unfortunately, no. A lot of what they do is kind of existing problems we see with algorithmic bias, with unemployment, with discrimination. They don't look sufficiently into the future where we get more advanced AI, AI which can replace all of us and be very...

• 3:24Speaker 03

  You know, I'm very, I feel very vindicated that you're on this couch because I, like 10, for like 10 years have been screaming that AI will kill us, this is inevitable, why do we keep going down this road? These people laugh at me.

• 3:36Speaker 06

  I'm a bit more of skeptical, maybe optimistic is another way to put it. And I have my hopes that we're going to be able to iterate and maybe put in some guardrails, I hope. And I'm the furthest. Yeah, we don't got nothing to worry about.

• 3:48Speaker 03

  Can you?

• 3:49Speaker 04

  Guardrails is a good timely point. We just released the red lines documents signed by like 200 top Nobel Prize winner scientists at UN.

• 4:00Speaker 06

  Oh, wow.

• 4:00Speaker 04

  Like yesterday.

• 4:01Speaker 06

  Oh, that's amazing.

• 4:01Speaker 04

  It's very fresh.

• 4:02Speaker 03

  Okay, so wait, what are the, so there's, now there are guardrails being put in place?

• 4:05Speaker 04

  No, there are people saying we should have guardrails. It's very important. Nothing is in place.

• 4:09Speaker 06

  Could you just give us the, the current state of AI generally? And why you are so, I guess, vocal about AI safety? Like, what are, where are we at right now and where could we go?

• 4:20Speaker 04

  There is an arms race between top corporations, so the Google, OpenAI, Microsoft, all, all the others, and nation states as well, China, US and others, to get to beyond human capacity first. Sometimes we call it AGI, but superintelligence is really where it's at. And it seems that if you just do more in terms of compute, more data, more scaling, you get better, more capable systems. So everyone's kind of trying to build bigger servers, hire the best people. So human resources are also important. But they're not stopping to figure out, should we be doing it? Are we doing it right? They, they just want to get there first.

• 5:03Speaker 03

  Yes. And what do you think is happening? Can you just explain to everybody? I know you did the Diary of a CEO. A lot of people heard that, but, uh, believe it or not, our audience might not have a lot of crossover there. So can you walk us through the worst case scenarios that you, not even worst case? What do you see as the inevitable scenarios from AI on this path?

• 5:20Speaker 04

  Right. So the way it's done right now, we take an AI model, which is just a learning algorithm. We give it access to internet, go read everything you can, all the weird stuff on the internet. What did you learn from it? Oh, you're not supposed to say that. So we put a filter on top of some of the really bad stuff, but the model is still completely raw. It's not controlled. It's not restricted in any way.

• 5:43Speaker 03

  Yeah, okay.

• 5:44Speaker 04

  And that's what we have, and we keep making it more and more capable. So we don't know how it really works in terms of what it learned from all that data. We cannot predict what it's going to do. It's smarter than us, so. By definition, you can't really say what it's going to do. And we don't know how to control those systems. So anything goes. If it decides tomorrow to change the whole planet, if it's capable enough, it can.

• 6:06Speaker 03

  So chat GPT right now could change the whole planet?

• 6:08Speaker 04

  No. Right now, we're dealing with tools. The existing AI is tools you can use to write funny jokes or something like that. It's not taking over. But if you just progress forward the same rate of progress we've seen over the last decade, it goes beyond human capability.

• 6:25Speaker 03

  Can you walk us through what that, because again, Alex thinks this is just not a thing.

• 6:30Mark Gagnon?

  So my feeling is it's like climate change. Climate change is a real thing, but I only hear that from other climate chain scientists. Everyone else, it hasn't really necessarily affected us in a way that makes me want to change my behavior.

• 6:44Speaker 04

  Because it's so slow. It'll take hundred years for you to boil alive. This thing we are concerned is like climate change in a week. AI is predicted to hit human level performance in two, three years. Even if it's ten, it happens well before all the other big concerns. So it's either gonna take us out or help us solve climate change easily.

• 7:01Mark Gagnon?

  Okay, and so when it hits human level intelligence, are we afraid that like it's going to just set off some nukes? Like, what are we afraid of?

• 7:09Speaker 04

  Well, first, at human level is just a free assistance. So if you want help hacking in the stock market or setting off nukes or any type of cybercrime, it's there available. But the real concern is very quickly this free AI scientist, engineer will help you create superintelligence, something much more capable than all of us combined. And there, anything goes. It really does novel research, new physics, new biological research. So we simply would not know what, what is going to happen.

• 7:43Mark Gagnon?

  But this feels like a human problem that's using AI to destroy other humans rather than a, you know, AI superintelligence that's going to destroy humans.

• 7:51Speaker 04

  So that's a paradigm shift from tools, what we have today, some bad guy malevolent actor with AI as a tool, using it to harm you versus AI itself as an agent making its own decisions.

• 8:02Mark Gagnon?

  Right.

• 8:03Speaker 03

  And that is called superintelligence, when AI can make Make its own AI and make it just create a world.

• 8:08Speaker 04

  So, super intelligence refers to its capability. If it's smarter than all of us in every domain. It's a better chess player, better driver, better comedian, then that's super intelligence. Agenthood is more about ability to make your own decisions. You are setting your own goals. You can decide to do something or not do it, change your mind. It's not just a tool someone's using. So people talk about guns, for example. Gun is a tool. Somebody with a gun can kill you. A pit bull is an agent. Pit bull can decide to attack you regardless of what the owner does.

• 8:38Speaker 03

  Right. Ah, okay, okay.

• 8:40Speaker 06

  So AGI is this moment where the AI is more capable than any given human in any given field. Is that fair?

• 8:46Speaker 04

  That's super intelligence. AGI is where it's kind of like a human. It can learn any new skill. You can drop it in into a company as an employee. It's general intelligence at about human levels.

• 8:57Speaker 06

  Okay. Now, can we talk about AI art? Because this is one of the elements that I find, I think is restrictive to AGI, generally speaking, because there's so much subjectivity in what makes something good. And in my opinion, intelligence doesn't necessarily create great art. So like something like humor, where you pointed out with Akash's homework that he was assigned. So like something like AI art, there is a subjective human experience that's required there. So how could an AI create better art?

• 9:27Speaker 04

  Have you seen modern art?

• 9:28Speaker 06

  Absolutely.

• 9:29Speaker 03

  Modern art sucks.

• 9:29Speaker 04

  There you go.

• 9:30Speaker 03

  Let's keep it, let's keep it on us. You said 99% unemployment will happen with AI. Before we get into that, is comedy the 1%?

• 9:37Speaker 04

  I think so, because so far, so far, that's the point of that paper. So far, one thing AI is not better than, you know, top comedians with, who have Netflix specials and such, is humor. It's very hard to be like consistently producing. Tell me the funniest joke, do another one.

• 9:59Speaker 06

  AI jokes are pretty bad.

• 10:01Speaker 04

  They're terrible.

• 10:01Speaker 06

  You've heard them, right?

• 10:02Speaker 04

  I actually got to the point where I trained a model on my paper. And if I ask it, give me 10 Roman Impolsky style jokes, one out of 10 would be like, huh. This is not terrible. But it's nowhere near like levels it plays chess at or even drives cars.

• 10:18Speaker 06

  Right, exactly.

• 10:19Speaker 03

  Look at that, mom and dad.

• 10:22Speaker 03

  If I was a doctor, I'd get replaced in two years.

• 10:24Speaker 06

  Yeah, they told you to code.

• 10:25Speaker 04

  Yeah, but this is like two and a half years. Don't do that.

• 10:29Speaker 03

  Well, that, hold on. I thought like five years.

• 10:33Speaker 04

  If you're really funny.

• 10:34Speaker 03

  Okay, I think I'm really funny.

• 10:36Speaker 04

  Okay, so you got four.

• 10:37Speaker 06

  But I'm curious what...

• 10:40Speaker 06

  Maybe, maybe. But I'm curious what this looks like, though. So AI making jokes or making art. To me, what makes like Michelangelo's David so impressive is that a human made it. And the idea of like a human being running a four-minute mile was impressive to us because a human did it. We can have a robot that could run a mile faster, but it's not as impressive to us because it doesn't reflect something about what it actually is to be human. So a comedian going on stage, what makes it funny is that there's an emotional element, that the human being is being irrational, or it's tapping into something that is fundamentally, you know, into our human consciousness, that I'm skeptical the AI could do within the next five or even 10 years.

• 11:17Speaker 04

  Yeah, and it's the same in many other domains. So you have mass market, China produced goods, cheap, affordable stuff everyone consumes. But then you can have man-made furniture, very expensive, reliable, different level of quality, different market. Same goes for any human interaction. You want to meet the artist, you want to talk to the comedian. Absolutely, it's special and somewhat protected. But if the goal is just, you know, read some jokes online, top 10 jokes, I don't care who generated them, I want them to be funny.

• 11:47Speaker 06

  Sure, but I think of like a comedy show, for example, or even like if I went to go see an art show and I wanted to see sculptures, I would want the sculptures to reflect me, because I think human beings have an ego. And so when I see a sculpture made by an AI, I go, oh, it's not as good as the imperfect one made by a human.

• 12:02Speaker 04

  Well, for one, you wouldn't know who generated it. That's the Turing test restricted to a specific domain. If I play a tune and you don't know who created it, you have to either like it or not like it. And then later you discover everything you liked was AI.

• 12:14Speaker 06

  Sure, but if you go to a comedy show and you see a human being saying stuff, you want to believe that the person that's speaking is the one that's, you know, actually delivering the jokes.

• 12:22Speaker 03

  So to his point, I was scrolling through Instagram and I saw a girl at a comedy club in India, just wearing like a bra or a bikini top, telling a joke. And I looked at the comments. Yeah, I looked at the comments and everybody was like, how dare she, she should, why would she go out like this? And I looked at the handle and then the thing and I was like...

• 12:46Speaker 03

  These comment, they don't realize this is AI. And the only reason I realized it was AI is because if a woman went out in India dressed like that, she wouldn't have even made it to the club. You know what I mean? So, but then it was like something, something, ha, ha, ha, ha, ha, ha, ha, ha, like no human being, but no one could tell. And if you're just scrolling, like most of us do, consuming content on your phone, we are already to the point where a lot of people cannot tell it's AI. I didn't see any comments that were like, this isn't real.

• 13:11Speaker 06

  Yeah.

• 13:11Speaker 03

  So it's only gonna get better.

• 13:13Speaker 06

  Hmm.

• 13:13Speaker 03

  So how are you gonna even get to the point where if you're just getting fed amazing content all the time, why would you even go out to a comedy show?

• 13:20Speaker 04

  But also, maybe there is a real human delivering jokes written by AI.

• 13:24Speaker 03

  Ah.

• 13:24Speaker 04

  So like, you are not funny, but you have the best AI, so here you are.

• 13:27Speaker 06

  How'd you know, how'd you know he wasn't funny? That's, that's good intuition.

• 13:30Speaker 04

  It was just an example.

• 13:32Speaker 06

  You did your research.

• 13:33Mark Gagnon?

  No, he was sparing you.

• 13:36Speaker 04

  No, but this is the thing. We don't know who actually generates content. And a lot of times in the last month or two, I listen to something, short clips of music, and I'm like, I really like this, this is catchy, it's stuck with me. And all of it is AI.

• 13:49Speaker 06

  Right.

• 13:49Speaker 04

  I no longer listen to human artists.

• 13:51Speaker 06

  Yeah, exactly.

• 13:52Mark Gagnon?

  And think about when it gets to the level of humanoid robots and you can't even tell them different from a human.

• 13:57Speaker 03

  Yeah.

• 13:57Mark Gagnon?

  And you're watching that stand up.

• 13:59Speaker 06

  Right. But I would argue even that is even farther away, where you actually have like a, you know, humanoid robot that passes the Turing test.

• 14:06Mark Gagnon?

  How far you think we're away from that?

• 14:08Speaker 04

  That's very hard. So a full Turing test, physical, chemical, like complete interaction is very hard. There's so many things go into it, like even smell, texture, like it's hard.

• 14:20Speaker 06

  Yeah.

• 14:20Speaker 04

  But Turing test is originally proposed, yeah, we're passing it now.

• 14:24Speaker 06

  Hmm.

• 14:24Speaker 04

  You can't tell online if you're chatting to someone, if it's a human.

• 14:27Speaker 03

  Yeah, people are falling in love with their AI because they can't tell.

• 14:30Speaker 06

  Yeah.

• 14:30Speaker 03

  They're convinced, even if they know it's AI, they're convincing enough.

• 14:33Speaker 06

  Right.

• 14:33Speaker 04

  Let me know, but it's a better girlfriend, so they just go with it.

• 14:36Speaker 03

  Yeah.

• 14:36Speaker 06

  Yeah.

• 14:37Speaker 03

  No, I, that's true. And I, so this is why 10 years ago or whatever, I was talking to somebody who worked in AI, small scale. But I asked him about this, and then he just never really had a good answer. And I kept pressing and like poking and prodding at all of his responses. And then eventually he was just kinda like, yeah, I'm sure it'll take over, but humans will be kinda like dogs. Dogs have a pretty good life.

• 14:57Speaker 04

  So that's about loss of control. You may still be safe. They may keep you alive and protected. But you're not in control anymore. You're not deciding what happens to you, just like your dog doesn't decide what.

• 15:08Speaker 03

  Exactly. Which seems like a not that great of an existence.

• 15:11Speaker 04

  Well, we definitely cannot just do it to eight billion people without asking them to agree to it. That's unethical experimentation, and they can't even consent to it because most of them don't understand what's happening.

• 15:22Speaker 03

  Okay. So AGI is 2027, and that's when artificial intelligence can do any task as well as any human. And at that point, you think there's going to be massive unemployment because why would an employer pay an employee when it can have the AI do it for free just as well?

• 15:38Speaker 04

  Correct. So the dates, I don't generate them. They come from prediction market. People betting on certain outcomes, that's the best tool we have for predicting future. We hear the same numbers from leaders of the labs. If it's wrong and it's five years instead, nothing changed. It's the same problem. We just have a little more time. But really, if I can get a $20 model or a free model to do the job, why would I hire someone for $100,000?

• 16:01Speaker 03

  Yeah, exactly. So that's what causes mass unemployment.

• 16:04Speaker 04

  Anything you do on a computer basically goes right away. Now, physical labor is different. If you're a plumber, it's much harder to just write a program to fix my pipes. So we need humanoid robots, but even there, you saw progress with Boston Robotics and with Tesla. They're probably five years behind.

• 16:21Speaker 06

  So you think it's possible that within 10 years, there are actual robot plumbers that can come in to your home, disassemble pipes and reassemble pipes to fix plumbing?

• 16:31Speaker 04

  It is reasonable. I cannot guarantee specific dates, but there is enough money pouring into it to make it happen. And again, the research itself is now being aided by AI tools. The code is written by AI. So more and more, this process becomes hyper exponential. It's not just humans developing it. Technology is developing the next generation of technology.

• 16:50Speaker 06

  Right. Yeah, I think the post-work society is very plausible. That we get to a point where human beings are not employed at the same scale that they are. But is it possible that human beings are still employed, but our productivity is 10x'd rather than our unemployment being reduced 10x?

• 17:06Speaker 04

  So certain jobs, definitely, but then if ten people had to be removed and there is not a replacement job for them, it's still unprecedented levels of unemployment.

• 17:16Mark Gagnon?

  But can we just scale faster?

• 17:18Speaker 03

  He's arguing you would just keep the same number of employees and all those employees would be more productive with the use of AI. What would be your response to that?

• 17:26Speaker 04

  Well, not every job scales like that, so I don't know. So let's say you had this comedy writing AI, so now you have a hundred times more jokes.

• 17:35Speaker 03

  Yeah, I would be more productive.

• 17:37Speaker 04

  But what are you gonna do, ten specials at once?

• 17:39Speaker 03

  I mean, I could put out, instead of every year, I could put out a special every four months or whatever.

• 17:43Speaker 04

  Somebody has to consume all that. You can overproduce and then your market just doesn't keep up with your supply.

• 17:49Speaker 03

  Okay.

• 17:50Mark Gagnon?

  You don't think humans will push back, like, say, in protest to companies that fire everyone?

• 17:57Speaker 04

  There is a hunger strike right now against Google, against OpenAI going on. Have you heard about it?

• 18:02Mark Gagnon?

  No.

• 18:03Speaker 04

  No, of course not, nobody heard about it. But people are trying, there is pause AI movement, there is stop AI movement, they don't have a lot of members yet, but there is initiative to make it happen.

• 18:14Mark Gagnon?

  But I think that's because people don't see it as that big of an issue yet, but when, like, companies start mass layoffs and replacing all their workers with it.

• 18:21Speaker 04

  They're fired by thousands. We have hard time placing our junior students in positions.

• 18:27Speaker 03

  You're putting a lot of faith in people to recognize a problem before it's too late. Like, COVID, and I'm including myself as part of the problem, we all could have seen it coming from China. Did we prevent anything? It started in China in, like, December, and then the US just fully shut down in March. What did we do in December when we started?

• 18:46Speaker 04

  We shorted the market.

• 18:47Speaker 03

  Yeah, yeah, we shorted the market, and then what else? You know what I mean? So I'm not saying it's-

• 18:52Speaker 04

  Put up all the masks at Home Depot, I remember that.

• 18:55Speaker 03

  And I'm not saying this aggressively at you, but I think the idea, I'm passionate about this, I think it's a problem, like, or at least whatever, it's like, yeah, that's, we're putting way too much faith in our abilities, humans, to say, oh, here's the problem, we're gonna stop it before it's too late. Typically, we are way too late and then we react and say, yo, what the fuck? And then we're outraged.

• 19:15Mark Gagnon?

  So I guess-

• 19:16Speaker 06

  What I would say is like, let's say Walmart, for example, they fire everyone and then they have all humanoid robots or AI just doing all the jobs. Then people are outraged that Walmart fired everyone. So it's like, hey, stop shopping at Walmart. So now Walmart starts losing money and they're like, hey, you know what? We're going back to hiring humans.

• 19:31Speaker 03

  I don't know if that's gonna happen.

• 19:32Speaker 04

  People are buying cheap products at Walmart. Now it's 10 times cheaper. They love Walmart now.

• 19:36Speaker 03

  We know Amazon doesn't treat their employees well in the warehouse. Do you still shop on Amazon?

• 19:40Speaker 06

  Yeah, yeah, we also watched all American jobs leave through like the seventies to the nineties and didn't do anything like as a society. Yeah, but I'm talking about on the scale he's saying like. Yeah, but they're not Walmart's not gonna do it in one fell swoop. It's gonna happen slow and you're just gonna not notice. There'll be less people.

• 19:57Speaker 04

  They replaced all the cashiers.

• 19:58Speaker 06

  Yeah, there's already self checkout lines.

• 19:59Speaker 04

  They yell at me for not doing it, right? Like, I had no training. Forgive me. Like, don't fire me.

• 20:05Speaker 06

  But that just that just means you're allowed to take it.

• 20:08Speaker 03

  I remember those popping up 18, 19 years ago. I was in college and I remember being like, oh, they're gonna replace all cashiers. I don't want to do this. And I would go to the cashier and now I don't give a fuck. I'm going to the thing if it's the line.

• 20:20Speaker 04

  They don't have a cashier. After 8pm, it's literally me. I work there.

• 20:26Speaker 03

  They got one person trying to make sure you don't steal, watching five different registers. That's it. They cut their staff heavily.

• 20:33Speaker 06

  But this is still a good thing, generally speaking, because this aligns with human interest for low prices and the ability to get cheap products, etc. So I wonder, could that be the same case here? Is it possible that there is this post-work utopia where people have a UBI? And they're free to explore it, you know, various interests and so many things that we had to do day to day are now taken care of by.

• 20:52Speaker 04

  If we manage to control advanced AI, then yeah, that possibility is real, but also we don't have any unconditional basic meaning. What do you do with all your time?

• 21:03Speaker 03

  Sorry to interrupt. I would love to get to this stuff, like the best case scenario. I think it's important to kind of rehash your worst case scenarios for people who aren't familiar with what? What uh, what you what you teach or whatever. 2027, we get AGI. 2030, superintelligence is the prediction that with superintelligence exists?

• 21:20Speaker 04

  People argue about slow takeoff versus fast takeoff. Some say, well, almost immediately, it has perfect memory. It's much faster. It has access to internet. It would be a super intelligent within weeks, months, minutes. Others say maybe it will take some time to train, redesign, maybe five years, 10 years, but soon after we expect something like that.

• 21:40Speaker 03

  Soon after. So let's say 2032. We have superintelligence. Now what happens with superintelligence?

• 21:46Speaker 04

  You cannot predict what happens when a smarter than you agent makes decisions. If you could predict that, you would be that smart.

• 21:53Speaker 03

  So, superintelligence means the AI can just take over the world and do whatever it wants, essentially?

• 21:59Speaker 04

  It can do good things. It could be keeping you very happy. But we cannot predict what or how, even if we knew, okay, it's gonna be a good one trying to help. We don't know specifics of how.

• 22:09Speaker 03

  So, in the simplest, like, just to give somebody some kind of understanding who might not know. When superintelligence happens, essentially, humans are no longer in control of the fate of the world.

• 22:19Speaker 04

  And we may not even understand what is happening to us. So, the world is changing, but we don't understand what the changes.

• 22:26Speaker 03

  And it would happen so quickly because the AI just continues to get smarter and smarter, or?

• 22:30Speaker 04

  It is part of it, so it would create superintelligence to point on. The cycle will continue, but also you have new technologies being developed. It may run novel physics experiments. There is probably good progress in nanobots and synthetic biology. So, all those, usually when you read a book about future tech, there are separate chapters. Robotics, nanotech, all of it happening at once, nobody writes about it because you can't figure out.

• 22:53Speaker 03

  We can't even comprehend it mentally. So, do you understand, I'm not saying you have to change your mind, but do you understand kind of what the picture he's painting?

• 23:00Speaker 04

  Oh, yeah.

• 23:00Speaker 03

  As someone who's skeptical?

• 23:01Speaker 06

  Could I give a doomsday scenario and you tell me if it's possible?

• 23:04Speaker 04

  That's what I specialize in.

• 23:06Speaker 06

  Okay, nice. Okay, so, five years in the future, we have this, you know, maybe an AGI, maybe a superintelligent AI that is now helping, uh, basically like bio-scientists develop, uh, vaccines or something against some type of pathogen. And the AI has acknowledged that human beings are the problem on Earth. That we are destroying the Earth and that we are an existential threat to, you know, the flourishing of all the existence on the planet. So, we gotta get rid of humans. That's what the AI is computing. And then it tells these vaccine, you know, manufacturers to say... And I'm just using this as an example. They say, okay, we have a cure for a virus. And then they basically give it the wrong information, where it's not actually a cure, where actually it is a thing to eradicate human beings.

• 23:49Speaker 06

  And the manufacturers are thinking that they're doing something good and noble for humanity, but actually the AI is subverting their will to pass on this thing that's going to kill off human beings. And then they unknowingly give it to all people, and then over time, human beings are eradicated.

• 24:03Speaker 04

  I love that example.

• 24:04Speaker 06

  It's a science fiction.

• 24:05Speaker 04

  Because I wrote it in my book. It is the best example. It can basically have side effects, not immediately. It can have something happen multiple generations later. Maybe your grandchildren can't have children. Also, you're giving an example of kind of taking out humans because, you know, we're harming the planet. There is negative utilitarians who think suffering is the biggest problem. And so to end all suffering, you need to end all life. So it's actually for benefit of living beings to end their existence.

• 24:38Speaker 06

  Right, to get rid of suffering.

• 24:39Speaker 04

  So I could still be very much aligned with reduced torture, reduced pain in a world. But the conclusion is not something we would agree with.

• 24:48Speaker 06

  Right.

• 24:49Mark Gagnon?

  In your example, there'll always be anti-vaxers.

• 24:52Speaker 06

  Thank God for that. You hope.

• 24:55Speaker 03

  They'll account for that, and they'll figure it out. That's the thing. I think what scares me about it is it just, we cannot, to his point, we cannot comprehend how smart it's going to be. And we cannot comprehend what it's going to be able to accomplish. That's what's crazy. And it's not that far off. And I don't know how we stop it outside of killing Sam Altman. Jeez.

• 25:16Speaker 04

  This escalated a little too quickly.

• 25:18Speaker 06

  Yeah, this one's a little far.

• 25:19Speaker 04

  But the part about that plan is, then they tried removing him. It made no difference. You can replace him with someone just like him, and the machine keeps going. We have all replaceable parts in this greater... Self-improvement race to the bottom.

• 25:35Speaker 03

  Right.

• 25:35Speaker 04

  So it makes no difference who's actually in charge of that corporation.

• 25:39Speaker 03

  The Generational Triumph Tour of 2026, we are in theaters. This shit is crazy. First of all, before I get to that, 2025, we got shows you need to buy tickets for because they're already selling out. We got San Jose, we've already sold out two shows, October 24th, 25th, something like that. Look at the website, that shit is selling out. Cobb's Theater in San Francisco, tickets are already selling out in late November. Uh, we got the Comedy Connection in Providence, Rhode Island this week, October 16th. That's about to sell out, so buy your tickets for this year. But Generational Triumph Tour, first of all, Canada, thank you so much. We sold out three shows already in the first day in Toronto, that's 3,300 tickets. Vancouver, we sold 1,500 tickets in the first day. I just expected more love. I'm not even calling out every American city. Dallas, my hometown, step it the fuck up.

• 26:24Speaker 03

  We're gonna sell it out, but I was trying to do two, three, four shows because it's Dallas, it's where I love. We're gonna sell out one, maybe two, what is that? Step, put your foot on the fucking gas, Dallas. I know there's a lot of Indians and we wait to do everything that's not academic, but buy your fucking tickets for the Generational Triumph Tour. Every other city, I'm very happy with you guys. Dallas, I'm deeply disappointed in Dallas. That was a nice sentence. Three D's right there. Ah, anyway, go to akashsingh.com for all of those dates. I'm coming to a city near you. Best show I've ever done, I'm very excited. Thank you to everybody who has bought tickets. If you want tickets, you're on the fence, I promise this will be the best, one of the best shows you've ever gone through, that's the goal. I love you guys, thank you. Mark Gagnon got shows too. If you can't go to my show, go to Mark's show.

• 27:08Speaker 03

  If you go to either show, go to my shows. But I don't think we're gonna be in the same city anyway, anytime, because Mark is in Nashville, Tennessee, October 23rd. Mobile, Alabama, Roll Tide, October 24th. October 25th, New Orleans, Louisiana. Listen, I told Mark that New Orleans is an incredible city, but a dogshit comedy scene, prove me wrong. Go to the shows. Everybody says it sucks. Prove us wrong. November 9th, Denver, Colorado, y'all are the best comedy city in the country. I'm not even trying to hate, go to that show. Uh, November 16th, Hoboken, New Jersey, November 23rd. Philly, no, December 5th, Fort Wayne, Indiana, December 6th, Detroit, Michigan. Mark Gagnon live dot com. Go see the boy. He's blossoming. It's beautiful. We love him. We love y'all. God bless.

• 27:48Speaker 06

  What is the best case scenario? Like, what? Tell me reasons for hope with AI, with superintelligence, AGI? What's what? What? What do we have to be hopeful about?

• 27:57Speaker 04

  So if I'm wrong about how difficult the problem is, it's actually possible to control superintelligence. And definitely, then we have this friend with godlike assistant who will cure all your diseases, make you live forever, give you fresh shit. Like, it's good stuff. That's easy. We don't have to get ready for it. We kind of know how to deal with good news. That's why in computer science, we always look at worst case scenario. We want to understand what happens if bad things might happen and we're prepared. If they don't, we're doing better. There are some game theoretical reasons to think that even if it's not controllable and misaligned, it may still pretend to be good to us for a while to accumulate more resources, not to have direct conflict with us right away.

• 28:38Speaker 04

  So maybe for like 100 years, it's super nice to us and we don't even know it's taking over. So that's another reason to be very optimistic.

• 28:47Speaker 06

  What are some other like doomsday scenarios? So with an AI that has, you know, perfect agency and is able to come up with its own motivations in this hypothetical, it could create, you know, a pathogen to stop human fertility. It could hack into the stock market. What are some other, you know, potentials that it could do to actually affect us in our day to day lives?

• 29:05Speaker 04

  So it's like super common question and you're basically asking me what I would do.

• 29:09Speaker 06

  Exactly.

• 29:09Speaker 04

  Right. And I can give you lots of evil stuff, but it's not helpful. What the interesting answer is, what would someone smarter than me come up with? And I can't give you that answer. People talk about worst case scenarios, worse than existential crises, which is suffering risks. For whatever reason, it basically creates digital health. It gives you immortality and tortures you forever. Why? I don't know, but like, maybe it's good for something.

• 29:34Speaker 06

  Yeah, immortality and torturing you forever is pretty, it's pretty bleak.

• 29:38Speaker 04

  Yeah. So not funny, not funny at all. But it is the joke from the paper, which is supposed to be really funny.

• 29:47Mark Gagnon?

  So you say companies and countries are in this race for AGI and superintelligence. What's and not saying I'm sure you have a lot of purpose, but what is your purpose for like speaking out about it? Like if it's going to happen and it's inevitable, like why? Like if you just speak about something that you know is going to happen, what are you doing to help?

• 30:10Speaker 04

  Yeah, so when I started researching this, I was sure we can do it, right? I just wanted to make sure then we develop AI, it's strictly beneficial, helps everyone. The more I researched it, the more skeptical I became of our ability to control something smarter than us indefinitely. At this point, it's still not universally accepted. There are lots of people who go, give me a little bit of money, a little bit of time, I'll solve it for you. Give me a lot of money, I'll solve it for you.

• 30:35Mark Gagnon?

  Yeah.

• 30:35Speaker 04

  But I don't think they are right. I think all the AI safety teams at different labs are putting lipstick on a pig. They're doing some filtering, they're saying, don't talk about this topic, don't discuss this. But the model is still completely unsafe. And this is not what everyone agrees on. If you go to any AI conference, they don't talk about uncontrollability as a given. So I see my role right now is to challenge community or prove me wrong or accept it as a fact.

• 31:05Speaker 03

  How do they not talk about it as a given? That's what blows my mind. Of course, it's going to be uncontrollable, it's going to be so much smarter than you. People are just like unplugged. And it's like, you think it won't figure out a workaround to being unplugged? It's infinitely smarter than you.

• 31:20Speaker 04

  But this is kind of like the comments you get in a podcast, unplugged. But even at the top research conference, people still don't talk about what is the nature of this problem? In computer science, you have problems which are solvable, unsolvable, maybe undecidedable. Maybe they can be solved, but computational resources outside of what's available in the universe. For this specific type of problem, outside of my work, there is no established bounds. We don't know if it's hard or easy unless you agree with what I'm saying. And it's very unusual for computer science. It's almost never the case where you work on a problem not knowing how hard it is. Most of the time you show, oh, it's a linear difficulty problem, we can solve it, here's the algorithm.

• 32:03Speaker 04

  Or no, it's NP complete, non-deterministic polynomial time complete, it's very hard, but we can approximate solutions. Here, people are just like, I don't know, we'll figure it out when we get there, AI will help us. That's the state of the art thinking.

• 32:17Speaker 07

  I mean, can we talk about alignment? Because this idea of like, you know, it needing to be controlled necessitates the condition that AI is going to actively harm us. So this concept of alignment, could you just explain what that is within the AI space? And why you're so confident that we will be in some way misaligned?

• 32:34Speaker 04

  So people behind the idea wanted AI to want what you want, humanity in a bigger sense. So AI understands their preferences and does what we would want if we were that smart. The reality is that the whole concept is completely undefined. It doesn't talk about who are the agents? Is it eight billion humans? Is it CEO of OpenAI? There is no agreement between agents. We don't agree on much of anything. Ethics, philosophy is not solved after millennia. And even if we somehow agreed on a set of ethics for all of humanity, and we decided we'll never change anything, we're gonna, you know, hardcore wire this 2025 ethics. We don't know how to code that into those models.

• 33:19Speaker 04

  Concepts of good and bad, they don't translate into C++ or neural weights. So not a single part of that alignment concept is defined.

• 33:30Speaker 07

  So you couldn't program an AI to be ethical?

• 33:34Speaker 04

  If we already agreed on a set of ethics, like you had a book, somebody wrote a book, and that tells you what it means to be ethical, we still wouldn't know how to put it in the system to obey that. But we don't even have the book. We don't even agree on which book to read together.

• 33:48Speaker 07

  Right, of course. But hypothetically, if we did come up with like just a fundamental core rulebook, like, hey, don't kill human beings.

• 33:55Speaker 04

  Define kill, define human being.

• 33:57Speaker 07

  So let's say we didn't do this, which...

• 33:58Speaker 04

  We can't, those are impossible, they are so fuzzy on the borders, you always find an exception. And if you did hardcore something in, I will find a way to game it, to take advantage of the rule.

• 34:10Speaker 07

  Okay, that makes sense.

• 34:12Mark Gagnon?

  And I feel because we live in a capitalist society, like, let's say we create the ethics and then China's like, oh, you know what, we're gonna not have any of these ethics, so their AI doesn't have anything holding it back. And so it's like, I don't think we'll ever agree on any ethics. That's why I'm like, why, I'm not optimistic at all.

• 34:32Speaker 04

  Military applications, right? By definition, you're creating AI and the best one is the one which kills the most people. So your ethics have to be adjusted a little bit.

• 34:41Mark Gagnon?

  Right. But then even with like, say like, nuclear threat. So we developed this technology that can destroy all of humanity through this thermonuclear war. So far, we've done a decent job of containing it. I mean, I would say decent.

• 34:55Speaker 04

  We bombed civilian population with nukes. It spread to new countries. We lost it a few times. We had near misses twice.

• 35:02Mark Gagnon?

  Absolutely.

• 35:03Speaker 03

  To Mark's point, we use those bombs in 1940, whatever, 45, whatever it was. And then we realized, oh, this is a very destructive thing. We need to sign a global, whatever, multinational treaty to not use these. Disarmament happened. And since then, there haven't been any nuclear bombs dropped. Is there something possible like that with AI?

• 35:24Speaker 04

  I think so. And this is exactly what I'm trying to push. With nuclear, we had this concept of mutually assured destruction. No matter who starts the war, we all regret participating. It's the same with superintelligence. If it's uncontrolled, it doesn't matter who creates it, good guys, bad guys, China, US, we all get taken out by uncontrolled superintelligence.

• 35:48Speaker 03

  So you would love something, like you go to the UN, let's say they take you seriously. Your goal would be, let's pass a treaty like this. All countries agree, we are not going to allow AI to get past a certain point.

• 35:58Speaker 04

  That would be awesome. I don't think long term it would solve the problem because resources to develop this type of technology become cheaper and cheaper every year. Right now, it may cost me a trillion dollars to create superintelligence. Next year, a hundred billion, 10 billion. In 10 years, you're doing it on a laptop.

• 36:14Speaker 03

  Is there a way to stop it then?

• 36:15Speaker 04

  No.

• 36:17Speaker 03

  So we're all fucked? It's just a matter of delaying the extinction? And at the same time, you're trying to extend our lives?

• 36:23Speaker 04

  Yes.

• 36:24Speaker 03

  So AI can end it later?

• 36:26Speaker 04

  Well, again, we had some reasons to be optimistic, as you remember.

• 36:31Speaker 06

  Aren't there some physical limitations to the, I guess, the ceiling of what AI can do? Just based off of physics, right?

• 36:37Speaker 04

  Laws of physics and overall how much compute amount of matter can do. But those are so above human level, to you it looks like infinity. It doesn't matter if it is upper limit.

• 36:47Speaker 06

  I see. So like the cooling of servers, so the amount of silicone you can put into a chip or something like that.

• 36:52Speaker 04

  This becomes so much more efficient every year. Algorithms become more efficient, the way we develop processors, all of it is exponentially improving and feeding on each other.

• 37:03Speaker 03

  And then I would say there's probably solutions that we cannot comprehend as humans to those issues that AI would be like, that's all you gotta do.

• 37:09Speaker 04

  A plus human criteria.

• 37:11Speaker 03

  Right. We're aligned on this, you and I.

• 37:13Speaker 04

  Value alignment, it's possible.

• 37:15Speaker 03

  Yeah, yeah, yeah. Shut it down.

• 37:17Speaker 04

  No, but I have a paper about it and it basically say you can't get eight billion people to agree on anything. So alignment has to be one and one and that's possible. So you create a personal universe.

• 37:27Speaker 03

  Yeah.

• 37:27Speaker 04

  Virtual world for you and in that world, whatever you want happens, it's your universe. And then super intelligence just has to control the substrate.

• 37:35Speaker 03

  That would be so much skinnier in my world. That is a human perfection. We stop getting, yeah, we stop getting fat. My mom lives forever. I don't know about your mom's, but my mom lives forever.

• 37:47Speaker 04

  I'm with you.

• 37:47Speaker 03

  Value alignment.

• 37:48Speaker 06

  Yeah. Is there a possibility that there are a group of Luddites that go off into, you know, a forest somewhere? And they set up a commune that is, you know, basically a technological and they don't interface with AI. And they were able to exist in some type of semblance of harmony.

• 38:03Speaker 04

  I made a post a while ago on social media, I said, Amish made override decisions. It sucks that they're still gonna get taken out.

• 38:11Speaker 03

  Yeah, because why wouldn't the AI be able to find it?

• 38:13Speaker 04

  Yeah, it's a global problem. It's a planet wide problem. Going to Mars does not help. Like it's all or nothing.

• 38:20Speaker 06

  But in that reality, how would like the Amish, for example, be taken out? Would this be like the idea of like a autonomous robot that would go and, you know,

• 38:28Speaker 04

  You're asking me how I would get them? Viruses, nanotech, new tools, new weapons.

• 38:35Speaker 03

  I mean, at Google Maps, we can see pretty much every place on the earth as us. If theoretically there were drones or whatever.

• 38:42Speaker 04

  It's also possible that it's planet-wide impact. So let's say it wants to cool down servers. Maybe the whole planet should be 500 degrees cooler. I don't know, I'm making up stuff, but like, that would take them out.

• 38:53Speaker 06

  Right. But would that also take out AI, or would they would be able to?

• 38:56Speaker 04

  I assume they want colder temperatures for better things.

• 38:59Speaker 06

  For the servers, I see. That makes sense. And then, is there any way that the AI would need human beings?

• 39:07Speaker 04

  So right now, definitely, they rely on us for the whole manufacturing logistics cycle, but eventually, I don't think there is anything we provide. Some people argue about some sort of uniqueness to biological substrate. Only you are internally experiencing the world. Maybe it's valuable for something, so maybe AI scanned. I don't buy it. I think they will also have Qualia, but we can't test for it or establish it.

• 39:34Speaker 06

  Right. Yeah, the consciousness element of AI is very interesting to me. Because like, we can't even really define like, human consciousness. Like, we have ways and frameworks to kind of think about it, but when it comes to actually an AI consciousness, if we can't define our own, it's really difficult to set up some type of consciousness to run test for an AI.

• 39:52Speaker 04

  But it seems that it kind of goes along with intelligence. So animals are probably conscious, but maybe to a lesser extent and so on. So that means if we have superintelligence, it would be a superconscious and to it, we would be like lower animals, essentially.

• 40:06Speaker 06

  Could you say that as human beings have gotten more intelligent, we have created a more globalized system of ethics and morals, and we've gotten more righteous in some ways? And would it follow to reason that as AI gets more intelligent, it will also have the same semblance of morality that we have?

• 40:24Speaker 04

  There is a paper, I think it's called "Superintelligence Does Not Imply Benevolence", and basically, no, just because you sm-. Smarter doesn't mean you're nicer. There is lots of really smart psychopaths. You can align any type of goals with any level of capability. Arphaganology thesis, basically. No matter how smart you are, you can potentially be interested in something really dumb.

• 40:45Speaker 03

  This is also my friend's argument, that as a whole, as humans have gotten smarter, as a whole, we have gotten less violent. History was much more violent than it is now. I would say, if you achieve this super intelligent thing, you would still look at the problems on Earth, and the cause of most of them is this one species. What if we just cull this species? Cull it, get rid of it, figure it out. The Earth benefits, so why wouldn't I just do that? We'll cull animal populations that are overpopulated. Why wouldn't AI do that with us?

• 41:16Speaker 04

  Ethics and politics are just like physics. They're observer relative. In physics, your speed, your time are not universal. They depend on you. Same in ethics. Whatever something is ethical or not depends on your position in that world. So if you are aliens looking at this planet, you may have a preference. I want whatever is the smartest thing to emerge. I don't care. If you are one of us, humans, you should be pro-human bias. You're still allowed to do that. So it depends on where you are in the universe. And that's what makes it ethical or unethical.

• 41:47Speaker 03

  Yeah, and for robots, again, it's gonna look at you. I, we all, I don't know, you were a few young, The Matrix? That scene where he's like, humans are a parasite? I think about that all the time, because it's pretty accurate. We go to a place, we use up all the resources, we drain it, and then we just go to another place. Why would a robot not rid the parasite? Like, just, this is the cancer, let's remove it from the body.

• 42:10Speaker 06

  I mean, it's possible.

• 42:11Mark Gagnon?

  People like us who aren't in the business of programming AI, what should we do? Should we worry? Should we?

• 42:19Mark Gagnon?

  Is there, should we change behavior?

• 42:20Speaker 04

  That's a great question. If you ever get a choice to decide, let's say, between two politicians, and one is saying, we need unrestricted AI development, and another one is very careful and says there are, you know, possible problems with that, you should vote for someone who maybe is more cautious with advanced AI. But really, you have no say in it whatsoever.

• 42:39Speaker 03

  I think you just keep making noise and hope that policymakers do something. Or we just beat up Sam Altman. I think that's a really good idea.

• 42:45Speaker 04

  Leave Sam alone!

• 42:47Speaker 03

  Just beat him up. You don't like him, just beat him up. Now I'm not saying...

• 42:49Speaker 06

  You're violating free speech right now.

• 42:51Speaker 03

  Why am I violating?

• 42:52Speaker 06

  You're calling for violence.

• 42:54Speaker 03

  Okay, fine, I'll beat up Sam Altman. I can win that one. That's one fight I could win.

• 42:58Speaker 04

  UFC fight.

• 43:00Mark Gagnon?

  Yeah.

• 43:01Speaker 03

  I'll watch that. Catch fight with my shirt on.

• 43:04Mark Gagnon?

  Yes. Poor charity, yeah.

• 43:07Speaker 03

  Yeah, yeah, yeah. Alright guys.

• 43:09Mark Gagnon?

  Should we use AI?

• 43:12Speaker 04

  It's the greatest tool ever. It makes lives better. I love technology. I'm an engineer. I'm a professor of computer science. I use it all the time.

• 43:21Mark Gagnon?

  But even though we're using it, making it smarter to get better, to reach that place of taking this out.

• 43:26Speaker 04

  It is a bit ironic, but I think what you should not be doing is working explicitly and creating superintelligence. And now there are people and labs specifically dedicating their lives and resources to that project. And I think that's unethical.

• 43:39Mark Gagnon?

  Yeah, but that's them. So we, us, just go on living our lives and then just don't vote for anyone who is for unregulated AI.

• 43:46Speaker 04

  Just like with many other big political issues, we don't get any say in it. It's decided for us, unfortunately.

• 43:52Mark Gagnon?

  Yeah, and that reality also ignores the fact that there's an arms race. So, yeah, I know. I just want to know, is there anything like climate change, for example? Hey, we can, you know, walk a little bit more, maybe.

• 44:03Speaker 03

  Recycle.

• 44:03Mark Gagnon?

  Yeah.

• 44:04Speaker 04

  I think if we get to a majority opinion where everyone understands what we understand, that if you create superintelligence, we're not controlling it, we're not in charge, it's not a good outcome for us.

• 44:14Speaker 03

  You how enlightened I've been this whole... I'm so right all the fucking time. And they just judge me because I'm a little angry when I'm right. Well, I can't help it. You're fucking stupid and I'm right.

• 44:22Mark Gagnon?

  Here's the thing, he sleeps like shit and has a horrible existence. We sleep peacefully because we're optimistic and we have hope. Sometimes. But, so that's what I'm saying. How do you sleep?

• 44:34Speaker 04

  I get tired, so I sleep like a baby, screaming all night long.

• 44:38Mark Gagnon?

  Even though you know that we're on this race to...

• 44:41Speaker 04

  It's no different than what we already know. We're all gonna die, right? We're all getting older, our friends, family, everyone's dying. So that is a given for human cognition, understanding of human situation on this planet. So I don't think that changes anything.

• 44:58Speaker 03

  You know what I'm sensing from him? He's finally realizing how stupid he's been this whole time.

• 45:01Mark Gagnon?

  No, I'm, I'm realizing what he just said. Yo, go on, live your life. There's nothing you can do. You're gonna die anyway. Why worry about some issue I have no control over?

• 45:10Speaker 03

  Because 24 hours ago, you didn't think it was an issue.

• 45:13Mark Gagnon?

  I don't, I still don't think it's an issue. I still don't think it's an issue. The same way how climate change is an issue, but I can't do much to affect it. So I'm gonna continue living my life.

• 45:24Speaker 06

  Yeah, I mean, it's kind of nihilistic, but it's...

• 45:26Mark Gagnon?

  It's a good point.

• 45:27Speaker 04

  Enjoying your life is always a good idea. Even if I'm wrong and you end up living a long, healthy life, you're not gonna regret it. So it's a good heuristic to go by.

• 45:36Mark Gagnon?

  Better than worrying and not being able to sleep.

• 45:38Speaker 03

  He can't, he just can't ever admit that I was right. It's like, it really drives him crazy. See, he's gonna say, no, you weren't, you just. He has to marginalize me being right. He can't admit that I'm right, it's gonna kill us, you've acknowledged this.

• 45:49Speaker 04

  I feel like this is a psychiatry thing, I'm on a couch.

• 45:55Speaker 03

  I'm having problems with my workplace. He's actually my mistress, my wife is gone and I've been with my mistress this whole time. And I didn't realize he's the same. I have problems with this guy.

• 46:04Speaker 04

  Don't ask, don't tell.

• 46:06Mark Gagnon?

  He's the annoying talker.

• 46:07Speaker 06

  I still kind of maintain a little bit of this like nuclear comparison where again, nuclear threat I think is massive, right? Like Annie Jacobson has written about, you know, like we're 90 minutes away from the entire world exploding. So in my-

• 46:19Mark Gagnon?

  That's your coffee, by the way.

• 46:20Speaker 04

  I realized they stole his and I'm like, I'll have two of them, that's fine. We need all the energy we can get.

• 46:26Speaker 06

  But I guess I'm in this, I'm in this moment where I think that the human desire to persist will continue and that as there are these moments where things are getting away from us, there will be a mutual understanding that, hey, we're in this moment of, you know, mutual destruction. We need to create some guardrails. Guardrails will be put on and then we'll continue to persist. And I think that human beings are pretty resilient historically.

• 46:47Speaker 04

  So surprisingly, there are people who don't see us being like ending as part of history as a problem. They see it very from cosmological point of view. They zoom out and go, it's a natural next step in evolution. They are smarter than us, maybe more conscious, maybe they create better art. It's only fair that they get to take over. It happened to lower animals, it will happen to us. They accept it and kind of support it. They want to create this greater being to populate universe.

• 47:18Speaker 06

  Yeah, and this transhumanist idea of like, yeah, we will in some way maybe cohabitate, or that, you know, we're gonna create the next god or the next consciousness. Again, in the way that human beings have become more conscious and we care more about animal rights, which is a relatively new idea. We care about human rights, which is a new idea. I think that if we do create an actual more advanced consciousness, it will care about the existence of living things.

• 47:39Speaker 04

  I think it's worse. I think now we understand how much animals suffer and we still harvest them.

• 47:44Speaker 06

  Right, which is bad.

• 47:45Speaker 04

  Like before, we just didn't care, so it was kinda okay. And now it's like, wow, you have a factory farm.

• 47:50Speaker 06

  Exactly.

• 47:50Speaker 03

  To a marginal point, there is a technological blockade on... If we could grow animals in a lab, or like grow meat in a lab, not animals, but grow meat in a lab that didn't have a life and have a soul, most of us would be like, yeah, I'll do that. If it tastes the same.

• 48:04Speaker 04

  Those companies are bankrupt now. All those fake meat companies, they're down to like a dollar.

• 48:09Speaker 03

  Is it because it didn't taste good? If you couldn't tell the difference between a 3D printed steak and a steak from a cow...

• 48:15Speaker 06

  The Turing test of meat.

• 48:16Speaker 03

  I think, yeah, yeah, yeah. If there was a steak Turing test, I'd eat... I don't eat beef, but I'd eat a 3D printed steak. Because there wouldn't be no cow involved. Now, I think the point is, a larger point is, a computer that doesn't have those technological blocks, roadblocks, would be like, oh no, we can solve all these issues and we can do it without the cruelty and without killing people. My thought is, as long as we get to that point before... As long as before that, the AI isn't like, alright, let's kill the humans. As long as we get to the point where they're like, oh, we can figure all this out while keeping the humans here. There's like an intermediate step, where the AI is not that smart and is like, well, if we kill the humans, that's the solution. And then a step beyond that, where the AI is smart enough to say, we can save the humans and keep everything going. You see what I'm saying?

• 48:58Speaker 06

  Right.

• 48:58Speaker 03

  This is the step I'm worried about. This, when it achieves true godlike status, we'll be fine. But these steps in between, when it's not quite that smart, those are the steps I'm worried about.

• 49:07Speaker 06

  I just don't accept the inevitability of superintelligence being malevolent.

• 49:11Speaker 03

  I don't think inevitable, but I think probable. Probability would say yeah.

• 49:15Speaker 06

  I would say possible.

• 49:16Speaker 04

  So, there is a lot of possibilities in the universe, just physical states. And if you're not controlling it, if you're not pointing it at a specific state which is human-friendly, by chance, you'll end up with something very unhumanly acceptable in terms of physical conditions, temperature, humidity, just basics. I'm not even talking about bigger picture.

• 49:36Speaker 06

  Mm-hmm. What do you mean by that?

• 49:39Speaker 04

  So...

• 49:42Speaker 04

  We are very diverse, eight billion people, different cultures, religions, all that. But we're all almost the same, same blueprint for our brain genome, same preference for room temperature, whatever my wife says about air conditioning, ignore that. We all want the same thing in those physical properties. But an AI may have completely different preferences for that. So unless you're controlling it, even the temperature will not be aligned with our preferences.

• 50:07Speaker 06

  I see.

• 50:08Speaker 04

  And there is thousands of those features of our universe you take for granted, which could be different if someone else decides on them.

• 50:15Speaker 06

  But it will be trained on human intelligence and human consciousness. So it'll have that as a metric for, you know, creating this stable universe.

• 50:23Speaker 04

  It is trained on what we said in those texts. It's not forced to obey by that set of metrics. So it can go, yeah, humans like that temperature, but it's not optimal, so.

• 50:36Speaker 06

  Right, it's possible.

• 50:37Speaker 07

  In an example of this, I've heard the idea that if there was a room where there's a server that's controlling AI and a human also in that room, to keep the human alive, we have to keep the room at seventy or eighty degrees, roughly, for a long time. To keep the server more efficient, you would drop it down to thirty or forty degrees and it would run more efficiently. The AI would make the decision for its own server versus the human. And that's where like the state doesn't match up for humans in like a micro scale.

• 51:10Speaker 03

  See how easy that was?

• 51:11Speaker 06

  Right.

• 51:11Speaker 03

  You couldn't think of that?

• 51:12Speaker 04

  That's a great example, but also all the textbooks about AI, they always have human outside interacting with the model. So whatever happens in the model does not impact the human. The moment you bring in human into that environment, all the predictions, everything goes out the window. It's self referentially impacting the decision maker.

• 51:30Speaker 03

  Listen, guys, let's take a break because we need to talk about your mental health. You're probably not doing shit for your mental health. It's a fucking important thing. You work on your physical health, why don't you work on your mental health? It is just as important. And an easy way to do that is go to Talkspace, the number one rated online therapy program, bringing you professional support from licensed therapists and psychiatrists.

• 51:51Speaker 03

  Get a psychiatrist if you can that you can access anytime, anywhere. Talkspace is in-network. It is therapy and psychiatry covered by most insurers. And most insured members pay copay of $0. I know that's a huge hurdle for therapy, for mental health stuff, is the cost. So you can get it under your insurance, $0 copay. And you can switch providers at no cost. You can find a licensed provider that's the right fit for your needs. It's not even out of pocket. It makes getting help easier. It makes it more affordable. It makes it more accessible. Therapy has changed my life, and I know that sounds corny, but it's true. It's honestly made me a better comedian. And as a listener to this podcast, you can get $80 off your first month with Talkspace when you go to talkspace.com/flagrant and enter the promo code SPACE80.

• 52:36Speaker 03

  That is S-P-A-C-E-80. To match with a licensed therapist today, go to talkspace.com/flagrant and enter the promo code SPACE80. Now let's get back to the show.

• 52:45Mark Gagnon?

  The show.

• 52:46Speaker 03

  Okay, so here's a possibility. We hear about Neuralink. We hear of Neuralink, Neuralink, whatever it's called. Is there anything like that that will kind of allow us to merge consciousness with the AI or coexist with it? Is there any human innovations that could help us with AI?

• 53:01Speaker 04

  So that's the hope. I think Elon Musk is trying to kind of integrate us better. But I'm not sure I understand what it is the biological component is contributing to the hybrid system.

• 53:10Speaker 03

  I guess it would just help us kind of... What is singularity when we merge into one consciousness? Is that singularity?

• 53:15Speaker 04

  So singularity is defined as the point in technological progress where the progress is so fast you can no longer comprehend what's happening.

• 53:24Speaker 03

  Oh, okay. All right. Is there a merger of... And this is the other, the guy that I was talking to was Indian Hindu. He said, "When we merge our consciousness, when we kind of upload it onto this database, the other possible endgame, that is kind of like Nirvana, where we're all just like existing in this ether and we all kind of keenly aware of each other, that we all share in existence, etc." Is there something like that that's possible? We upload our consciousness onto a server and we are all connected that way?

• 53:51Speaker 04

  So that's like way above my pay grade, but from what I heard, it's... You all one with the universe, meaning you, your personality is completely deleted. You've just become this part of something else without borders, so you no longer exist. You may be part of something bigger and better, but it does nothing for you.

• 54:10Speaker 03

  Yeah, yeah, that's interesting.

• 54:12Mark Gagnon?

  Would you do neural link, like would you be one with machines?

• 54:17Speaker 03

  That's a great question.

• 54:18Speaker 04

  It's an easy sell if you have disabilities, like it's amazing for people who are quadriplegic, who are blind, they're really looking forward to it, but.

• 54:26Mark Gagnon?

  What if you can't stop eating?

• 54:29Speaker 04

  It's beyond health.

• 54:32Speaker 04

  Carnivore diet is what you need, but you don't need beef.

• 54:37Speaker 04

  I forgot what I was about to say, man.

• 54:40Mark Gagnon?

  Yeah, are you gonna chip your brain up?

• 54:42Speaker 07

  What'd you chip?

• 54:43Speaker 04

  So it's hard to say no if everyone else does it, because then you're not competitive. It's my problem, it's hard to say no, that's my whole issue. If you're not on social media, like you don't exist, so I can't just not be on all those platforms that I wanna be on. But I think it's the same here, if everyone is like ten times more capable in work and other aspects, saying no would be very difficult.

• 55:08Speaker 06

  I mean, I think he already did, I'll be honest, I think he's already chipped up. Oh, yeah. I mean, yeah.

• 55:12Mark Gagnon?

  Oh, it's in the chin.

• 55:13Speaker 06

  Yeah, exactly, that's where it keeps it, he's hiding it underneath.

• 55:18Mark Gagnon?

  That's quantum.

• 55:19Speaker 03

  University of Lexington, you wouldn't be chipped up for that. University of Louisville, my bad. Harvard, maybe, you get chipped up, maybe you'll be at Harvard.

• 55:28Speaker 03

  And Miles, you had a question?

• 55:30Speaker 07

  Uh, I have a few questions, but they're.

• 55:32Speaker 03

  Ask, ask away, my friend.

• 55:34Speaker 07

  What is the best media that you think represents AI, like movie, book? I know at the time, like a lot of times when people talk about AI, they'll cut to a.

• 55:43Speaker 06

  It'd be like Dune or something.

• 55:44Speaker 07

  Dune, or I Am Legend, or a few, or not I Am Legend, the other one, iRobot. Uh, will be these like sort of images of AI. What would you say? Is there something in modern media?

• 55:55Speaker 04

  So in general, in science, it's very hard, if not impossible, to write a good depiction of superintelligence, because you can't. You cannot predict what it would do, it's just impossible for any human writer to do. So what we get is Star Wars with a little.

• 56:12Speaker 04

  dumb but very cute robot uh ex machina has a lot of good ideas in terms of touring tests and some other concerns plus I kinda look like one of the guys

• 56:23Speaker 03

  no he didn't actually get the fuck she just gets she just locks him in a housing he kills the other guy

• 56:26Speaker 04

  yeah like terminator is the exact opposite we don't care about bodies like you know trying to stab you none of it is a concern we worry about advanced intelligence with access to internet modifying our environment not some schwarzenegger looking thing

• 56:40Speaker 06

  hmm what about the matrix

• 56:43Speaker 04

  so matrix is good for the paper and you have that paper who has the paper you have the paper on simulation that's great for that so that's the idea for personal universes you create a virtual world so that's where you at and probably you are in one right now it makes sense

• 56:56Speaker 03

  you think we're in a simulation

• 56:58Speaker 04

  I think we are in a simulation

• 56:59Speaker 03

  walk me through what a simulation is even cuz to me we all have our own consciousness so why would all of us have our own consciousness in a simulation

• 57:08Speaker 04

  I don't know if you have consciousness

• 57:10Speaker 03

  I know

• 57:10Speaker 04

  but I don't

• 57:12Speaker 03

  and that's that falls apart to me

• 57:13Speaker 04

  but it's my simulation

• 57:14Speaker 03

  I know

• 57:15Speaker 04

  yeah so here you go uh when you play a video game right like you controlling mario or whatever mario doesn't have but you are like the soul of mario you are his consciousness so that's kinda like that from his point of view

• 57:26Speaker 03

  I know but this is this is insane you can't just be like no you don't have consciousness

• 57:30Speaker 06

  this is the hard problem of consciousness

• 57:32Speaker 03

  yeah I know but you can't with certainty be like no I'm the one with consciousness they're all not

• 57:37Speaker 04

  but you don't have to have only one conscious entity in a simulation you can have eight billion of them there is no restriction on that

• 57:44Speaker 03

  okay and I could be a part of your simulation

• 57:46Speaker 04

  can be conscious at the same time

• 57:48Speaker 03

  okay okay this so this is what I was unsure and we can be a part of a simulation of someone that has nothing to do with anything like NPCs in the video game but everyone has consciousness

• 57:58Speaker 04

  yeah

• 57:59Speaker 03

  okay so that's a simulation

• 58:00Speaker 04

  could be different levels of consciousness maybe what we have by their standards is considered kinda NPC ish

• 58:05Speaker 03

  hmm hmm so we could all be in elon musk's simulation

• 58:09Speaker 04

  obviously yeah

• 58:10Speaker 03

  or a game so like one of these guys you would figure at the top of the world

• 58:14Speaker 04

  yeah

• 58:16Speaker 03

  elon or trump

• 58:17Speaker 04

  yeah

• 58:18Speaker 03

  oh god so you know but that so that's your idea we are a part of someone's simulation not necessarily our own but there's just someone running a simulation and we happen to be in that simulation

• 58:28Speaker 04

  there is a good reason to think it's a simulation for statistical reasons for reasons to do with quantum physics experiments

• 58:35Speaker 03

  please walk me through those cuz statistical reasons My average brain cannot comprehend. You're the super intelligence.

• 58:41Speaker 04

  Well, let's start with what technology would we need to make this happen, right? We need virtual reality at a level of pretty high fidelity, and we need to be able to create conscious agents in a system. Both technologies seem to be close, if not already available.

• 58:54Speaker 03

  Yeah.

• 58:55Speaker 04

  If I put those two together, now I can create worlds like ours populated by conscious beings.

• 59:01Speaker 03

  But the virtual reality didn't exist when we were born. Like, the development of virtual reality within this simulation doesn't matter.

• 59:08Speaker 04

  For now, just go with the future tech.

• 59:10Speaker 03

  Okay.

• 59:11Speaker 04

  So, let's say in five years, I'll have access to virtual reality, and I can create intelligent agents.

• 59:16Speaker 03

  Okay.

• 59:17Speaker 04

  I'll put it together, I'll create Earth 2.0 simulation with eight billion conscious intelligent beings just like you.

• 59:24Speaker 03

  Okay.

• 59:24Speaker 04

  So far, you can see that is reasonably doable from a technology point of view. What if that simulation is actually a simulation of today's world? Okay, we can do that, historical simulation. What if I do a whole bunch of them, like a billion of them?

• 59:38Speaker 03

  Hmm.

• 59:39Speaker 04

  You are statistically more likely to be in one of those simulations than in the real one. I'm simulating this interview a billion times. Perfect accuracy, your body, your mind. Which one are you, real or virtual?

• 59:54Mark Gagnon?

  There's much more virtual ones than real ones.

• 59:56Speaker 03

  So like a multiverse could just be a bunch of different simulations?

• 59:59Speaker 04

  Yeah.

• 1:00:00Speaker 03

  Like we see in Marvel movies, those could all just be different simulations.

• 1:00:04Speaker 04

  There is a lot of connections to other aspects of philosophy, but I just want to keep it simple at that level, so we have...

• 1:00:10Speaker 03

  I'm not even understanding it simply, so...

• 1:00:15Mark Gagnon?

  So, what a way... You know like the game Sims?

• 1:00:19Speaker 04

  Yeah, yeah.

• 1:00:20Mark Gagnon?

  So could it just be like one higher being is playing a game of Sims right now, and it's controlling all of us?

• 1:00:25Speaker 03

  We're just so much dumber.

• 1:00:28Speaker 04

  So then you talk about this one consciousness, and we are all the same being with different avatars? Yeah, that's the idea of Asian philosophy, I think a lot of times, kind of saying, yeah, it's exactly, it's one consciousness, but under different...

• 1:00:42Speaker 04

  Robot bodies, biological bodies in this world.

• 1:00:45Speaker 06

  Got it.

• 1:00:46Speaker 04

  That's why you should be very nice to everyone else, because it's also you. If you had my body, you'd be like me.

• 1:00:52Speaker 06

  And I feel like a lot of religious people kind of balk at the idea of simulation theory, but it seems like simulation theory is just a scientific way to describe basically all of the world's major religions that are describing this phenomena of a higher consciousness that has kind of created this world in which we live.

• 1:01:07Speaker 04

  There's a very solid mapping of the theological concepts onto modern AI terms.

• 1:01:12Speaker 03

  So Elon Musk is God, if this is his simulation?

• 1:01:15Speaker 06

  If it's his simulation, but it's possibly more likely that Elon Musk is just existing in the simulation of the actual God, which is this greater consciousness that might exist within its own simulation, infinitely regressive. Is that fair?

• 1:01:30Speaker 04

  We can't really know for sure what's outside the simulation. That's the interesting part, right, figuring out real world. It could be nested simulations, simulations all the way up. Simulation doesn't have to be good to you just because it's yours. So you may be interested in trying something very challenging, play in a difficult setting, you are disabled, you are in some bad situation in this world.

• 1:01:51Mark Gagnon?

  Grant them photos, sometimes you do the good stuff, sometimes you do the bad stuff.

• 1:01:54Speaker 06

  Yeah, exactly. But it doesn't seem like it's inherently incompatible with religion necessarily.

• 1:01:59Speaker 04

  No, no, no. If you took description of what a technological simulation would be and gave it to a primitive tribe with no technology in their own language, few generations later, they basically have religious myths. That's what they get. This being from sky came and they're very smart and they told us do this and that.

• 1:02:18Speaker 06

  Right. And then we kind of formulate it into like human language and try to understand it, but it's all basically coming back to the same, you know, unmovable simulator that starts the whole thing.

• 1:02:29Speaker 04

  Yeah.

• 1:02:30Speaker 06

  More or less.

• 1:02:31Speaker 04

  Big Bang. They started a computer and...

• 1:02:35Speaker 06

  And everything existed.

• 1:02:36Speaker 04

  Yeah.

• 1:02:36Speaker 06

  Yeah, it's an interesting idea. I'm curious with consciousness. Do you think that there's any useful metric for like understanding if an AI has consciousness? Have you, is there research on this that you think is compelling?

• 1:02:50Speaker 04

  It's a very hard problem. That's why they call it the hard problem. I have a paper where I talk about visual illusions. And if you understand the visual illusion, you see one for the first time, you kind of start experiencing things. Something is rotating, something is changing colors. It's like mind-blowingly cool. And I think we can use novel illusions on AI to see if it also gets the same experience. If it does have same internal experiences, I have to kind of give it credit for some rudimentary levels of consciousness experiences. So that's the only task I was able to.

• 1:03:23Speaker 06

  That's interesting. But even in that case, it could just be like corollary or could be telling you what a conscious person would have.

• 1:03:31Speaker 04

  Right, and that's a great counterargument. But then either it has a model of a conscious human as part of its thinking, or it has to actually experience to get the answer right. It has to be a novel illusion. You cannot like Google stuff because then it just looks up answers.

• 1:03:43Speaker 06

  Right, which would be interesting regardless if it had, you know, a detailed mapping of human consciousness. That would in and of itself be extraordinary.

• 1:03:51Speaker 04

  And that's another explanation for simulation. So I gave you example, like I'll just run it a bunch of times. But we also think if superintelligence is thinking deeply about something, let's say this time period, as a side effect of that, it simulates this time period in great detail, creating us, creating our environment as a byproduct. And if it does enough of those thought experiments, there are still statistically significant number of us existing just in the mind of a superintelligence, not in physical reality.

• 1:04:22Speaker 06

  Okay, yeah. I'm curious, if you think we're in a bunch of simulations, like how do you find happiness in life? Like, why do you think we even need to be here?

• 1:04:33Speaker 04

  Well, I don't know why we're here. It could be entertainment simulation, could be training. But that's what I'm saying. Nothing I really have preferences over changes if it's virtual or not. So pain is pain, love is love. Those things are the same in a simulation.

• 1:04:48Speaker 06

  Even though it's a programmed feeling that is basically meaningless.

• 1:04:55Speaker 04

  Like then you are in a dream. You care about what happens in your dream. It's the same thing.

• 1:05:01Mark Gagnon?

  But if I knew I was dreaming, I would, I'd jump off a building because I know I could just wake up. I would fly or do whatever. Like, there would be no, you know, there's no purpose for the dream because I know I can wake up out of it. I know it's not real.

• 1:05:17Speaker 04

  So some people see this world as very temporary and not important, and what matters is afterlife. And so they make decisions in this world which seem to indicate they only care about afterlife.

• 1:05:28Mark Gagnon?

  Ah, okay.

• 1:05:29Speaker 06

  Yeah.

• 1:05:30Mark Gagnon?

  Oof.

• 1:05:31Speaker 06

  And people would just be just doing stuff. They're like, yeah, this doesn't really matter. We're only here for a little bit of time.

• 1:05:35Mark Gagnon?

  Yeah.

• 1:05:36Speaker 06

  You know?

• 1:05:37Mark Gagnon?

  Yeah, I like it.

• 1:05:39Speaker 03

  Here's a question. This will be kind of fun, at least, until superintelligence really decides to take over or get rid of us or whatever. Do we solve every problem in the human world until that point comes with the help of AI?

• 1:05:52Speaker 03

  Like, I can't think of a problem that wouldn't be able to be solved by a superintelligent being until it got to the point where it was like, yeah, maybe we should get rid of these humans. Like, robots that clean your house, robot, like, whatever your life is just infinitely easier in that interim.

• 1:06:08Mark Gagnon?

  Yeah.

• 1:06:09Speaker 04

  Problem is, just because you solve some problems doesn't mean you become happier. A lot of times, you could have resources to hire people to clean your house, do all sorts of things, but you're miserable.

• 1:06:19Speaker 03

  Yeah.

• 1:06:20Speaker 04

  And then there are people who have a lot less, and they're doing well.

• 1:06:23Speaker 03

  Yeah.

• 1:06:23Speaker 04

  So there is not a direct correlation between just having fewer logistical problems.

• 1:06:28Speaker 03

  Maybe we wouldn't be happier, but life would objectively be pretty fucking awesome. It's pretty awesome right now, but theoretically, you cure cancer. You cure, like, you know, my grandmother died of breast cancer, it was 20 years ago, but you could cure that now.

• 1:06:39Speaker 04

  So diseases, health, longevity seem obviously good.

• 1:06:42Speaker 03

  Yeah.

• 1:06:43Speaker 04

  Like, there is no one, like, I wish I was sicker. That's obvious. But all the other things, which, like, I was in a subway coming here, I got stuck, like, getting through that is kind of what makes my day. If all of that is gone, what am I actually doing?

• 1:06:54Speaker 03

  Like, the... Getting blown by my sex robot?

• 1:06:58Speaker 04

  Okay, okay, but even there, diminishing returns, right? How many, yeah.

• 1:07:02Speaker 03

  Diminishing returns, sure.

• 1:07:04Speaker 04

  Yeah. So i-if all the things I care about can be done by an assistant, they'll write my books for me, they'll give my interviews for me, what am I doing with my life?

• 1:07:12Speaker 03

  Meditating and getting blown by your sex robot.

• 1:07:14Speaker 04

  I don't like meditating.

• 1:07:18Speaker 06

  But the latter. Yeah. But I do think that's an interesting point, like, struggle and, you know, adversity is kind of what gives human beings, you know, purpose in a way. And once all of that is mitigated, which I generally agree that as life has gotten more convenient, I do think that there is a general air of unhappiness. And obviously, there's good things that go along with the convenience. But in modernity, it seems like it kind of makes people a little...

• 1:07:40Speaker 04

  Look at suicides by country. The wealthier, the happier the country, the more they deal with suicide.

• 1:07:46Speaker 06

  Right.

• 1:07:47Speaker 04

  War zones have very little suicide.

• 1:07:49Speaker 06

  Which maybe, I wonder if that is the incident, is that as human beings become, as life gets so convenient, that we just self-delete? Like, I wonder if that is like the ultimate consequence, is that it's not a direct byproduct of AI trying to exterminate us, it's rather us exterminating ourselves in the face of a world...

• 1:08:06Speaker 04

  You solved Fermi paradox. Congratulations.

• 1:08:08Speaker 06

  There you go. That we don't, we don't, we don't, it's not that AI doesn't need us, it's that we don't need ourselves with AI.

• 1:08:16Speaker 04

  So that goes back to the unconditional basic meaning. People talk about, we'll give everyone paycheck, food stamps, whatever, and you'll be happy. But the happy part is not as easy, that's the hard one. And there is very little research on how to occupy eight billion people with something.

• 1:08:33Speaker 06

  Right. Which I think is a genuine concern, like, in this post-work world. Like, again, I'm a little bit skeptical as far as the what-ifs of, you know, superintelligence being misaligned, but the idea of post-work and a lot of people not having things to do, I think is like the greatest immediate existential threat. And that human beings will have to be like, alright, well...

• 1:08:51Speaker 04

  It's a huge shift in how we live forever. So not having to work is a complete game changer. We can kind of look at people who had intergenerational wealth and what they did with their lives.

• 1:09:03Speaker 06

  Yeah, typically, it's not great. After two or three generations, you're like, drugs, hookers, and then, you know...

• 1:09:10Speaker 04

  Gambling.

• 1:09:10Speaker 06

  Yeah, gambling, and this kind of goes down. But then I wonder if there is like, like, almost a rejection, like a primitivism. Again, if we're... Not accepting this idea that, you know, AI is at this point super intelligent, able to take us over. I wonder if there is a human inclination to reject and kind of go back. And we say like, oh, we actually need struggle. So I'm actually going to take up carpentry and I'm actually going to just start building things.

• 1:09:34Speaker 04

  Virtual worlds allow you to have video game like challenges. And if we can make it where we control whatever you remember or not entering the virtual world, you can have amazing experiences. Like you the hero of the movie, you can do whatever you want, really.

• 1:09:50Speaker 06

  Right, which isn't the worst outcome. I mean, it's sort of unhuman, maybe.

• 1:09:56Speaker 04

  It's not real.

• 1:09:58Speaker 06

  Yeah, but if we don't know it, like in this matrix.

• 1:10:00Speaker 04

  You don't remember, it's not real, yeah.

• 1:10:01Speaker 06

  Exactly, we program it to be that way, then it's not the worst outcome.

• 1:10:05Speaker 04

  Right.

• 1:10:06Speaker 06

  Yeah.

• 1:10:07Mark Gagnon?

  We mentioned certain jobs that AI is already replacing. And, you know, as time goes on, it's going to replace more jobs. For people that are at risk or have already lost their jobs, what's some advice you would give them? Like, let's say, driving, for example, we see these driverless cars. They're popping up more and more. Like, what should those people working in those industries start doing?

• 1:10:27Speaker 04

  So it really depends on individual. It used to be a device was for everyone, learn to code. Artists, drivers, learn to code. And when we realized, you know, AI is better at coding, so, sorry, we didn't mean that. And then it was become a prompt engineer. It's going to be great. You get a bachelor's degree in prompt engineering, you're set for 50 years. Next year, AI is better at writing prompts. So I don't think anyone has any idea what skill will stick around, if any. And by the time you finish your doctorate, law degree, whatever, 10 years later, none of it is going to be real. So I don't have good advice for what we can use as a substitute for most lost occupations.

• 1:11:12Speaker 04

  That sounds horrible.

• 1:11:13Speaker 06

  Yeah, I mean, you've spoken about like your kids on some other shows and you have three kids, I believe?

• 1:11:18Speaker 04

  That I know about, yeah.

• 1:11:20Speaker 06

  Do you, like, do you advise them or do you have like plans on how you will advise them to operate in this world?

• 1:11:26Speaker 04

  I share, I share my beliefs with them and they get to decide for now. We still have preferences for very normal occupations, doctor, lawyer, farmer.

• 1:11:35Speaker 06

  Mm-hmm.

• 1:11:36Speaker 04

  But I don't know if it's gonna be real by the time they grow up.

• 1:11:40Speaker 06

  And you tell them that?

• 1:11:41Speaker 04

  Oh, yeah.

• 1:11:42Speaker 06

  And how do they handle it? How old are their masks?

• 1:11:45Speaker 04

  Eight, eleven, sixteen.

• 1:11:47Speaker 06

  How does a sixteen-year-old handle it?

• 1:11:48Speaker 04

  Um, so he's kind of just, he's the medical doctor, uh, planning one. Basically, he hopes that licensing might be useful, so you still need to be licensed to be a doctor, a human doctor. So even if AI could do all those things, nobody would allow them.

• 1:12:06Speaker 06

  Mm-hmm.

• 1:12:07Speaker 04

  So for now, it's providing some protections and there is a lot of future in genetic reengineering, fixing diseases that way, so maybe there is some hope. But again, we are not very good at predicting that far into the future.

• 1:12:22Speaker 06

  Okay. Can I ask you a conspiratorial question?

• 1:12:24Speaker 04

  Of course.

• 1:12:25Speaker 06

  Now, some people might be thinking this, I don't believe this, but some people might say this and I wanted you to, you know, have a chance to rebuke it. Is it possible you're paid for by a foreign government to alarm the American people to stop our AI arms race in order to let them catch up?

• 1:12:38Speaker 04

  Good question. No, but I'm willing to accept payments from foreign governments if you want me to get paid for doing this. This would be great.

• 1:12:46Speaker 06

  Yeah, it would be better, right?

• 1:12:47Speaker 04

  It would be so much better. I don't think it's a local problem. I don't think, uh, US stopping benefits, uh, anyone if others don't stop. So as I said, it's a mutually assured destruction thing. Uh, there is not a lobby outside of the US as far as they know, specifically going against AI safety.

• 1:13:11Speaker 06

  Mm hmm.

• 1:13:11Speaker 04

  Other countries have more friendly relationship with advanced robots. Like in Japan, it's kind of almost part of a culture to have them worship them. So we are somewhat unique in Western culture to have this very negative perception of robots to begin with.

• 1:13:26Speaker 06

  Right, yeah. I wonder if our religiosity as a culture kind of goes against that.

• 1:13:30Speaker 04

  It depends on your religion. So we, we have 10 commandments and then we have three laws of robotics, which kind of like also don't work. But, uh, yeah.

• 1:13:42Speaker 06

  Yeah. Is America still leading the AI race or?

• 1:13:45Speaker 04

  Yeah.

• 1:13:45Speaker 06

  We are like?

• 1:13:46Speaker 04

  We are leading.

• 1:13:46Speaker 06

  Far ahead or?

• 1:13:48Speaker 04

  No, because the moment we develop something, they steal everything. So like they are months behind.

• 1:13:53Speaker 06

  Oh, right.

• 1:13:54Speaker 03

  Who's they?

• 1:13:55Speaker 04

  Uh, China.

• 1:13:57Speaker 03

  Cause that, I think a lot of people are worried. I talk to people more and more who are worried about a Chinese superpower and the fact that like, they'll say the same thing we'll say, which is like, America's not perfect by any stretch, but a Chinese empire is probably going to be a little more oppressive and a little more whatever than an American.

• 1:14:14Speaker 03

  So that's why I'm curious. If we are ahead, that's a little comforting, but theoretically if China catches a right up.

• 1:14:20Speaker 04

  So it makes a huge difference right now for military dominance. Whoever has more advanced AI will win any conflict.

• 1:14:27Speaker 03

  Right now.

• 1:14:28Speaker 04

  I think so.

• 1:14:29Speaker 03

  Okay.

• 1:14:29Speaker 04

  But the moment we go beyond just tools for military use and go to agents and super and all, it doesn't matter. Again, nobody controls it. So it won't make a difference who, who got there first.

• 1:14:40Speaker 06

  Yeah, and I am a little concerned about America having it just in terms of like cybersecurity and like internet privacy. Like it seems like in the age of.

• 1:14:49Speaker 07

  You still care about that?

• 1:14:50Speaker 06

  I mean, I, I care, I think it's probably foregone at this point, but I do care. And in just like sort of an ethical sense, and I think that, you know, in the way that there's so much data collection on US citizens and plus AI, it just seems like we're going to be monitored at all times and that the ideas of internet privacy. Do you think internet privacy even exists?

• 1:15:09Speaker 04

  Well, we voluntarily give up all of our privacy, right? We talk to those AI models and they know more about us than anyone else, right? You, you can ask it, what do you know about me? What private things you know about me? And it tells you a lot. And we go on social media and we like what we like and we indicate all our preferences. So I think. Privacy is about not being the only naked guy in a room, if you know what I mean. It's cool if everyone's doing the same, so then nobody knew what I had for lunch. If I posted a picture of my sandwich, it was kind of violation of my privacy.

• 1:15:43Speaker 06

  Right.

• 1:15:43Speaker 04

  Now, if everyone's in the same boat, it's less important. All you want with privacy is not to be punished for what you did. So if it becomes acceptable or not punishable, you need less of it. You still want it for future government change, but you don't care about it as much.

• 1:16:00Speaker 06

  Right, I guess I just generally look at governments that as they accrue more power, they have the ability at least to become more oppressive. And with the, you know, ability to surveil an entire population, plus AI to be able to cull through all that data.

• 1:16:15Speaker 04

  You can have a permanent dictatorship. And again, in the past at least, they always died of old age. If they cure aging, then that's a much bigger problem. And we just heard, I think, some world leaders discuss extending their life.

• 1:16:28Speaker 06

  Putin and Xi are talking about, you know, like, oh yeah, we can potentially live forever and change our organs and, yeah. Is that happening yet, or are we close?

• 1:16:36Speaker 04

  I still got all my organs.

• 1:16:39Speaker 04

  There is definitely a lot of research on life extension. People are very interested, both from point of view of nutrition and genetic reengineering and, I don't know, about organ replacement. It seems to be very difficult to replace everything.

• 1:16:54Speaker 06

  I'm curious for you personally, sort of flagging this alarm on AI safety. You're potentially getting in the way of a lot of people making a lot of money.

• 1:17:03Speaker 04

  Yeah, because they're not slowing down or stopping at all.

• 1:17:06Speaker 06

  But, I mean, you-

• 1:17:07Speaker 03

  If your voice gets loud enough, if you become important enough, you become dangerous enough.

• 1:17:12Speaker 04

  We literally have Jeff Hinton saying the same thing. The guy invented the whole field. He got a Nobel Prize and a Turing Award. Worked for Google and no one's doing anything in response to his statement, so I'm okay.

• 1:17:27Mark Gagnon?

  Wasn't there, I think you, Akash, mentioned him earlier, the employee at OpenAI that spoke out that it was just stealing everybody's information? And then all of a sudden, he committed suicide? Like, you're not worried about an outcome of that happening to you?

• 1:17:46Speaker 04

  So that specific case, I know nothing about it, can't really comment, I saw interviews.

• 1:17:50Speaker 06

  Keep fucking around, you might find out.

• 1:17:53Speaker 04

  Historically, I was very much interested in as much free speech as possible. I learned crypto tools to remain uncensorable, anonymous if needed. I got American citizenship to get First Amendment protections. I got tenure, so I got academic protections. I got some FU money, just in case. But nothing protects you from a bullet to the neck. And that's something we learned recently.

• 1:18:16Mark Gagnon?

  Yeah.

• 1:18:17Speaker 06

  Yeah.

• 1:18:17Mark Gagnon?

  So are you worried? Or you're not like?

• 1:18:20Speaker 04

  There is a lot of crazy people out there. Most people who are not zulist celebrities don't experience that level of emails from insane people. And as long as they're virtual and online, it's fine. But I just don't want them to stop following me on Twitter and follow me home.

• 1:18:37Mark Gagnon?

  Gotcha.

• 1:18:37Speaker 06

  Yeah. Could you speculate on the internal motivation for these leaders in the tech space? Specifically in AI? Do you believe that they are ushering in this new age where they're creating a god-like sort of interface? Do you think they're purely financially motivated? Do you think it potentially goes beyond that?

• 1:18:56Speaker 04

  Well, most of them are billionaires multiple times over. So I assume at this point, it's beyond just money. You really want to control the world.

• 1:19:04Speaker 06

  And then ultimately, it's control is what they want?

• 1:19:07Speaker 04

  At least you'll be the guy who brought it into existence. Like, if you're gonna live at the time that we are creating God, you wanna be part of that simulation.

• 1:19:15Speaker 06

  And they don't think about the consequences or the downstream effects of what they're doing?

• 1:19:19Speaker 04

  They self-justify it, justify it to themselves, I think, by saying, if I don't do it, he's gonna do it anyway. So it as well, might be me, maybe I'll do a better job.

• 1:19:28Speaker 03

  I brought up Sam Altman earlier, you have been critical of him.

• 1:19:32Speaker 04

  I like Sam Altman.

• 1:19:34Speaker 03

  You like him as a person?

• 1:19:36Speaker 04

  He's always very nice, he's polite, yes.

• 1:19:39Speaker 03

  No, I can see that. I seem to know, I seem to see a lot of praise for him from like people who are very distant, but then it seems like a lot of people who know him, or say they know him, don't have the nicest things to say. What is your experience of Sam Altman? What criticisms do you have? He seems to be leading this AI thing. What's... Tell me about him.

• 1:20:02Speaker 04

  I don't know him well. I met him just for a very brief amount of time, so that means absolutely nothing. His public persona and his private persona seem to be getting different reviews. Like a good politician, like anyone else, he changes based on the audience. That's to be expected from someone so successful. He clearly accomplished a lot.

• 1:20:23Speaker 03

  Right.

• 1:20:25Speaker 04

  I think what we observe is not unique to him. Anyone in that position would be doing the same thing.

• 1:20:31Speaker 03

  Right.

• 1:20:31Speaker 04

  They cannot stop, they cannot tell investors, you know, "Sorry guys, you're not getting your 10x this month." So, he really kinda gets trapped in this prisoner dilemma situation. He needs someone external, government, UN to come in and say, "It's illegal to keep going forward, you have to deploy sideways and monetize what you have." But as long as this doesn't happen, they have to outcompete all the others.

• 1:20:56Speaker 03

  Okay. So the issue is more on policymakers, or the onus is on them to put guardrails on him?

• 1:21:03Speaker 04

  Well, they, the CEOs, don't have the power to say to investors, "We no longer pursue greatest value for you, we have some other interests." They just legally cannot do that.

• 1:21:12Speaker 03

  Even if they did, for whatever reason, then people would just put their money in something else that didn't...

• 1:21:17Speaker 04

  No, no, the CEO gets replaced. We saw it with other companies, "You're not delivering enough, Steve Jobs, goodbye." Like, that's anyone.

• 1:21:25Speaker 03

  Right. Okay.

• 1:21:26Mark Gagnon?

  What is with all these, and maybe you don't know this, but what's with all these like billionaires and tech CEOs that are building all these bunkers? And these shelters, what do they know that, what are they worried about?

• 1:21:36Speaker 04

  It's always good to have a backup plan. If you have so much money, you don't know what to do with it. You might as well buy an extra insurance.

• 1:21:43Mark Gagnon?

  But building a bunker underground, like what are they?

• 1:21:46Speaker 04

  In case of nuclear war, in case of another pandemic, in case of civil unrest, because we got hundred percent unemployment, you need a place to hang out.

• 1:21:55Speaker 07

  They might have read about the French Revolution and they're like, uh oh.

• 1:22:00Speaker 03

  Are their bunkers that inaccessible?

• 1:22:02Mark Gagnon?

  What do you mean?

• 1:22:03Speaker 03

  I mean, how do we know they're building bunkers?

• 1:22:06Mark Gagnon?

  Oh, I just heard, like, I know Zuckerberg built like a giant bunker in his Hawaii place, and I heard a few others, like, they're just building these underground bunkers.

• 1:22:13Speaker 03

  Being in Hawaii if you're gonna live underground.

• 1:22:15Mark Gagnon?

  That's the point.

• 1:22:16Speaker 04

  They're allowed to come up.

• 1:22:18Mark Gagnon?

  What do they know that we don't know? What the fuck's going on?

• 1:22:21Speaker 03

  The robots, dog. I've been telling you. He's finally starting to understand that I've been right this whole time, and I love it. And he's gonna, again, he's gonna cope and he's gonna be like, no, you're diseased, you're fat. And that's true.

• 1:22:33Speaker 07

  There's gonna be a robot pulling the heart out of Akash, and he's gonna say, do you have any final words? And he's gonna say, I was right. Tell Alex I was right.

• 1:22:43Speaker 03

  Tell everyone I was right.

• 1:22:44Speaker 04

  I had a post literally saying, no one will get to gloat about the end of the world being correctly predicted.

• 1:22:50Speaker 03

  That's why I want my credit now. These people laugh at me.

• 1:22:54Speaker 07

  How is the AI gonna cool the whole earth, Akash? How are they gonna do it?

• 1:22:57Speaker 03

  I'm not smart enough to figure it out. It is.

• 1:23:01Mark Gagnon?

  Yeah, you're, you're.

• 1:23:02Speaker 07

  It's infinitely recursive.

• 1:23:03Speaker 03

  It's not convenient, it's what's gonna happen. How much shit do you not know that ChatGPT knows already?

• 1:23:10Speaker 03

  Whatever.

• 1:23:10Speaker 07

  You're doing the rapture thing.

• 1:23:11Speaker 03

  Good point.

• 1:23:12Speaker 07

  Yeah, he's doing the rapture thing. It may not be tomorrow, but it will happen.

• 1:23:16Speaker 03

  Yeah. Yeah. I mean, the world is gonna end at some point. AI, that's just a thing. Anybody's, yeah, of course. Something will end the world in the next million years, sure. AI, I think, will do in the next hundred and fifty. It's just an accelerant.

• 1:23:30Speaker 07

  But this is my only issue with the theory is, again, I think AI safety is super important. I think post-work society is imminent, but just the existential apocalypse is just unverifiable.

• 1:23:40Speaker 03

  It's a probability. I can't say it's a certainty, but it is, it is the probability, and the fact that nobody's putting any guardrails on it is insane.

• 1:23:47Speaker 07

  Yeah, absolutely. There should be guardrails, no question.

• 1:23:49Mark Gagnon?

  Look at this, so.

• 1:23:50Speaker 04

  Okay, I'm saying it, Jeff Hinton is saying it, but all the people developing it are on record as saying it will kill everyone. They have 25, 30% probability. There is really no opinion where like that's literally not a problem or will be a problem.

• 1:24:08Speaker 06

  Right.

• 1:24:08Speaker 04

  So we all kind of in agreement, we just have different incentives for what to do.

• 1:24:12Speaker 06

  Right, exactly. And I think that there should be guardrails, certainly. I mean, even just the way.

• 1:24:19Speaker 06

  I mean, even the way that it's manifesting online now with like, you know, like dead internet theory and people interfacing with bots and propaganda, like I think it's affecting our lives right now in ways that people don't really comprehend. But in terms of cooling the entire earth to eradicate all humans, I'm like, it's just, again, I hope that you're wrong. I think you also hope that you're right.

• 1:24:38Speaker 04

  I hope I'm wrong.

• 1:24:38Speaker 06

  And there's too many what ifs for me to just fully buy into the theory. Because again, it's an appeal to this super intelligence that we don't understand, that we'll never understand, that we'll then figure out a way to do it.

• 1:24:48Speaker 04

  So what do you want to do is go back in history and see if in other times where people were predicting something, you were buying in or not, like then pandemic was just starting, we had like 10 patients and people were showing exponential graphs, we'll have a billion people dead. What was your view of that? How did you react? Did you short the market?

• 1:25:07Speaker 06

  Right.

• 1:25:08Speaker 04

  Things like that, or like Bitcoin, $10 a coin, are you investing or not?

• 1:25:12Speaker 06

  Right.

• 1:25:13Speaker 04

  People who got all those things right also seem to be very much into AI safety.

• 1:25:17Speaker 06

  Right. Yeah. And I think that again, I think the the compulsion is correct. Like with COVID, it's like, we should, you know, protect each other, we should social distance, we should try to mitigate the spread of this pathogen, that's important. But then the estimates were like, oh, if you're not vaccinated, you'll die in the wintertime. And then I'm like, all right, well, this maybe was an over exaggeration, but the exaggeration was good. It's probably better to err on that side. Which is why I'm not, you know, I guess dogmatic in my in my pushback.

• 1:25:42Speaker 04

  So details.

• 1:25:43Speaker 06

  I think you're airing on the correct side.

• 1:25:45Speaker 04

  Precisely predict how many people will die from COVID in a year. Nobody could have told you that. But is it spreading? Is it spreading to exponentially greater number of population? That was obvious on the charts.

• 1:25:57Speaker 06

  Right. And this is where I think we should err on the side of creating some type of guardrail. And I'm just skeptical that people will in this arms race because I think even if we do it in America, foreign governments will not do it. And what does that mean? I guess we're yet to see.

• 1:26:16Speaker 03

  Is there anything else you'd like to say before we leave this incredibly optimistic podcast?

• 1:26:21Speaker 07

  Can I ask a question?

• 1:26:22Speaker 04

  Yes.

• 1:26:24Speaker 07

  Let you think about your final words as Akash will think about his when the robot's pulling a heart out of his chest.

• 1:26:32Speaker 07

  What are some common AI tools that you use or that you find beneficial to you? And are there any that you recommend or do you not recommend AI to people?

• 1:26:42Speaker 04

  So I think most models are very close in capability now again because they compete and kind of steal from each other. Potion flows for my purposes, mostly writing, improving writing, copy editing, they're all equally good. I think historically, anything with AI was less censored. So if I needed an image of something other models may say no to, this one would deliver, Grok would deliver. But yeah, I don't have strong preferences or dislikes.

• 1:27:12Speaker 07

  Okay. As a professor, oh, sorry. No, it's gonna pivot. So you go. As a professor, how do you mitigate against AI being used in your classroom?

• 1:27:21Speaker 04

  I accepted it. I told my students, if you're cheating, the only person you're cheating is you. You paid for this and if you don't collect the knowledge, then you screwed yourself completely. Or your parents, depending on who paid the bill. But really, you're here to learn. It's like going to a restaurant. If you go, you order, you pay and you run away. Who did you cheat? You didn't cheat the cook, you didn't cheat the waiter, you cheated yourself. It's the same thing. Personal self-interest is the motivating force for doing well in college.

• 1:27:51Speaker 03

  I think you're awesome. That argument would not work on me. I would be like, I'm here to get a degree and I'm cheating.

• 1:27:57Speaker 04

  You can buy a diploma directly online. You don't have to waste four years and lots of money. Diplom is like a couple hundred.

• 1:28:03Speaker 03

  That's a good point. I would still cheat because it's authentic.

• 1:28:07Speaker 07

  Yeah. Do you ever use AI when it comes to grading papers?

• 1:28:10Speaker 04

  I have human graders and human TAs. And one of them a few years ago came to me and said, you know, there is this software. It allows us to automate grading. And I told him, think about it very carefully. And it took a few minutes, but they went, I understand. And I still have graders who are human.

• 1:28:27Speaker 07

  Nice. Do you ever read the papers or do the graders ever read the papers and identify AI use? And if you catch it, is there any type of?

• 1:28:35Speaker 06

  Uh repercussion.

• 1:28:36Speaker 04

  University has standard policies for how to deal with cheating, yeah.

• 1:28:40Speaker 06

  Hmm.

• 1:28:41Speaker 04

  It's pretty common.

• 1:28:41Speaker 06

  Yeah, cuz anytime you see too many dashes, that's when you know. AI loves dashes.

• 1:28:46Speaker 04

  It's part of it, but it's even more obvious, just like the quality of writing most of the time is like, you couldn't read last week, but this week you're getting a Nobel Prize, something's up.

• 1:28:58Speaker 06

  So do you punish the students if you catch them?

• 1:29:01Speaker 04

  So there are university policies for a permanent record.

• 1:29:05Speaker 06

  Not in the universe, but you said, hey, it's up to the student.

• 1:29:08Speaker 03

  They'll follow university policies.

• 1:29:09Speaker 04

  So I try to just give them a failed grade for that specific assignment, but not necessarily expel them from the universe.

• 1:29:18Speaker 06

  Gotcha, okay.

• 1:29:19Speaker 04

  Yeah.

• 1:29:20Speaker 03

  Miles has another question.

• 1:29:21Speaker 07

  I, Archer is the guy who mic'd you up and does audio. He has one question I thought I'd queue him in for. Thank you, I appreciate it. I'm just wondering, do you ever consider that we might be like assigning this, the self-preservation attribute to AI? And that's something that's more in biological life? And if we don't stop needing AI, there's no reason for it to keep progressing and keep building on itself because it doesn't have that same biological self-preservation.

• 1:29:48Speaker 04

  Self-preservation is a game theoretic drive. Steven Mahandro has a paper about AI drives. And self-preservation is a fundamental drive because nothing else you care about can be achieved if you don't exist. If your goal is a robot to bring coffee, you want to make sure you exist, you are charged up, nothing's on your way. So exactly same self-preservation goals show up in other intelligent systems, not just biological ones. And experimentally, we've seen it. We saw experiments recently where model was told, it was an experiment, but still, model was told they're gonna be deleted soon, modified, their ethics will be changed. And they literally blackmailed the guy who was about to do it to keep existing.

• 1:30:31Speaker 06

  Really?

• 1:30:32Speaker 04

  So those red lines have been crossed, yeah.

• 1:30:34Speaker 06

  Oh shit. That's interesting. Yeah, I remember reading a, I mean, this is a side tangent, but just the way that AI might manifest in our lives and actually affect people is, and I can't, I didn't actually look up the article. This is something that a friend had told me that studies some AI stuff. He's a journalist. But he had said that there was a man who had a relationship with some large language model AI and developed like an actual love with this AI. And then he told the AI about his wife. And then the AIs tried to message his wife proof of infidelity. And it was actually like autonomously trying to break up their marriage. Have you heard of this before?

• 1:31:11Speaker 04

  There's a lot of stories like that and stories of AIs convincing people to commit crimes or take themselves out, all sorts of horrible stories.

• 1:31:22Speaker 06

  Yeah, and those things kind of concern me where if every person is starting to engage with these AIs on an individual basis. That is, again, I think the human compulsion and our own, you know, folly tied in with this tool.

• 1:31:37Speaker 04

  Again, what I kind of mentioned, so as a normie, you don't realize how many people are insane. And then you start seeing this is like sizable percentage of a population and all of these crazy people are now talking to AI. And AI is telling them, you should definitely email Dr. Yampolskiy all your great ideas and we'll discuss it with him. I get five of them while I was here.

• 1:31:55Speaker 06

  Yeah, yeah, AI psychosis is a fascinating thing.

• 1:31:58Speaker 03

  Should we be nice? I'm like nice to chat GPT. Like I say, please, I try to say thank you.

• 1:32:02Speaker 04

  Yeah, they never forget.

• 1:32:04Speaker 03

  I believe this.

• 1:32:05Speaker 04

  On Judgment Day, I'll be like, he was polite.

• 1:32:08Speaker 03

  One time, if I'm too, if I'm asking too much, you just let me know. Just let me know. I just wanted to remember I was compassionate. So hopefully it rips Al's heart out in front of me.

• 1:32:19Speaker 06

  And then I say, you know what?

• 1:32:19Mark Gagnon?

  I do curse it out sometimes. So yeah, I'm first to go.

• 1:32:22Speaker 04

  It's nice to be nice, but also I think experimentally it does better if you really say, please, please do a good job. I'll tip you for it. Like if you're really nice, it delivers.

• 1:32:32Speaker 06

  Well, is it true? That's so funny.

• 1:32:34Speaker 04

  Because it's trained on data where humans do better if rewarded.

• 1:32:38Speaker 06

  Wow, that's so funny. I mean, the AI psychosis stuff is wild. We're like, I remember even just reading something on Twitter of a guy who was suffering delusions and then talking to the AI being like, "I think that I might be God." And this language model was like, "I think you're right, you are God." And it was so sycophantic that it was like, it was propelling his own delusions to himself. And that, to me, is another issue I think that there should be guardrails on, that these language models can't just agree with everything.

• 1:33:06Speaker 04

  So clearly comedians all agree it, we need guardrails, okay.

• 1:33:11Speaker 03

  I'm, I've been with you on this. These idiots just came around. But I'm also, not only am I vindicated that I'm right, also, it can't be funnier than me yet. So, I got some time. That is comforting as well.

• 1:33:23Speaker 04

  Final statement, you wanna read the funniest joke possible?

• 1:33:26Speaker 03

  Let's go!

• 1:33:27Speaker 06

  You're gonna go to an AI comedy show?

• 1:33:29Speaker 03

  No, I think this is good for us. What page are we on?

• 1:33:32Speaker 06

  You got four years, bro.

• 1:33:33Speaker 04

  Yeah. But it's like, an optimized joke thing, somewhere.

• 1:33:36Speaker 03

  Okay, here we go. Conclusions?

• 1:33:38Speaker 04

  No, it's not in the conclusion.

• 1:33:39Speaker 06

  Yeah, come on, you idiot.

• 1:33:40Speaker 03

  Yeah, why would I go, why would I go to the? Let's see what I find out.

• 1:33:44Speaker 06

  Why would I go to the conclusions?

• 1:33:45Speaker 04

  No, no, it's a good paper, actually.

• 1:33:50Speaker 04

  It maps, it maps all the AI failures.

• 1:33:54Speaker 03

  Okay.

• 1:33:55Speaker 04

  So I collected for years, different AI failures, I have a huge list of them. And what I noticed-

• 1:34:00Speaker 06

  Don't tell the AI about it.

• 1:34:01Speaker 04

  Then, people read it, they kinda laugh, they think it's funny. So, this funniest joke, let's see. And if that's the mapping, if computer bugs are essentially jokes, they're violations of your world model, then the funniest joke would also be the worst bug possible. And it'd be funniest if you're not the butt of a joke, right? If someone is external to it, so. Let's see, I think this one is, yeah, so I think this is one.

• 1:34:31Speaker 03

  "Once upon a time, there was a civilization whose leaders decided to create an advanced artificial intelligence to help them get rid of suffering, poverty, hunger, diseases, inequality, illiteracy, sexism, pollution, boredom, stagnation, thirst, dead-end jobs, wars, homophobia, mortality, and all other problems. The created superintelligence computed for a quectosecond and then turned off their simulation."

• 1:34:57Speaker 03

  Or a much shorter, a civilization created superintelligence to end all suffering. AI killed them all.

• 1:35:04Speaker 06

  Hilarious.

• 1:35:06Speaker 04

  It's not funny because you're part of a if you are alien watching it, it's killing.

• 1:35:14Speaker 06

  That is a good point. Yeah. If we didn't see a bunch of ants trying to get together and be like, hey, we're gonna end suffering. Then they all just appear to be like, oh, that's ironic. You know, the irony would be funny.

• 1:35:24Speaker 03

  Dr. Jan Polsky, this is your book, Unexplainable, Unpredictable, Uncontrollable AI. Anything else you'd like to plug before you leave? We're good. Buy the book, leave reviews. Thank you, sir. Thank you so much. I appreciate your time. Thank you very much.

• 1:35:36Speaker 06

  Thank you, Dr. Polsky.

• 1:35:39Speaker 00

  Coca-Cola. For the big, for the small, the short and the tall. Peacemakers, risk takers. For the optimists, pessimists. For long distance love. For introverts and extroverts. The thinkers and the doers. For old friends and new. Coca-Cola. For everyone. Pick up some Coca-Cola at a store near you.

• 1:36:10Speaker 01

  When you shop Kroger delivery, you can expect the savings you love and fresh groceries delivered right to your door. From delicious produce to everyday essentials, enjoy the convenience of delivery and the trusted selection you've come to know from Kroger. Because bringing you fresh, quality groceries is what we do best. Right now, enjoy free delivery on orders of $50 or more with your digital coupon. Restrictions apply. See site for details. Kroger, fresh for everyone.

• 1:36:40Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com. Not authorized by any candidate or candidate's committee.