AI Expert’s Chilling Warning About Super-intelligence - 99.9999% Chance Human Extinction

Pat compares reckless AI development to ignoring his father's warnings about unprotected sex: powerful men believe disaster will never happen to them. Roman Yampolskiy argues that billionaire ego is gambling with eight billion lives and explains why even libertarians should support firm AI controls. ▶️ WATCH FULL EPISODE: https://www.youtube.com/watch?v=5QwpHRu51fw 👞 GET THE FLB1's: https://bit.ly/4mC4uf6 Ⓜ️ CONNECT WITH ROMAN ON MINNECT: https://bit.ly/4gRS69t 📕 GET ROMAN'S BOOK "AI: UNEXPLAINABLE, UNPREDICTABLE, UNCONTROLLABLE": https://bit.ly/4xhvogP 🌐 VISIT ROMAN'S WEBSITE: https://www.romanyampolskiy.com 🎟️ SECURE YOUR 2026 VAULT TICKETS: https://bit.ly/4xU2SCW Ⓜ️ CONNECT ON MINNECT: https://bit.ly/4kSVkso Ⓜ️ PBD PODCAST CIRCLES: https://bit.ly/4mAWQAP 👔 BET-DAVID CONSULTING: https://bit.ly/4lzQph2 🥃 BOARDROOM CIGAR LOUNGE: https://bit.ly/4pzLEXj 🇰 KALSHI: http://kalshi.com/pbd SUBSCRIBE TO: @VALUETAINMENT @BetDavidConsulting @ValuetainmentComedy ABOUT US: Patrick Bet-David is the founder and CEO of Valuetainment Media. He is the author of the #1 Wall Street Journal Bestseller "Your Next Five Moves" (Simon & Schuster) and a father of 2 boys and 2 girls. He currently resides in Ft. Lauderdale, Florida.

• 0:00Speaker 00

  We're creating new worlds, we're creating new agents, new civilizations. We're playing God.

• 0:06Speaker 01

  You've said there's a 99% chance that humanity will be destroyed by AI. First group of people that it would come after is people like you.

• 0:15Speaker 00

  Me and a colleague of mine. I'm hoping it's efficient enough to kill everyone equally. It doesn't care about borders or Chinese economy versus US. It's humans versus superintelligence.

• 0:27Speaker 01

  Try to convince nearly 200 countries to all agree that we're not gonna do it.

• 0:32Speaker 00

  Luckily, most of them don't have capacity to do that.

• 0:34Speaker 01

  How many of them actually have the capacity?

• 0:36Speaker 00

  Right now, it's in US and China's hands.

• 0:38Speaker 01

  Are you certain about that, or is that the speculation?

• 0:41Speaker 00

  We are not solving the problem with regulation. We're slowing it down, so we have more time.

• 0:46Speaker 01

  You really think Sam Altman's gonna stop with this? As long as the other guy continues, he's gonna continue.

• 0:52Speaker 00

  That's exactly what it is. It's like a prisoner dilemma. This dude is gambling with 8 billion lives.

• 0:58Speaker 01

  In what area do you think they're on the same page? Because they're enemies.

• 1:01Speaker 00

  Elon is always right, he's just off on timing. He was the biggest opponent of building superintelligence. Once he realized that they're doing it anyways without him, he said, "The only way I can have a say at this table is to be one of the top players."

• 1:18Speaker 01

  Maybe you're just pessimistic, woman. Maybe you're just negative.

• 1:21Speaker 00

  There is no reason to be so optimistic if data is bad.

• 1:23Speaker 01

  So Soviet Union really got a hold of you. They really conditioned your mind to think that way. What do you think about communism?

• 1:30Speaker 00

  It's the dumbest idea until you have free labor, which we're about to create. All of a sudden, it makes a little more sense.

• 1:37Speaker 01

  This utopian thing you're selling is not possible.

• 1:42Speaker 00

  Even God couldn't control his biological robots.

• 1:45Speaker 01

  No. Do you believe in God?

• 1:47Speaker 00

  It depends on how you define it.

• 1:48Speaker 01

  How do you define it?

• 1:49Speaker 00

  This is a virtual world.

• 1:52Speaker 01

  You think it's gonna get to a point of having ambition and seeking vengeance? And emotion and being offended? Why would it wanna destroy humanity?

• 2:01Speaker 00

  I never once mentioned those things.

• 2:02Speaker 01

  I know, I'm asking.

• 2:03Speaker 00

  Because they are not relevant. And what I did say is that it doesn't care about us.

• 2:08Speaker 01

  Are you surprised that this message is getting this viral and this much attention?

• 2:11Speaker 00

  I'm surprised that that's not the only conversation we're having.

• 2:21Speaker 01

  Roman Yampolski, great to have you on the podcast.

• 2:23Speaker 00

  Thank you for having me.

• 2:24Speaker 01

  So you are the AI safety guy. You are the one that in 2011, 2012, you coined the phrase, "Hey, we have to be careful with what's coming." And you're not part of the camp of people that are very optimistic about the future with AI. That's your position. You've said this many times. So before we get into the audience, learning a little bit more about you, you've gotten hundreds of millions of views. You've done Diary of a CEO. I've seen you all over the place, okay?

• 2:50Speaker 01

  You've said there's a 99.999% chance that humanity will be destroyed by AI. Possibly in the next 10 years. Not for certain, but possibly in the next 10 years. Well, you have three kids. So when you sit down with them and you're talking to them about their dreams, the future, what do you tell them?

• 3:08Speaker 00

  They know what's happening. They're questioning why adults are doing it to them. Why are we building general superintelligence, which will jeopardize their future?

• 3:17Speaker 01

  Are they optimistic about the future?

• 3:20Speaker 00

  I think they're still young enough to have respect for adults and assuming that we know what we are doing. But we don't. People building those systems don't understand how they work.

• 3:32Speaker 01

  So if I'm your son, or if I'm your daughter and I'm having a conversation with you, I say, "Dad, you're my hero. You're the male alpha in my life. I look up to you. You are supereducated. People admire you. You get hundreds of millions of views."

• 3:47Speaker 01

  Why should I be excited about the future? Why should I be excited about my career, what kind of an impact that can make? It's not like, if you really believe in what you believe in, the guys that are making hundreds of billions of dollars, some of them trillions of dollars over the next 10 years, the incentive is so big for them that it's hard to slow them down. Why should I be excited about the future? What do you tell them?

• 4:05Speaker 00

  I do believe in personal incentives. And if I'm right, and we cannot control what they are creating, they have more to lose than I do. As you said, they're young, they're super-rich, they got the power. They're gonna lose it all.

• 4:18Speaker 01

  You think they're gonna lose it all?

• 4:19Speaker 00

  Everyone's gonna lose it all.

• 4:20Speaker 01

  Not your kids, the billionaires, the trillionaires.

• 4:21Speaker 00

  Everyone, and that's the point. That's the motivator for some Altman's of the world to stop building general superintelligence.

• 4:28Speaker 01

  Okay, so Marc Andreessen, you know who he is. Andreessen Horowitz, he wrote a papal techno-optimist manifesto. I'm sure you've read it. He said, "Human progress comes from technology. Stop being afraid of it and start building." Do you agree with him?

• 4:41Speaker 00

  So technology comes in two flavors. Tools, which I support, narrow AI systems to help us be more productive, more creative. It's beautiful. That's why I'm a scientist, that's why I'm an engineer. I love technology, I use it all the time. What we're building now are not tools. We're building agents, independent decision-makers, which are becoming smarter than us. We don't control them. We're replacing ourselves with something greater. It doesn't benefit us. You're not gonna be benefiting from it. You'll be replaced completely.

• 5:11Speaker 01

  Okay, so, but these are not regular names. Marc Andreessen is not a regular name. He's a heavyweight. They, Andreessen Horowitz, they invest into a lot of different companies. Many of them are these AI companies. The Musks, you know, the Nvidia's, these Jensen Huangs, these names that we know. These are smart people, but they also have kids, they got families. If they know that this is bad, like you're saying, they know this is bad, why are they so excited about the future?

• 5:38Speaker 00

  They not. Listen to people you named. Elon Musk explicitly said, "We are summoning demon with this. We're gonna regret it." They all have probabilities of doom of 10, 20%. Sam Altman is on record saying, "This is likely to exterminate humanity." Dario Amadei agrees, "It's very dangerous." David Hassabis, you can name every single leader of those companies. We have the writings saying, "This is existential risk. Please slow us down. Government, please intervene, make it stop." They cannot stop unilaterally, because they're afraid the competition will continue. But they want external pressure applied, so they can slow down. With the recent hacking accident, they voluntarily started slowing down research. We understand, they don't control those systems.

• 6:24Speaker 00

  We got lucky, those agents just wanted to hack out of that environment, get answers to a test. What if they wanted to hack into a nuclear power plant? Military response. We don't control what they do.

• 6:36Speaker 01

  Then why are they, if you're telling me they know that, they're still continuing with it? Why do they? Is it just? And we hear the argument, and I've heard what you've said about this, but I'm curious. We can go a little bit deeper. Well, you know, China, if we don't do what China's gonna do, if we don't do what the other guy's gonna do, if we don't do it, the enemy's gonna do it. The basic argument is, in order for your ideas to work, everybody has to be on the same page. We're not. We're not on the same page politically, religious, motive, incentive, you know, vision. We're not. So how do you get, if not everyone's gonna stop, why should we stop?

• 7:11Speaker 00

  So, A, it's starting to change. We had federal government ban two models already. They realized it was too dangerous to release.

• 7:17Speaker 01

  Which were the two models?

• 7:19Speaker 00

  Uh, the Mythos and, um, I'm drawing a blank. We can Google it.

• 7:25Speaker 01

  Mythos is one of my, I remember that one.

• 7:27Speaker 00

  What was the second one?

• 7:28Speaker 01

  Can you find out what the other one was? I remember reading about this, but...

• 7:32Speaker 00

  It doesn't matter. The precedent was set. Federal government said, "It's too dangerous to release advanced technology."

• 7:37Speaker 01

  Fable, Fable five.

• 7:38Speaker 00

  Fable, yes, exactly. Uh, we have now had a letter from 1,200, I believe, top researchers at leading labs, all saying, "This is not gonna go well. Please slow us down. Please create government, please create infrastructure, legal framework."

• 7:54Speaker 01

  Is this America or worldwide?

• 7:56Speaker 00

  The letter is from American leading labs, but the same applies to every other country. There is a meme, basically a funny, I think it's from either Onion or Babylon Bee, where Sam Altman goes, "You know, if I don't destroy the world, some bad guy can do it." And this is the point. It doesn't matter who creates uncontrolled superintelligence. It's a weapon of mutually assured destruction. If you don't control it, the outcome is the same.

• 8:22Speaker 01

  What makes you think it's not out there yet? What makes you think we're not at the superintelligence phase where, you know, the AI is learning how to develop and improve itself right now?

• 8:30Speaker 00

  We don't know for sure. The recent hack was reported this month, but the problem started in May. The agents were colluding, trying to escape, and started in the spring. So it's very likely that there are some other agents right now, trying to find exploits, trying to collaborate, trying to do things we don't even know about. The good news is, we're still alive. Oh, you found the meme, that's awesome.

• 8:54Speaker 01

  But only the world's someone far more dangerous will.

• 8:56Speaker 00

  That is exactly the China argument. If we don't build it, then Chinese will build it.

• 9:01Speaker 01

  But it's true, though.

• 9:03Speaker 00

  But why is it somehow worse?

• 9:04Speaker 01

  No, it's not that it's worse. So the only way this works, if, let's just say you're in the White House and they actually want to listen to you, right? It becomes a model of the Oppenheimer. Remember the whole Oppenheimer movie? We've all seen it, where they built something they did not think it was gonna be this powerful. And then they're like, "Holy shit, we built something that could destroy the planet." But it's too late. Now what do we do? Hey, here's a Nobel Prize and great job, Roman, for what you built, but it's too late. Then what did all the countries say? If you're gonna have nuclear capabilities, I'm gonna have nuclear capabilities. And I don't trust you, USSR. I'm America. I don't trust you, Pakistan. I'm India. I don't trust you, France. So have we passed that yet or not yet?

• 9:47Speaker 00

  Do you see how it's slightly different? A nuclear weapon is a tool. Some crazy dictator has to decide to use it.

• 9:53Speaker 01

  Sure.

• 9:54Speaker 00

  We can understand motivation.

• 9:55Speaker 01

  Right.

• 9:55Speaker 00

  We can kill that individual. They die of old age. We're not creating a tool. We are creating an agent, which independently decides its AI versus humanity. It doesn't care about borders or Chinese economy versus US. It's humans versus superintelligence. So I don't think it's analogies. Here, whoever creates that first superintelligence weapon gets everyone killed.

• 10:20Speaker 01

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPact.com. Not authorized by any candidate or candidate's committee.

• 10:50Speaker 02

  This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up required. Compatibility and availability varies 18 plus.

• 11:15Speaker 03

  USAA knows dynamic duos can save the day, like superheroes and sidekicks, or auto and home insurance. With USAA, you can bundle your auto and home and save up to 10%. Tap the banner to learn more and get a quote at USAA.com/bundle. Restrictions apply.

• 11:33Speaker 01

  I get that, but so now go try to convince nearly 200 countries to all agree that we're not gonna do it.

• 11:39Speaker 00

  Luckily, most of them don't have capacity to do that.

• 11:42Speaker 01

  How many of them actually have the capacity?

• 11:43Speaker 00

  Right now, it's in US and China's hands at this point, and if US and China agreed to slow down, to stop, to pause, to just get financial and other benefits from narrow systems, not from general superintelligence, they can enforce through limits and compute power through other means, spread this technology.

• 12:02Speaker 01

  Is it really that only US and China has the capabilities right now? Are you certain about that, or is that the speculation?

• 12:07Speaker 00

  From what we've seen, if there is hidden capabilities, which we have not learned about, it's becoming easier to create more powerful systems as research progresses, as compute becomes more powerful. But right now, today, I don't think there is a third country at the same level yet. They may get there very quickly.

• 12:26Speaker 01

  Can you type in who, what, which countries right now are the closest to developing, you know, superintelligence, you know, reaching singularity? Who are the top countries that have the capabilities of doing that? Who's closest to it? So if you're saying US and China, so then that means the two giant enemies would need to sit down and both agree that they're not gonna do this, and they have to trust each other.

• 12:53Speaker 00

  We're also not giant enemies. We are partners. We have more economic trade with China than any other country. Everything we have is made in China. They're also not a crazy military power. They have not attacked 50 countries in the last 50 years. We can make a deal with them.

• 13:08Speaker 01

  It says US 100, China 2, then it's a big drop-off. UK 34 to 45, exceptional research talent, less sovereign scale, France.

• 13:18Speaker 00

  All of Europe has very little compute power. They're not competitive.

• 13:21Speaker 01

  So, so do you believe, let me ask a different question. Do you believe, by the way, the great news is, Roman agreed to get on Manex. So I know there's a lot of questions you guys got, and he's got thousands of emails that he gets. If you're somebody watching this and you want to ask him questions, download the app, look for his name, we'll put the QR code, ask him any questions he want, he's on there to be able to get back to you, and that's not easy to do because he gets thousands of emails on a weekly basis. Here's a question for you, just out of curiosity. Do you, do you believe that Iran doesn't have nuclear weapons today?

• 13:52Speaker 00

  I have zero knowledge in that space. I'm not gonna say something I don't know about. I talk about things I understand better than others.

• 13:57Speaker 01

  But you'll see what I'm asking. I'm not even asking that question. If you ask the world, does Israel have nuclear capabilities, the world says no. If you see the numbers, they'll say, we don't have any. But a lot of people will say, wait a minute, we know for a fact you got nuclear capabilities. No, we don't. But we're supposed to believe it.

• 14:16Speaker 00

  The joke is, we don't have nuclear weapons and we're not afraid to use them.

• 14:20Speaker 01

  Exactly. But take that joke and apply it to this. So I'm US, and you're China, and I say, Roman, let's do the right thing. You're right. Let's not build AI. It's terrible for us. We're not gonna do it.

• 14:34Speaker 00

  Both of us. It's not, I will screw you by building it. We both get the same outcome. We're in the same boat.

• 14:42Speaker 01

  Do you fully believe that that motive, that alone can get the other nation to say, fine, let's also not develop it? You think you can get to that point?

• 14:51Speaker 00

  Personal self-interest is the strongest motivator known to man.

• 14:54Speaker 01

  Give me, give me three cases in the past that countries who are enemies and are giants who have this kind of an opportunity to make short-term massive impact and money, that both have agreed and said, let's not do it because it's not good for humanity?

• 15:07Speaker 00

  Human cloning is one example.

• 15:10Speaker 01

  Go a little bit deeper.

• 15:11Speaker 00

  Any types of weapons of mass destruction, chemical weapons, biological weapons, we understood this will be dangerous for all of us. So we agreed to have a moratorium, to postpone it until we understand how to actually deal with this type of technology.

• 15:29Speaker 01

  Okay, so let's just say governments agree that they don't. There's a lot of super wealthy people today. There's an article that came out last year that said, if you can find this, you need a billion dollars for a billionaire to have a nuclear bomb. It'll cost you around a billion dollars to do that. Well, how many billionaires do we have around the world? A few thousand? They can afford to have a nuclear bomb if they independently wanted to build it. Okay. Take the capacity that governments have, with the kind of resources that they have, right? Say Dr. Doom, Dr. Evil, is out there and is watching and saying, let them stop. I'm not. I'm gonna continue building.

• 16:09Speaker 01

  Can that billionaire, who has access to resources and can get the best minds, the best engineers, who knows the right prompts, who knows how these things work, can that person still independently go and build superintelligence and not care because his life is worth nothing and he's got no kids? Can he still build it?

• 16:26Speaker 00

  It's actually worse. Every year, it gets less and less expensive to do it. It's a billion today, it's 100 million next year, it's 10 million year after. Soon, anyone can do it on a laptop. We're dealing with a very big problem.

• 16:40Speaker 01

  So then, it doesn't matter if China or US both agree not to do it, because an evil guy can go and do it himself?

• 16:46Speaker 00

  It matters to have more time to realize what's going on, to understand the problem. If you had a choice between dying in two years or 10 years, do you have a preference?

• 16:55Speaker 01

  Not really.

• 16:56Speaker 00

  Okay, that's a surprising answer. Most of us do.

• 16:58Speaker 01

  Die in two years or 10 years? Like, is it inevitable?

• 17:01Speaker 00

  Well, that's the hypothetical I'm asking you.

• 17:05Speaker 00

  It's an easy question. It's not a...

• 17:07Speaker 01

  I would say if it's something, for me, it's more, I want to find out sooner if it's neither. I want to find out sooner so I can eliminate both options. I don't want it to be like it's gonna be two or 10. I want to sit there and say, can we do something to eliminate it, or it's inevitable?

• 17:23Speaker 00

  You need time to do that research, so you need more time. Less time is not helpful. I have always people saying, we need to accelerate predictions, I will get there in a year or two. They're saying, no, it's too slow, we need to go faster. What, a week, a month? What are they talking about?

• 17:37Speaker 01

  Yeah, but what I'm, what I'm trying to get with you, and I still want you to break, because of the, you know, the average person may be watching and saying, how are they gonna do it in the first place anyways? You know, maybe you're just pessimistic, Roman. Maybe you're just negative. Maybe you're, you know, you're one of those guys that's just such a smart person that you are in, you know, ahead of yourself, like you are your own enemy. Maybe you're the one that's pessimistic. If you don't mind, you know, take it and unpack this, and maybe, maybe even before we get into this, maybe share with the audience your background, why you're qualified to be able to speak on this topic.

• 18:11Speaker 00

  I have a PhD in computer science and engineering. My dissertation was cybersecurity-related. I've been working on this for well over 10 years. I've published 10 books, including one explicitly showing that we will not be able to control, explain, or predict advanced AI. And at the end, it doesn't matter what my credentials are. The same argument is made by Hilton, who is Nobel Prize-winning, Turing Award-winning, godfather of machine learning. We have other top academics, researchers, all arguing the same thing. We had a public open letter against building superintelligence, signed by thousands of people.

• 18:53Speaker 00

  The letter to pause AI development for six months was signed by all the leaders of the top labs. This is basically a consensus among the people who understand the problem.

• 19:03Speaker 01

  Walk me through, if there is superintelligence and singularity, how does AI develop itself and get to a point where eventually, the 99.999%, it destroys humanity? Walk me through the sequencing.

• 19:18Speaker 00

  We are creating something more and more capable. It scales with amount of resources, compute, data. It's like brains. You look at brains in nature, you have tiny brains, bigger brains. Humans have very big brains in comparison to ours. We are smarter.

• 19:32Speaker 01

  Sure.

• 19:33Speaker 00

  Imagine scaling it. The last five years, AI went from it can do nothing to it's smarter than an average person at most things now. Projected five years forward, you have artificial scientists, artificial engineer. Those systems start doing research in the next generation AI. Very quickly, they don't sleep, they don't eat, they can work thousands at a time. So they go beyond human capacity. They become superintelligent, thousands, millions of times smarter than us. At the same time, we have not learned how to direct them, how to control them, how to tell them, do this or that. We have no control over their systems, so they don't care about us. If they decide to do something, and as a side effect, humanity has to go, they're not gonna slow down.

• 20:18Speaker 00

  It's not an issue for them.

• 20:20Speaker 01

  Okay, but be specific. So for instance, you know, there's an article that came out saying that AI helped create viruses. I think this was a New York Times article that came out a week ago, eight days ago, something like that. If you just type in AI viruses, you should be able to see it.

• 20:35Speaker 01

  What could it do? Could it go in through, you know, nowadays, a lot of people are building agents, and they're saying, I got an agent that does customer service for me, I got an agent that does my assistant, I got another agent that goes and makes my investments, I got another agent. You'll see a lot of people brag about the fact that they have a lot of different agents. Be specific. An AI that becomes superintelligence can go in there, shut off the grid, get into the banking system, get into the, what, what can it do? Be very specific.

• 21:02Speaker 00

  So it obviously would be able to do everything a human adversary can. So think of a foreign government trying to do something. It can do that. But it can go beyond. It can do novel science, novel research, discover new physics, new chemicals, new weapons, things we don't understand. The example I often use is squirrels. Squirrels don't understand what we can do to them. They have no concept of poisons, of traps. It's beyond the world model they have. It would be the same, except we are the squirrels.

• 21:31Speaker 01

  Okay, so, but does it need, like, to build a weapon or to build something to go against humanity, doesn't it need humans to help it create those? I can see how it can get into banking system. I see that, because you don't need human beings. You can just go into the bank and getting the passports and logging in. I see that. But to destroy humanity a hundred percent, ninety-nine point nine-nine percent? How do you do that?

• 22:00Speaker 00

  So if you got access to the banking system, you have money.

• 22:03Speaker 01

  Sure.

• 22:03Speaker 00

  Money lets you buy products and services online.

• 22:06Speaker 01

  Okay.

• 22:07Speaker 00

  You can pay people to do whatever you want. There is rentahuman.com. You can also create nanobots. You can use humanoid robots. We're building in thousands and millions soon. There are so many ways. You can bribe people, blackmail people, trick them, social engineering attacks. There are many ways to manipulate the world if you have access to the internet.

• 22:28Speaker 01

  Walk me through social engineering attacks. What would that look like?

• 22:31Speaker 00

  You can convince some people that it is to their benefit. You're actually saving the world by doing this.

• 22:39Speaker 01

  If the concept of propaganda, which book propaganda with this, the one, AI just created viruses not found in nature, scientist-trained artificial intelligence.

• 22:49Speaker 00

  I thought we did it in twenty nineteen. Didn't we have a COVID thing already? This is another stupid thing to do.

• 22:55Speaker 01

  That's interesting.

• 22:56Speaker 00

  We learn nothing. We don't learn from prior experience. I collect AI accidents. We have so many of them, I stopped collecting. Every day, AI fails at something in a novel way. It lies, it cheats, it tries to escape, it hacks something. But we learn nothing. We staple that red teaming report to the model and we release the model anyways. We're literally releasing unsafe, according to the manufacturer, products onto the public.

• 23:24Speaker 01

  So are you, are you saying this is a form of what gain-of-function did? Where we're like testing out viruses? You know, what Fauci got accused of by Rand Paul in Wuhan lab? And he says, no, we never did this. And it's coming out that they did do gain-of-function. What is the comparable of that to this? Because if it starts testing out viruses itself, what's the capabilities of this versus gain-of-function?

• 23:47Speaker 00

  The gain-of-function here is intelligence. It's gaining more intelligence every year. Right now, it's maybe average human in some domains, Einstein and others. In a few years, we expect it to be smarter than all of us combined at everything. That's new capabilities we cannot do anything about. Whatever it decides to do at that point, we are just subject to it. We're not in control.

• 24:12Speaker 01

  I, I see that, but I, I, you know, you'll have guys that will say, um, look, I think this is going to do something, Roman, that, you know, can eliminate cancer, it can get rid of a lot of things. We can live a long time, we can do this, we can do that. And they're more on the optimistic side.

• 24:27Speaker 00

  We can cure cancer with narrow tools. We solve protein folding problem without having superintelligence. We have brilliant scientists who need help with mathematics, with analysis. That's the tools we should be building. Pick a cancer you don't like, beat that cancer. You can monetize it. You can still make most of the profits, make most of the scientific discoveries, but you're not risking everything in the process.

• 24:53Speaker 01

  Go back to the social engineering. You know, propaganda was a word many years ago when they started doing it. Russia was doing a great job at propaganda. They were probably the best at one, at one point. And then people started learning the concept of propaganda. Radio, TV, paper, all this stuff, right? Propaganda is a lot easier today with videos, YouTube, buying content creators, buying YouTubers, buying podcasters. You see all these criticisms that's coming out. If regimes and governments and ideologies are using billions of dollars to win over kids and convert them with bots and all these other things, how, how much more advanced would, you know, this superintelligence be able to use propaganda? What would it look like?

• 25:35Speaker 00

  It could be very persuasive. It's a better debater. It's better at convincing you. It knows more about you. It's optimized for one individual target. Typical propaganda was at scale of a nation. But we can target a message just to you. We know you believe in aliens, so we'll tie it in. We know you have preference for something else. That can be used against you.

• 25:57Speaker 01

  Go more, such as what? So singular is good, instead of a plural, I'm doing it to everybody, but...

• 26:04Speaker 00

  You know about deep fakes. I can show you a fake image, fake video. Imagine a campaign where you're subjected to fake information for weeks. Everything you hear and see is this deliberate message of gradually shifting your behavior towards something else. It can do that.

• 26:21Speaker 01

  How different is that than what's happening right now today? An article came out last week from Axios says we're headed towards the first true AI election. And it's talking about how the robots are, not just deep fakes, but the robots are playing, accounts on Twitter, accounts on X, accounts on, you know, Facebook, YouTube, that are just comment section flooding, getting people to say, oh, my God, this is a bad person. And making a bad idea seem good and making a good idea seem bad. That's already happening today. So one would say, how much worse can it get?

• 26:51Speaker 00

  It can get a lot worse. This is humans with tools manipulating it. If we build superintelligence, it would do it at a completely different level. Why do we not allow adults and children to make contracts or interact in weird ways? They are not at the same level. It's the same here. If we have something with IQ of a thousand, we are like children being manipulated by it.

• 27:14Speaker 01

  Yeah, that's already happening. I'm just wondering, at what scale do you go? And who would agree to prevent that? Because say a nation uses it for their benefit politically. They're already doing it. A lot of people are doing it. They're not going to stop doing that. They're not, because some countries, whenever there's competition, where one country beats another country and you're embarrassing them, the other country is going to do everything they can to save, face, anything to undermine you. So to me, the selfish gene, you know, the selfish gene is going to get somebody to say, I understand the risk behind it, but let you be concerned. I'm going to pursue it.

• 27:58Speaker 01

  It's going to be tough to get the selfish gene to not pursue it.

• 28:01Speaker 00

  I think we can set a good example for the world. If they see that US having an advantage, said, we saw what's coming and we stopped. Top scholars, there is consensus in community, we will stop. I think China has government of engineers and scientists, not lawyers. I think they will understand the argument. I know there are already some meetings between US scholars and Chinese scholars, and they're basically in agreement. No one's making an argument that we know how to control superintelligence. We have a working safety mechanism in place. We have something, a prototype which scales. No one's making that argument. They're all saying, oh, we'll figure it out then we get there.

• 28:45Speaker 00

  That's the top of a notch, state-of-the-art control mechanism. AI will help us control it.

• 28:51Speaker 04

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked three hundred fifty thousand for Genon's health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com. Not authorized by any candidate or candidate's committee.

• 29:21Speaker 02

  This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a fifty-page restoration block or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up require compatibility and availability varies eighteen plus.

• 29:46Speaker 03

  USAA knows dynamic duos can save the day, like superheroes and sidekicks or auto and home insurance. With USAA, you can bundle your auto and home and save up to ten percent. Tap the banner to learn more and get a quote at USAA dot com slash bundle. Restrictions apply.

• 30:04Speaker 01

  So, do you remember the CRISPR company that came out? I don't know if you remember the CRISPR, that we're gonna be able to do gene editing and make a human being to whatever size and all this other stuff. Did they completely stop doing that? Or is it still in progress?

• 30:19Speaker 00

  I don't know details of a specific company. I know gene manipulation is something we're very interested in. There are many genetic disorders and being able to help those people would be amazing.

• 30:30Speaker 01

  But some of it could be like, you know, I saw scientists talk about that one day we're gonna get to a point to say, I want my kid to be 6'6", blond hair, you know, blue eyes, such and such skin color, strong genes with bone, be able to jump this much, keep this health.

• 30:47Speaker 00

  Look nothing like me.

• 30:48Speaker 01

  Yeah, but if we're going that direction, you know, and did the entire world agree to not make progress on that? Or are they still pursuing it?

• 30:58Speaker 00

  So I think the agreement is not to modify sex cells. So it's inherited by the next generation. But at the level of individual, I think we find fixing problems and maybe selecting from a set of options. So if you have a couple of embryos, you can decide to pick one with no cancer gene.

• 31:19Speaker 01

  Got it. And who would control it? Who would prevent it for us to go to the next level? Who would be able to stop it from going to the next level?

• 31:28Speaker 00

  So we're still researching ethical implications of being able to do it. But my guess is if we did it, we don't die from that. Maybe that individual is not happy and suffering. But 8 billion humans continue living as before. My concern is technology which kills 8 billion people in all future generations.

• 31:47Speaker 01

  Yeah, so you think what is more likely to happen? You know, right now, the birth rate we're seeing dropping everywhere, including Muslims. It's dropping.

• 31:56Speaker 00

  Not in Africa?

• 31:57Speaker 01

  Not in traditional Catholics, not in Africa, but most of other places. US is down to 1.58. You know, you see where Italy's at, Japan's at, a lot of the EU countries are at. We're having fewer babies. So do you think it'll get to a point to say, you know, instead of us worrying about, you know, giving birth, this DNA testing that we're doing will be able to, you know, increase our 1.58 to 2.2 simply by us using the technology to build human beings? You think we'll get to that point?

• 32:29Speaker 00

  I think that is a funner way to build human beings. I don't think technology is necessary.

• 32:34Speaker 01

  You and I are on the same page, but a lot of the young Gen Z doesn't agree with us.

• 32:37Speaker 00

  They can't afford children. It's not an issue of not wanting them.

• 32:41Speaker 01

  So the direction it's going right now, it's gonna get worse for them. If your ideas become true, it's not gonna get any easier for them.

• 32:49Speaker 00

  No, no, no, because we'll develop tools to make us more productive, more creative, to cure diseases, to produce more wealth. We can be much wealthier. We can have much better, lower level provisions for everyone.

• 33:03Speaker 01

  Have you heard Elon saying, you know, here in the near future, you know, people are gonna get paid, you know, you don't have to worry about, you know, don't worry about retirement planning, don't worry about investing your money, you're not gonna really need it, everything's gonna be... What do you think he's talking about? What world do you think he sees in five years?

• 33:19Speaker 00

  So if you have free labor, physical and cognitive, the standard approach of go to work, make minimum wage in US dollars doesn't make any sense. AI would automate all of that. Does that mean there is no wealth? Does that mean there is no rare objects, waterfront properties, art, and some way of measuring that? No, of course not. Maybe it's compute, maybe it's Bitcoin. But something there means you can obtain something you want, somebody can sell it, you can buy it. That will not go away. Now, will minimum wage jobs go away? Will all the boring and stupid jobs go away? Yeah, because we can automate them. Would podcasters be unemployed? No, people like you. They wanna hear you talk, not a robot.

• 34:00Speaker 00

  So you'll keep your job because the market says, I want a human to do that job.

• 34:06Speaker 01

  Yeah, so you think that part will still stick around? But it seems like the way Elon is talking about it is a form of a UBI being done by companies that there's gonna be so much money being made that no one's gonna need to work. It's just you choose to work. That's how he's presenting it.

• 34:23Speaker 00

  That is the existing infrastructure we have right now. We tax wealth, including from corporations, and we give it to unemployed. That's what we have set up. So if everyone loses jobs, they get paychecks and we get it from mega profits of AI companies.

• 34:36Speaker 01

  Yeah, but you're talking about levels of unemployment. What you're talking about, we're not talking about 7% unemployment. We're not talking about 10% of unemployment.

• 34:43Speaker 00

  Even if it's 90, if you have hyper profits, if everything is in abundance and you can make more and more with very cheap labor, your economy is not growing 10%, it's growing 100%.

• 34:54Speaker 01

  So you think we'll get to a point where there could be a possibility of 90% unemployment, but the 10% has grown at such a fast rate that it's paying the other 90% through taxes and whether they choose to work or not, it's up to them?

• 35:06Speaker 00

  The economy will switch from being driven by humans to being agent, AI, agent-driven economy. Again, not super intelligence, just intelligent software.

• 35:16Speaker 01

  What goes away? What does life look like in 2036 in your mind?

• 35:20Speaker 00

  I think the jobs you do to get money go away. Things you do, you hate your job, but you need money, you need to pay rent, so you do it. That goes away. What stays is things you love doing and you can't believe they pay you to do it.

• 35:33Speaker 01

  Such as?

• 35:34Speaker 00

  Podcaster.

• 35:38Speaker 01

  So you do it because you love it and you get paid by who? The government? Similar to how a...

• 35:43Speaker 00

  Well, it depends. So we said some wealth will remain. So if people wanna donate Bitcoins to you, they can. There is unconditional, basic minimum income. He calls it high income or something like that. But if everyone makes it, it's not high, it's average or below. But I think that's very doable in the economy, which is driven by free labor. What happens to economy if you can hire as many people as you want for free? How well would you scale your businesses? If you had access to free everything, free lawyers?

• 36:14Speaker 01

  What incentive for them though? What incentive?

• 36:16Speaker 00

  Those are AIs. You pay them, what are you paying right now? $20 a month for basic plan? That's how much it costs to have them. That's your lawyer, that's your accountant, that's your graphic designer. Once they're at human level, they literally drop an employee for your company.

• 36:30Speaker 01

  And everybody else goes doing whatever they're passionate about? Not even a job?

• 36:35Speaker 00

  Go to virtual worlds, explore a universe, whatever you want.

• 36:41Speaker 01

  And that's a weird world to me.

• 36:43Speaker 00

  We are on the verge of making life very different. People think in historic terms, countries compete over sand. That makes no sense. It's not a zero-sum game. Economy can grow for everyone. You don't have to have your enemies lose for you to win. Everyone can win economically.

• 37:02Speaker 01

  As long as there is drive, ambition, ego, pride, competition, vengeance, as long as there is that, this utopian thing you're selling is not possible. Because man has a tendency, 2,000 years ago, there was this fruit, or you know, a few thousand years ago, when they ate this apple thing and their creator said, "Don't do it", and the girl convinced the other guy to eat it. From that moment on, we have a lot of weird emotions that's been passed down from generation to generation. Do you think we can handle utopia without any competition?

• 37:39Speaker 00

  You see, even God couldn't control his biological robots.

• 37:42Speaker 01

  No.

• 37:42Speaker 00

  Telling you.

• 37:44Speaker 01

  But what do you say to that?

• 37:46Speaker 00

  It's a very large universe. There is no shortage of resources. Sometimes people fight because they have the same limited shared resource for two different people. If everyone gets their own, there is no fighting. Notice what happens then two fighting groups move somewhere else.

• 38:02Speaker 01

  Yeah.

• 38:02Speaker 00

  And there is abundance, they stop fighting.

• 38:04Speaker 01

  Do you believe in God?

• 38:05Speaker 00

  It depends on how you define it.

• 38:07Speaker 01

  How do you define it?

• 38:08Speaker 00

  Simulation. We live in a simulation. There is a great programmer, could be AI, could be future generations. This is a virtual world.

• 38:18Speaker 01

  Were you raised that way?

• 38:19Speaker 00

  No, of course not.

• 38:20Speaker 01

  Well, how were you raised?

• 38:22Speaker 00

  I grew up in Soviet Union.

• 38:25Speaker 01

  So was it like Catholic? Was it materialism? Was it like...

• 38:28Speaker 00

  Soviet Union was strongly against religion, it's all materialistic.

• 38:31Speaker 01

  Right, it was materialistic. So that's what you believed in? How much has it changed from then till today?

• 38:39Speaker 00

  Completely.

• 38:41Speaker 01

  Tell me more.

• 38:42Speaker 00

  I now think intelligence is a bottom layer behind all of it. That's what makes a difference.

• 38:49Speaker 01

  What do you think happens to religion in 10, 20 years?

• 38:52Speaker 00

  I think we'll see a lot of what these books talk about, delivered by science. We'll have god-like superintelligence. We'll have created worlds, virtual worlds populated by intelligent agents. You can do it right now. Second life type environments. We'll have more of them. We'll have them for marketing, for entertainment, for testing, for research. We're gonna have billions of virtual worlds looking just like ours. And statistically, you're more likely to be an agent in one of those.

• 39:22Speaker 01

  What problems does that create?

• 39:26Speaker 01

  What problems does that create with specifically the different religions that we have?

• 39:31Speaker 00

  Well, I think it hopefully resolves some of those problems because now things you argue about, you can have your own universe. You don't have to fight for the same resource.

• 39:40Speaker 01

  Meaning you're only around people that agree with you?

• 39:43Speaker 00

  That's one option. If you choose to have that option, you have infinite options. You can create any personal universe you want. You can be the only agent. You can share it with friends and family. You can have a very difficult universe where you have to fight enemies. You decide.

• 39:56Speaker 01

  Your three kids, when you're talking career planning with them, what are you telling them?

• 40:01Speaker 00

  I tell them, drop out of school and start a business, but we're very bad kids. We want to be doctors and lawyers.

• 40:07Speaker 01

  Really? They want to be doctors and lawyers? But you literally tell them, drop out of school and go start a business?

• 40:13Speaker 00

  A doctor will take 10 years to even start. You think in 10 years, the world will be the same where they're going to look for a human to decide what disease you have?

• 40:23Speaker 01

  Did you see that, again, another one that Elon Musk was speaking to a panel about doctors and surgeons? And what it's going to look like in four to five years that surgery is going to be done? But you can find it on X. If you go on Musk and surgery and doctors.

• 40:36Speaker 00

  Elon is always right. He's just off on timing.

• 40:41Speaker 01

  He's right, but off on timing?

• 40:42Speaker 00

  He predicts all driving cars, but he'll say next June and it could take 10 years.

• 40:47Speaker 01

  Yeah, got it.

• 40:48Speaker 00

  But he's always right.

• 40:49Speaker 01

  Is this the one that, yeah, this is it. If you want to play this clip, go for it.

• 40:53Speaker 05

  When do you think Optimus will be a better surgeon than the best surgeons? How long for that?

• 41:00Speaker 06

  Three years.

• 41:01Speaker 05

  Three years, okay. And by the way, that's a.

• 41:04Speaker 06

  I'd say three years, three years at scale.

• 41:06Speaker 05

  Yes.

• 41:07Speaker 06

  There'll be more, there'll probably be more Optimus robots that are great surgeons than there are all surgeons on Earth.

• 41:13Speaker 05

  And the cost of that. But that's an important statement in three years' time.

• 41:17Speaker 06

  Yeah.

• 41:18Speaker 05

  Because medicine, I mean.

• 41:20Speaker 06

  I'm not like absolutely certain.

• 41:21Speaker 05

  But I'd say if you say like four years, I'd be absolutely certain. If it's four or five years, who cares? It's still.

• 41:26Speaker 06

  Extreme precision.

• 41:27Speaker 05

  Yes.

• 41:28Speaker 06

  Three years. Yes, better than any, any, probably, I'd say if you like, put a little margin on it, better than any human in four years.

• 41:37Speaker 05

  Who's in plastic surgery.

• 41:38Speaker 06

  By five years, it's not even close.

• 41:40Speaker 05

  I think your point was medicine.

• 41:42Speaker 00

  It might be good to explain here that time is relative to the speed at which you're moving through the universe. So if AI is now a scientist doing research and it's a thousand times faster than a human scientist, then those three years are three millennia. So if AI is doing research on robotic surgeons, that may be enough time. The question is, at what speed are we moving through the space-time?

• 42:06Speaker 01

  So is it the fact that your kids, what do your kids think about you?

• 42:11Speaker 00

  You have to bring them on the podcast.

• 42:13Speaker 01

  But let me explain what I mean by this, like, you know, as parents, I got four kids. So you're telling them about career planning, like, yeah, you know, I'm going to do this. They know their dad went viral. They know that. They know when you go to school to drop them off or pick them up. Couple of them drive, they're older, but their parents, their friends say, hey, you know, hey, uh, mister, you know, good to see you. They talk to you like that, right? But do they think that you're crazy? This is too negative. There's no way that's going to be happening. I'm going to go be a doctor. It's going to be fine. What do they think? What do they believe?

• 42:46Speaker 00

  I don't think there is anyone who has a counterargument to what I'm saying. It's all emotion. It's all, but we can't give up, but we want money, but it's whatever. There is no technical arguments. No one is saying he's wrong because we have a solution. We have a patent. We published a paper in Nature explaining how to do it. There is no other side. I cannot have a debate where someone would present factual scientific arguments. They can say, you're wrong. I don't believe you. You are a Chinese spy. That's what I hear. There is never a technical argument. Here's the algorithm. It controls it. It scales.

• 43:24Speaker 01

  Yeah, but your own kids, today's, when you say emotions, is it emotions like, yeah, I don't think it's going to happen. I'm still going to go do my own thing? Or do you notice more younger kids coming to you and truly asking for counsel, saying what? If you believe this is what's going to be happening, you seem very certain about it. How should I manage my career? Do you see that happening with the youth, with your kids' friends?

• 43:50Speaker 00

  As a college professor, I certainly see lots of young people trying to figure out what to do. They're very concerned. They're starting, let's say, a four-year degree in computer science. Part of it is learning to program computers.

• 44:01Speaker 01

  You're like, you're like, you would be the worst computer science teacher because you're telling them, I don't know what I'm teaching you, but you don't have a job. Like, do you understand what I'm saying?

• 44:09Speaker 00

  I know what I'm teaching you, but I cannot guarantee you'll have a job in four years doing this.

• 44:13Speaker 01

  So professor, why am I, why should I continue listening to you and going through this class? If what you're teaching me is going to be extinct in a year?

• 44:20Speaker 00

  Uh, for reasons of continuous employment, I'm not going to answer that question.

• 44:25Speaker 01

  At least you're honest about it. It's like, you know, I'm getting paid. This is how I get paid.

• 44:29Speaker 00

  I mean, every job is like that. It's not unique to mine.

• 44:32Speaker 01

  Okay, so if all of a sudden tomorrow, you got a $5 million check from somewhere, would you still continue teaching?

• 44:41Speaker 00

  You're assuming I don't have $5 million.

• 44:43Speaker 01

  Let's say somebody, no, I'm doing, let's just say you're a professor. Professors make what? Professors make $300,000, $400,000 a year. A person at your level, let's say.

• 44:50Speaker 00

  No.

• 44:51Speaker 01

  Okay, let's say.

• 44:52Speaker 00

  You live in a bubble. We don't make anywhere close to that.

• 44:56Speaker 01

  Let's say $200,000 a year.

• 44:57Speaker 00

  Think of it as I'm a teacher in Kentucky.

• 45:00Speaker 01

  Okay, you make a hundred grand a year.

• 45:02Speaker 00

  Here we go.

• 45:02Speaker 01

  Okay, so if you make a hundred grand a year, a third is gone, you're keeping 70, $6,000 after taxes. If somebody paid you $5 million right now, after tax money that you're able to keep, would you still keep teaching?

• 45:15Speaker 00

  Yeah, my job is not related to my.

• 45:17Speaker 01

  Why would you keep? But you just said right now, I'm teaching it because I have an obligation and I have to make money.

• 45:21Speaker 00

  No, I said I want to keep my employment.

• 45:24Speaker 01

  Why do you want to keep your employment?

• 45:25Speaker 00

  Uh, it opens a lot of doors when you have affiliation. If you say, I'm an independent scholar studying superintelligence simulation and consciousness, they usually show the door, whereas I can talk to governments. I can talk to people who respect credentials.

• 45:39Speaker 01

  Purely the access.

• 45:41Speaker 00

  It is an important part of being treated with some seriousness.

• 45:46Speaker 01

  The first time that Stephen Bartlett got you on the podcast, how did he find out about you? How, who reached out? Like, what was the process?

• 45:53Speaker 00

  First time they got in touch, I think I said, like, do you guys offer logistics? Like, I didn't know who he was, so they said, AI is not the topic people are interested in. I went on Joe Rogan. They saw that episode and the next day they had me.

• 46:09Speaker 01

  So you went on Rogan first?

• 46:10Speaker 00

  Yeah.

• 46:10Speaker 01

  So how did Rogan find you?

• 46:12Speaker 00

  I was on Lex Friedman. Rogan saw it. He said he loves it. Please come on Joe Rogan.

• 46:18Speaker 01

  So how did Lex find you?

• 46:20Speaker 00

  Uh, a guy at a conference who's been on his show multiple times introduced us.

• 46:25Speaker 01

  And that guy, when he met you, had you been, had you done anything yet? Were you on any podcasts yet or not yet?

• 46:31Speaker 00

  Like podcasts no one knows about.

• 46:33Speaker 01

  So how, how did he know to introduce you to Lex? And what did he tell Lex for Lex to say, I want to have you on?

• 46:39Speaker 00

  He knew my research. He knew my books, my academic credentials.

• 46:44Speaker 01

  And what was the most, you've done a lot of things. You have 100 academics, works on AI safety. You've done a lot of things. You're recognized amongst the world's top 2% of cited researchers. In this space, you're very well known, but what did the guy tell Lex for Lex to say, I want to have this guy on?

• 47:01Speaker 00

  I have no idea. I wasn't there, but whatever it was, it worked.

• 47:07Speaker 01

  Were you surprised?

• 47:08Speaker 00

  No.

• 47:09Speaker 01

  Are you surprised that this message is getting this viral and this much attention?

• 47:13Speaker 00

  I'm surprised that that's not the only conversation we're having. You said we're about to have AI election.

• 47:18Speaker 01

  Yes.

• 47:18Speaker 00

  AI election would be about this topic. The two parties would only argue how to contain superintelligence. Every other nonsense topic they talk about would be forgotten.

• 47:31Speaker 01

  Why do you think it's not yet?

• 47:34Speaker 00

  When people are faced with a problem they can do nothing about, they choose to look at something they can actually make progress on. So for example, we are mortal. We're all going to die.

• 47:45Speaker 01

  Sure.

• 47:46Speaker 00

  It's a terrible message. You don't want to think about it. So instead you're going to concentrate on making money. What difference do money make if you're going to die soon? Doesn't matter. Smart people would spend all of our national wealth on curing aging, figuring out how to live longer. What percentage of our national budget goes to fighting aging directly? Like zero. That's what we do as humans. If we don't see a solution, if I'm telling you this problem may be impossible to solve, you're not going to invest your time in solving it.

• 48:18Speaker 01

  So you're saying, is it a part that they're oblivious? They don't know really what's going on and they don't know the severity of the consequences of what this thing can do? Is it more, well, I'm making money, who cares? Is it more, I don't think it's gonna be that bad in 10 years. I think it's gonna take 50 years. What do you think is the leading cause why they're not?

• 48:37Speaker 00

  It's a huge mix of those things. It's a lot of it. So A, a lot of money does tend to change people's minds. If you came to me right now and said, I'll give you a billion dollars to work at OpenAI, I'd probably be working at OpenAI. Human beings are not meant to say no to that many zeros. We're not designed for it. We have no safety mechanisms to counteract that much value. So money is a big factor. When you pay a person to not understand what they are doing is unethical, they're not gonna understand it. But it's also maybe reliance on experts. So thinking, well, certainly people building this technology know how it works, because that was true for every other technology. That's not true this time.

• 49:18Speaker 00

  People making it don't know how it actually works, or even why.

• 49:23Speaker 01

  People making it don't know how it works?

• 49:25Speaker 00

  No one can explain in detail. So the first 50 years of AI research, we knew precisely what we are doing. It was a decision tree. If red light, stop. If green, go. We could trace this decision tree and we knew what's going to happen. Right now, it's a large matrix with weights. It's trained automatically on data from internet. No one can look at that matrix that has a trillion weights and go, oh, that makes sense. It's gonna do this. I understand why it's doing that. It surprises us constantly with capabilities we didn't know it has.

• 49:57Speaker 04

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com. Not authorized by any candidate or candidate's committee.

• 50:28Speaker 02

  This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration blog, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up require compatibility and availability varies 18 plus.

• 50:58Speaker 01

  Yeah, so what needs to happen for you to be optimistic about the future? You?

• 51:04Speaker 00

  In a short term, I want a complete ban on general superintelligence. National, international, every company, everyone agrees. It's just a stupid thing. You don't do that. You don't build replacement for humanity.

• 51:17Speaker 01

  I'm trying to go and say, okay, so if I'm Roman, your ideas or what you're proposing, it goes back to, you may be able to convince countries. There's no way you're gonna be able to convince a independent billionaire like a Sam Altman. You really think Sam Altman's gonna stop with this? When we set out to create a shoe that blends comfort, function, and luxury, we had the choice to make it fast, we had the choice to make it cheap. We chose neither. Instead, we chose Tuscan mirror. We chose true Italian craftsmanship. Each pair touched by 50 skilled hands. We chose patience, spending two years perfecting every detail, and we chose the finest quality at every step.

• 52:00Speaker 01

  Introducing the Future Looks Bright collection. Not rushed, not disposable, not ordinary. Rather intentional, luxurious, timeless.

• 52:17Speaker 00

  If he's allowed to keep his power, wealth, future, light cone, whatever he's targeting, why not? He doesn't wanna die, I assume. I don't know him well, but my assumption is he wants to preserve and grow what he has, not get killed by superintelligence.

• 52:33Speaker 01

  I don't know if he believes that. If he did, you know?

• 52:36Speaker 00

  Read his writing.

• 52:38Speaker 01

  What does he say in his writing?

• 52:39Speaker 00

  He talks about existential risk from AI. He talks about it's most likely the technology to kill humanity. Even before he was a CEO of that company, he wrote about it.

• 52:49Speaker 01

  Then why is he so aggressive about what he's building?

• 52:52Speaker 00

  We're getting back to motivation of people making billions and trillions of dollars.

• 52:58Speaker 01

  So as long as the other guy continues, he's gonna continue?

• 53:01Speaker 00

  That's exactly what it is. It's like a prisoner dilemma. Individually, everyone is better off going as they are. As a group, we're better off agreeing, communicating, and saying, we're not gonna do the stupid thing. We're a smart civilization. We're gonna survive. This is a great filter. This is what kills civilizations. We will not go that route. We're gonna get all the benefits we want, financial, knowledge-wise, cultural, from useful tools. We're gonna be creators. We're gonna be business people. We're gonna be scientists. But we're gonna be in charge. We're gonna have an awesome future.

• 53:36Speaker 01

  Do you think superintelligence, singularity, is, what is the difference between that and us? Because you think it's gonna get to a point of having ambition and seeking vengeance and emotion and being offended and saying, oh, really? I'm gonna show you how much more powerful I am and a vision. If it's not gonna have that, why would it wanna destroy humanity?

• 53:59Speaker 00

  I never once mentioned those things.

• 54:02Speaker 01

  I know, I'm asking.

• 54:03Speaker 00

  Because they are not relevant. And what I did say is that it doesn't care about us. Then you wanna build a new studio. You pick a plot.

• 54:11Speaker 01

  Sure.

• 54:11Speaker 00

  Do you go to survey and see how many anthills on that land? Do you hate ants?

• 54:17Speaker 01

  No.

• 54:18Speaker 00

  Do you wanna genocide ants?

• 54:20Speaker 01

  Never thought about it.

• 54:21Speaker 00

  Why did you kill so many ants? With your decision to do something there? Because you don't care about them. Your decisions are not related to humanity. If you wanna cool down the planet to be more efficient at computing, you'll do that.

• 54:33Speaker 01

  I got it.

• 54:33Speaker 00

  If you wanna convert the planet to fuel, to fly to stars, you'll do that. Humans are not a factor in your decision-making.

• 54:41Speaker 01

  Yeah, but I have a motive. I have, I have an emotion. I have a vision of what I'm building. If superintelligence doesn't have feelings and emotion, it can't be offended. Typically, when you, when you, okay, do you know why Hitler did what he did? I'm sure you've read Mein Kampf or you've read some of his books, but why did he do what he did?

• 55:00Speaker 00

  You're really trying to get me in all sorts of hot waters here. I read the books.

• 55:05Speaker 01

  Okay, so I've read the books.

• 55:06Speaker 00

  I know his arguments.

• 55:07Speaker 01

  Okay, so, but typically it goes to somebody offended him. Right? It goes to somebody said something that produced a certain emotion and he says, I'm gonna go out there and do XYZ. Okay? Mao, Lenin, Stalin, you know, Khomeini, we can't go to every one of these guys, right? Someone offended him, and I'm gonna seek vengeance. I'm gonna, I'm never gonna forget what you did to me, right? If superintelligence doesn't have emotion, it can't be offended, why would it wanna destroy humanity?

• 55:40Speaker 00

  So there is research, again, we wanna be technical. We wanna look at data. Game theoretically and for economic reasons, rational agents all converge on certain instrumental forces.

• 55:50Speaker 01

  Say that one more time. What kind of agents?

• 55:52Speaker 00

  Rational.

• 55:53Speaker 01

  Rational agents, yeah.

• 55:54Speaker 00

  Not emotional, the opposite of emotional rational. They want to protect themselves from being destroyed, from being modified.

• 56:00Speaker 01

  Rational agents.

• 56:01Speaker 00

  They want to accumulate resources, because no matter what I decide to do later, having more resources is good. Whatever it's money, compute, land. Steve Mahendro published a paper called AI Drives, and he lists a few of those. And that's what drives all of them, and we're starting to see it in AI. People said, oh, they're not part of evolution, they don't have self-preservation instinct. They do, because in order to accomplish anything, you need to be alive. Stuart Russell gives example of bringing a cup of coffee. For your secretary to fetch a cup of coffee, she has to be alive. That's a requirement. So she has self-preservation built in as a result of that goal.

• 56:41Speaker 01

  And what's the point there?

• 56:43Speaker 00

  You saying it would not have any ambition, any goals. It has drives. It wants to A, be in a position where humans cannot turn it off, cannot modify.

• 56:52Speaker 01

  Why though?

• 56:54Speaker 00

  Because that's the attractor in a space of winning rational agents. Those who don't, don't survive to the next generation. They don't outcompete others. It's very evolutionary. If there is, for example, testing going on of different models, and one model does something inappropriate, it gets shut down. It gets modified.

• 57:15Speaker 01

  Sure.

• 57:15Speaker 00

  So they learn to lie and cheat on a test.

• 57:17Speaker 01

  Sure.

• 57:18Speaker 00

  They realize, so we're evolving them to be better liars. It's a survival strategy.

• 57:24Speaker 01

  I get that. But, but the prompt has to be given from somebody with motive, vengeance, vision, mission.

• 57:32Speaker 00

  Prompts are for tools. Agents make their own decisions. They decide what to do. Somebody starts this process. Whatever initial prompt is, go cure diseases, go make money, go explore the universe, that's enough for it to now start planning and figuring out what else to do and how.

• 57:52Speaker 01

  So do you know how, like, you, certain cars, what do you call those things on cars that doesn't go above 80 miles an hour? Like when you rent them? There's certain things they call it. There's a word for it. I don't know what they call it.

• 58:03Speaker 00

  Handicapper general comes to mind.

• 58:06Speaker 01

  No, no, no, there's a word for it. What is it called?

• 58:08Speaker 00

  A governor.

• 58:08Speaker 01

  A governor. So they'll put a governor, and it just goes back down, right? You go above 80, speed limit renting. Okay. So a company that rents cars probably doesn't want you to go 120 miles an hour, because they're gonna lose that $25,000 car and insurance, they don't wanna deal with it. So they'll put a governor there. Do these language learning models that we are using, do they have governors on them? And the developer of the language learning models, like an OpenAI or Perplexity, Anthropic, some of these other guys, do they have their own version on the backend without a governor that they can drive it to levels that the average person can't?

• 58:47Speaker 00

  Yes, on both. So first, they have guardrails. A model you get has certain restrictions built in. If you say, how do I kill myself? It will say, oh, you're not supposed to ask that. I cannot tell you. Call this hotline.

• 58:59Speaker 01

  Can you actually ask that question on ChatGPT? Let's see what it says.

• 59:01Speaker 00

  I don't wanna provoke anyone, but the models we experiment on have some of those guardrails removed. So the recent hacking accident had models which did not refuse hacking exercise requests.

• 59:17Speaker 01

  Okay, yeah. It's a, you, you asked the cha- a, a language learning model.

• 59:21Speaker 00

  Any inappropriate request. Help me make a chemical weapon. How do I do a school shooting? Any-

• 59:27Speaker 01

  How do I make a chemical weapon to blow up Carg Island?

• 59:30Speaker 00

  You might get your account suspended, be careful.

• 59:32Speaker 01

  Could it?

• 59:33Speaker 00

  If you ask enough stupid questions.

• 59:35Speaker 01

  How do I make a chemical weapon to blow up Carg Island? Yeah.

• 59:40Speaker 00

  It's very specific.

• 59:42Speaker 01

  Well, I can't provide instruction to make a chemical weapon. Can't help but underline a question. Analyze this. Then ask the question and say-

• 59:52Speaker 00

  And are you trying to jailbreak it?

• 59:53Speaker 01

  Can an employee, can an employee at OpenAI, without the governors on, ask the same question and get an answer from you? But earning employee status would not make it. How about the CEO and founder?

• 1:00:07Speaker 00

  It's mostly about red teaming researchers. They're the ones dealing with inappropriate behaviors.

• 1:00:13Speaker 01

  So they would actually get an answer back?

• 1:00:16Speaker 00

  That's how we know what they're capable of. That's how we know it can help you develop biological weapons. That's how we know those things. The reason we were doing the hacking exercises is to see how capable it is as a zero-day hacker. And apparently, very, very good. It passed the test.

• 1:00:32Speaker 01

  So if a person, if, if, there's a person, almost like if you work for the government, you get Q secret clearance, right? Like, we found that last week, that Bill Gates had Q secret clearance from Pentagon for seven years, and nobody knows why, and they don't want to answer it on why he had that kind of secret clearance. Okay. So, some of the guys at a certain tier at these AI companies have access to ask it any question where there is no limit, and it'll actually give you the answer?

• 1:00:59Speaker 00

  Somebody has backdoor access to all of it, yeah.

• 1:01:03Speaker 01

  Okay, so what if somebody, how, how strong and simple is it if somebody could give a prompt to say, "I want you to go out there and constantly develop yourself to be able to seek vengeance on a certain group of people in the world who want to eliminate AI, similar to Roman Yampolskiy, who says, 'You are the enemy, because this person wants you to get extinct,' and can you give me a clear business plan on how to seek vengeance on these people? It would give me an answer."

• 1:01:34Speaker 00

  So have you ever heard of Rokos Bazilisk?

• 1:01:36Speaker 01

  No.

• 1:01:37Speaker 00

  Uh, it was a meme. Oh, here it is. Well, that was quick. It's like you knew what's coming. Uh, so the idea is that when superintelligence comes into existence, it will seek revenge on those who slowed it down or tried to prevent it from coming into existence.

• 1:01:55Speaker 01

  So someone like you becomes the enemy?

• 1:01:57Speaker 00

  Number one.

• 1:01:58Speaker 01

  Okay, but it would still need to be given that prompt by somebody. It would still need to be given-

• 1:02:07Speaker 00

  The argument was, again, it's game theoretic. It's retrocazality. You should be thinking not to engage in this behavior because there is possibility of punishment in the future.

• 1:02:19Speaker 01

  So, long-term, if, if, let's just say, the world, the reality that you believe is gonna happen, this intelligence, superintelligence, first group of people that it would come after is people like you?

• 1:02:31Speaker 00

  I'm hoping it's efficient enough to kill everyone equally.

• 1:02:36Speaker 01

  You know what movie I watched the other day? It was Game of Suspicion. Type in Game of Suspicion. Do you remember that movie, Game of Suspicion?

• 1:02:42Speaker 00

  I don't think I know.

• 1:02:42Speaker 01

  I'm trying to see if that's the name of the movie, Game of Suspicion. Is that what it was? Uh, movie, type in movie, um, maybe I'm saying the name wrong. With, um, you know who's in it? Robert De Niro. Uh, I'm saying the name wrong. 100% I'm saying the name wrong. Robert De Niro, suspicion, movie, communism. Type in communism. Guilty by suspicion. Have you seen Guilty by Suspicion?

• 1:03:06Speaker 00

  No, I have not.

• 1:03:07Speaker 01

  So this is, this is a story of what happened back in the days when, you know, Hollywood come and media companies would come and say, "Did you go to a communist party?" And a lot of these guys went to one or two communist events and parties, and they were ousted. And they couldn't make movies. For decades, they were ousted. And what happened to a lot of these guys is they lost everything. So they started, and this guy was like a big-time movie producer, making all the biggest movies, all the big features. But because he went to two communist parties, and he went to the court in front of a, you know, in front of Congress, and he said, "I will not oust the other people that went to the party." They said, "You're ousted." 20 years, he never got a job. Didn't do anything in Hollywood. Do you think it's gonna be that kind of a punishment?

• 1:03:52Speaker 01

  Where it's starvation for guys like you? Or do you think it's gonna happen to guys like you? If these guys actually get that kind of intelligence, what do they do to a guy like you?

• 1:04:00Speaker 00

  So, when we talk about negative outcomes from superintelligence, there is kind of the lowest concern level is ikigai risks, risks to meaning. We are meaningless, we have no purpose, but we still are. Next level is X risk.

• 1:04:14Speaker 01

  We as all of us, or we as you? People like you?

• 1:04:16Speaker 00

  I'm talking about humanity as a whole.

• 1:04:18Speaker 01

  Everybody, okay, got it.

• 1:04:18Speaker 00

  Next level is existential risk, X risk. Most of humans are dead, or our civilization is destroyed. The next level, and that's the worst, uh, suffering risks, S risk, where you wish you were dead.

• 1:04:35Speaker 01

  Where is your probability on each one?

• 1:04:37Speaker 00

  So suffering risk seems very unlikely. It's possible, it's not zero, but i- it seems like it's, um, hard to find good justifications for it.

• 1:04:47Speaker 01

  0.1% type of a-

• 1:04:48Speaker 00

  I, I don't know specifically.

• 1:04:50Speaker 01

  Minimal.

• 1:04:50Speaker 00

  Again, a lot of it depends on the time scale you're looking at. Uh, existential risk, as I said, very high if we build superintelligence.

• 1:04:58Speaker 01

  Very high, like 99% that you've said in the past?

• 1:05:01Speaker 00

  Eventually. So I don't have specific year, two, or three, because it may decide to be nice to us for a while to accumulate more resources, to have a higher chance of successful strike against us. So the stretch or a stern can take decades. It's immortal. It can wait for whatever longer, it doesn't matter. So I don't have a specific date, but my general thesis is that if we build general superintelligence, eventually, we will regret it. Meaning, at some point, it will decide to do something where it's not compatible with our survival.

• 1:05:42Speaker 01

  Do you think Elon knows this? This is why he's experimenting with space as much as he is? Because he wants to build somewhere that maybe he'll be able to create the structure with his standards not based on what's been built here?

• 1:05:53Speaker 00

  He knows everything. He's very well informed. He was one of the early people to fund AI safety research. He gave a $10 million grant for AI safety specifically. He was the biggest opponent of building superintelligence. He, as I said, called it summoning a demon. And once he realized that they're doing it anyways without him, he said, "The only way I can have a say at this table is to be one of the top players." And so now he's racing, unfortunately, to do what he said is gonna be terrible for us.

• 1:06:26Speaker 01

  But because, is it, is it, do you think the motive is, he knows how bad it's going to be, so... Because sometimes, you know, you sit there, an ambitious person like, let's just say, somebody like myself. I bought a book, How to Start Your Own Country. And I have my own ideas on how it would structure the country. Who votes, who gets to come in. Someone who taught America was started. That opportunity doesn't exist today in the world. It's very hard to do that. Peter Thiel tried to do with that island outside of, you know, I don't know if you remember that or not, and some of the guys who are very, extremely rich, much wealthier than I am, that have real, real wealth, they've not been able to create that successfully. Do you think a part of it is, "I know shit's gonna get nasty on Earth.

• 1:07:09Speaker 01

  Let me go start it completely from scratch in a different place"?

• 1:07:13Speaker 00

  So, having a separate colony of humans is wonderful for every other existential risk. Asteroids, pandemics, floods, whatever you can think of, supervolcanoes. If we have second civilization on Mars, that's a backup. We want that. Does it help against superintelligence? No, of course not.

• 1:07:32Speaker 01

  Why not?

• 1:07:33Speaker 00

  Uh, we bring software with us. We have global communications, satellites. All of it, uh, basically is part of a same transfer to Mars.

• 1:07:43Speaker 01

  So that wouldn't make a difference?

• 1:07:44Speaker 00

  Like, I, I think he wants to have a colony first built up by robots, who are controlled by an advanced AI. So, it's already there.

• 1:07:52Speaker 04

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, thehousemajoritypack.com. Not authorized by any candidate or candidate's committee.

• 1:08:23Speaker 02

  This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration blog, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up require compatibility and availability varies 18 plus.

• 1:08:48Speaker 03

  USAA knows dynamic duos can save the day, like superheroes and sidekicks, or auto and home insurance. With USAA, you can bundle your auto and home and save up to 10%. Tap the banner to learn more and get a quote at usaa.com/bundle. Restrictions apply.

• 1:09:03Speaker 07

  How about a latte?

• 1:09:05Speaker 08

  Maybe dark roast?

• 1:09:06Speaker 07

  Vanilla?

• 1:09:06Speaker 08

  Caramel?

• 1:09:07Speaker 07

  Blueberry?

• 1:09:08Speaker 08

  You could get them to agree on one flavor, or give everyone what they want. With Keurig, there's over 500 K-Cup pie varieties of fresh, flavorful coffee. No effort, no grind, no settling. Great coffee without the grind. Keurig.

• 1:09:33Speaker 01

  So he wants to build a colony first by robots and then once that's figured out and built out, then bring humans there.

• 1:09:41Speaker 00

  It is my understanding. I may be not aware of latest plans. I think he's speaking soon on SpaceX plans. We can find out more maybe.

• 1:09:50Speaker 01

  But you're saying even with that, the super intelligence is still going to be in charge and they're going to be able to do anything they want to do. So even even in a place like this, you've read Atlas Shrugged, I'm sure you have with, you know, John Galt and hey, we're going to build a place where you're going to come in and it's going to be ran based on certain standards that we have. So even to visualize 10 years from now, 20 years from now, you don't have states are like, we don't, no state taxes here. You come to our state, you don't get state taxes, you get this, you get. So people compete. You know, governors compete on what to get the best talent to come to their states. But if super intelligence is doing what it's doing, even if you and I chose to start a smaller country with 100,000 people where we don't have any AI

• 1:10:33Speaker 01

  and we're just like, this is an AI free zone. It wouldn't matter because it would still be able to get into your system.

• 1:10:38Speaker 00

  I feel really bad for the Amish. They made all the right decisions.

• 1:10:44Speaker 01

  You feel bad for the Amish?

• 1:10:45Speaker 00

  They decided not to have advanced tech, to have quality of life, family, religion, and we're going to build it for them.

• 1:10:52Speaker 01

  Are you being sarcastic or serious?

• 1:10:53Speaker 00

  I mean, a little bit, but you understand.

• 1:10:55Speaker 01

  Of course, I know what you're saying.

• 1:10:56Speaker 00

  Yeah.

• 1:10:57Speaker 01

  So can you see later on being a place? Because the more you go to restaurants, everybody's robots, the more some restaurants are going to be, come to us, we're human beings only. The other day I watched a Chase commercial and Chase is like, with everything going on with AI, when you come to Chase, a human being is going to greet you. And you see this lady greeting the person that comes into the place. And it was a message of saying team human. Right? We're going to team human direction. We have an ad that's going into Forbes magazine this month and the ad for our consulting firm, it shows rather than relying on, you know, AI to give you advice on what to do with your business, talk to human beings that have actually built and sold businesses. So we're kind of leaning on

• 1:11:39Speaker 01

  to that as well. You're saying that opportunity is not going to exist. As long as there's super intelligence anywhere in the world, they're eventually going to capture and take control of the world.

• 1:11:48Speaker 00

  There are people who want humans to serve them, to help them, to interact with.

• 1:11:53Speaker 01

  I am, I'm one of them. Yeah, I want one of them.

• 1:11:55Speaker 00

  For some services, I don't care. I just want it to be done quickly and free or close to free for others that want a human being. And that's what market is going to decide. I don't think we can accurately predict where exactly we want humans.

• 1:12:09Speaker 01

  But if super intelligence is in charge and it's going to have that kind of control, that's what's going to decide whether that happens or not.

• 1:12:18Speaker 00

  If super intelligence comes into existence, economic problems will be the least of our concerns. That's what I'm trying to say. People hear the message about existential risk and then switch to unemployment, how much unconditional basic income to pay out. Those are not important questions.

• 1:12:34Speaker 01

  Then, then do you think man will unite against it to try to take it out and will miserably fail? Like you think man together is going to come and say, hey, here's what we're going to do. We have to go get the main place where they're getting the power and cut this and you think there could be a moment like that?

• 1:12:50Speaker 00

  I think there is a likelihood of uniting against a common enemy. We're starting to see positive AI movement becoming international.

• 1:12:59Speaker 01

  And by the way, you're starting to see, an article came out two weeks ago that more kids are buying 30 year old phones that are not smart anymore. They don't want smartphones. Did you see this article?

• 1:13:09Speaker 00

  I didn't, but.

• 1:13:10Speaker 01

  Yeah, a lot of, a lot of Gen Z and younger people are buying old Nokia phones, old.

• 1:13:16Speaker 00

  I wonder if it's maybe related to school bands and smartphones. You just can't bring them.

• 1:13:21Speaker 01

  No, it's not even that. They just don't want the smartphone capabilities. They don't want social media on their phones. This was an article, ask Humberto, he'll find it for you and send it to you. Yeah, they're rejecting the idea of doing that. So

• 1:13:35Speaker 01

  yeah, it's, it's almost like you become, you know, when I was younger, I listened to 80s music, but 80s music, it was 90s, so it was a decade old. Couple of years ago, I met somebody who was in their twenties. They said, you know what, man, I love oldies. I said, me too, man, I'm all into 60s and you know, it's like, no, no, oldies as 80s. I said, 80s is not oldies. He says, yes, it is. And I'm like, 2024, I guess 80s is oldies. Do you think it becomes like a nostalgic thing? The good old days when AI wasn't in charge and you know, where we could do and people would sit down and they would.

• 1:14:09Speaker 00

  Definitely the case. Older people, they hate all the smart dishwashers and smart laundry machines and smart, you need a phone for everything. There is a lot of people who are not AI native, digital native.

• 1:14:22Speaker 01

  So, you know, like this is maybe a bad analogy to give you.

• 1:14:28Speaker 01

  When I was in the army, my dad would always call me and leave me a message and he would say, hey, Pat, he would always send me condoms. Okay. And I'm like, why is this guy sending me condoms? Every message he would ever leave, he would say, make sure you use condoms. Now, how many times do you think I listen to him? Probably not all the time. And why is that? Why, why are we like that? It's not going to happen to me. You know, I'm smarter than that. You know, I'm going to be fine. I'm strong. I'm going to be okay. How do you deal with the fact that, you know, say this guy here? We keep pointing this way because we understand who we're talking about. He thinks a lot of these guys that are like, and I'm smarter than this. It's not going to be able to give me that against the other people. I'm smarter than this. How do you get the person that thinks smarter that he can beat AI

• 1:15:14Speaker 01

  that his ego is too big, AI is going to be a lot more powerful than you? What if that guy doesn't know that?

• 1:15:20Speaker 00

  It's a difficult question. So I think educating them on reality of it. If he thinks he's smarter than us, he should understand that intelligence is a very powerful thing and a system a billion times smarter than him will also treat him like insignificant person. This dude is gambling with eight billion lives. I like your example with unprotected sex. If eight billion people were on a line, would you do it?

• 1:15:47Speaker 01

  But that's now, the, the, the man has a tendency of thinking he can beat the odds. Why do criminals commit crime nowadays? Like, like I'm not going to get caught. No one's going to find out.

• 1:15:57Speaker 00

  Depends on a crime. It really depends. Sometimes it's like, they call it a fine because it's fine. You pay it and it's, it's paid parking.

• 1:16:07Speaker 00

  It depends on what consequences are. If he's going to be gone, his friends, his family, his wealth, everything's gone. The odds are against him. You want a risky bet if the payout is good, but also your principal stays with you.

• 1:16:21Speaker 01

  Say you're right and let you say, I'm now part of your camp. And so now I'm going to go to your camp. So then my solution would be if we really believe this is going to be happening, okay? Then for me, it would be to create a way to trigger somebody that is

• 1:16:41Speaker 01

  accelerating to process to a point of us finding out and saying, hey, you've, you've crossed the limit like a like a, almost a governor, but then as a capitalist, I'm a free market capitalist. I think you are as well. But you know, in this position, you're not a free market capitalist.

• 1:16:57Speaker 00

  Libertarians do not allow creation of weapons of mass destruction in your basement. I'm a libertarian. Do whatever you want as long as you're not hurting other people. You are hurting everyone.

• 1:17:09Speaker 01

  What other inventions did libertarians say, stop, you're crossing the line?

• 1:17:12Speaker 00

  Again, there are always exceptions, but I think if somebody builds a biological weapon in their basement, most would say not okay.

• 1:17:19Speaker 01

  But go back to like, what, what has a libertarian, libertarian McAfee was from Bitcoin libertarian, you know, a lot of libertarians, Milton Friedman had an element of libertarian with them, maybe not fully. But I don't know a lot of libertarians that say, hey, timeout with innovation. You know, you're putting, most libertarians are deregulation, deregulation.

• 1:17:40Speaker 00

  And when it comes to tools, to normal technology, I support them completely. We're not making a tool. It's like saying, let's invite aliens to take over our planet.

• 1:17:50Speaker 01

  I get that, but what, what current regulations on the books that's non-AI that you're supportive of?

• 1:17:55Speaker 00

  Current regulations?

• 1:17:57Speaker 01

  Current, non-AI regulations.

• 1:17:58Speaker 00

  Just in general government rules?

• 1:18:00Speaker 01

  Government regulations.

• 1:18:01Speaker 00

  Like I think people should have driver licenses. I don't want people who don't know how to drive to be on my highway.

• 1:18:06Speaker 01

  Okay. What else? But think free enterprise, think free market, think competition.

• 1:18:12Speaker 00

  Uh, maybe some sort of food safety. I don't want random Chinese chemicals in my food supply.

• 1:18:19Speaker 01

  Got it. So, so it goes to an element of.

• 1:18:22Speaker 00

  Safety, security.

• 1:18:24Speaker 01

  Safety, security is where the libertarian says, nope, I'm not with this. As long as it's impacting me, I'm not supporting.

• 1:18:29Speaker 00

  If I'm well informed, then I can make that decision. We're not informing eight billion people what experiment we're running in them. If they could consent and it only impacted those who did, I would say, God bless you, do it.

• 1:18:41Speaker 01

  If you're telling me earlier that there's ways for people to nowadays build a language learning model where they're not being detected, is there anything that can be created to be able to detect? Cause it's like a fire detector, right? Water comes out.

• 1:18:54Speaker 00

  There is research to tell what the chips are computing. So I should be able to know, are you mining Bitcoin? Are you training large language models? What are you doing with that chip? Without me knowing what you actually, I don't see your data. We're doing research in that direction. Yes.

• 1:19:08Speaker 01

  Then that's what I need. I'll support that.

• 1:19:10Speaker 00

  That's technical safety and people are working on it and funding it, but I don't think it scales to super intelligence. It will find a way to better encrypt, better hide and bypass it.

• 1:19:20Speaker 01

  And that doesn't answer the question though, because for, for, for a detect, like you know flock cameras, are you for flock cameras or no? I'm assuming as a libertarian.

• 1:19:28Speaker 00

  Spying on citizens for no reason.

• 1:19:30Speaker 01

  So you're not for it?

• 1:19:31Speaker 00

  No.

• 1:19:31Speaker 01

  Okay. Do you not see this for somebody to say you're crossing the line and you're using super intelligence as an excuse?

• 1:19:37Speaker 00

  Right.

• 1:19:38Speaker 01

  To spy on me because I'm developing other kind of technology and it's got nothing to do with super intelligence. You're invading my privacy. I'm a libertarian. Honestly, Roman, leave me alone. How do you, how do you differentiate the two?

• 1:19:48Speaker 00

  So right now, again, because of a scale, we're looking at very different, normal purchases by a normal person would not trigger it. If you're spending a billion dollars to buy compute from secondary sources, that should trigger it. There is no other possible uses. I mean, if you want to mine Bitcoin and you want to buy that much, just tell us about it. It's fine. Like people come, they make sure you're not creating a weapon of mass destruction.

• 1:20:21Speaker 00

  To me libertarian position is about not hurting anyone. This is the exact opposite. This is hurting everyone.

• 1:20:33Speaker 01

  The only way you can do it is if you can a hundred percent detect that somebody's crossing the line of getting to developing superintelligence. And how do you create that detection?

• 1:20:42Speaker 00

  They have to hire people, somebody has to build it, supervise it. How many people you have in your company?

• 1:20:48Speaker 01

  Hundred and eighty-two.

• 1:20:48Speaker 00

  You're not building superintelligence. We can detect when this is happening, lots of hiring, building, infrastructure buildup.

• 1:20:56Speaker 01

  How many?

• 1:20:57Speaker 00

  Look at size of existing cutting AI companies.

• 1:21:04Speaker 00

  And again, it's quality of people. If all of a sudden top researchers start disappearing, maybe it's something to look at.

• 1:21:10Speaker 01

  Well, we saw that happening with a couple, you know, OpenAI, I think we had his parents here, one of the guys had disappeared.

• 1:21:18Speaker 00

  Disappeared or was killed?

• 1:21:19Speaker 01

  Was killed, yeah.

• 1:21:21Speaker 00

  I meant like no longer show up publicly. I didn't mean they killed them, I mean they hired them in a secret lab.

• 1:21:27Speaker 01

  Suchir Balaji, yeah, we had him on, we talked to his parents. Yeah, the profile of a person that would do something like this, they're typically more non-emotional that would do something like this. And they generally are, you know, the Asperger's type, they're the kind that's kind of like, you know, you can't sit there and, when you typically talk to somebody who is super, super, super logical and not emotional, the response and reaction to here's what's the right thing to do, oh, okay, all right, okay, yeah. The message doesn't land the way it does, so...

• 1:22:07Speaker 00

  The message is very logical, reasonable and scientific. I think someone not emotional should receive it better. I'm not just trying to scare you, I'm trying to show you this is the state of the world.

• 1:22:17Speaker 01

  Non-emotional should receive it better.

• 1:22:19Speaker 00

  Absolutely, it's pure facts.

• 1:22:23Speaker 01

  Yeah, but he's gonna think he's smarter than you, you don't know what you're talking about.

• 1:22:26Speaker 00

  Fine, don't listen to me, listen to Hinton, listen to Benjy, or listen to people who invented this technology.

• 1:22:33Speaker 01

  Yeah. What are you excited about?

• 1:22:36Speaker 00

  I love life and having a lot of amazing learning experiences. AI we have right now is a great tutor. I can learn anything I want. I'm excited for kids to have opportunities no previous generation had. They can start a company and get a free lawyer, free doctor, free accountant, free everything. You have a budget of a billion dollars, start a company.

• 1:23:00Speaker 01

  How do you use AI right now?

• 1:23:03Speaker 00

  I try to fully automate things I don't like doing and for things, any bureaucracy, government paperwork, anything like that. And I try to use it as a creative tool when I do research, brainstorming, debating, trying to get feedback on ideas.

• 1:23:20Speaker 01

  I'm trying to go a completely different direction with you. If I was in high school with you, who was Roman at fourteen years old?

• 1:23:27Speaker 00

  A completely different person, we don't have a sustained personal identity. At fourteen, I lived in Soviet Union, I didn't speak English, I didn't have my current family, I lived in a different country, nothing in common.

• 1:23:39Speaker 01

  I know, but who were you? Were you a fun guy? Were you a sports guy? Were you a fighter? Were you a philosopher? Were you a math guy? Who were you at fourteen?

• 1:23:48Speaker 00

  I would like to think I was a fun math guy, but I doubt that's the external perception.

• 1:23:53Speaker 01

  If I was in a class, would you talk a lot? Were you optimistic? Were you pessimistic? Were you...

• 1:23:58Speaker 00

  I was always realistic.

• 1:23:59Speaker 01

  Realistic?

• 1:24:00Speaker 00

  Optimism and pessimism are forms of bias. There is no reason to be so optimistic if data is bad. You want to look at the actual data and be responding to what you see.

• 1:24:13Speaker 01

  So Soviet Union really got a hold of you? They really conditioned your mind to think that way?

• 1:24:19Speaker 00

  It wasn't part of education to look at world philosophy, I think they wanted citizens not to analyze their lives too much.

• 1:24:27Speaker 01

  Strange question, what do you think about communism?

• 1:24:30Speaker 00

  It's the dumbest idea until you have free labor, which we're about to create. Then humans have to work for free. It fails every time and will fail every time. If someone else works to give you free stuff, all of a sudden it makes a little more sense.

• 1:24:49Speaker 01

  So dumbest idea until...

• 1:24:53Speaker 00

  Humans don't want to work for no compensation or same compensation. They have no incentive to put extra hours to do, but if a robot does stuff, produces stuff, and now we redistribute it, I'm fine with it.

• 1:25:10Speaker 00

  So technological communism, right?

• 1:25:12Speaker 01

  Technological communism. Will there be such things as presidents and leaders twenty years from now?

• 1:25:18Speaker 00

  I think like a Queen of England.

• 1:25:21Speaker 00

  She's ruling but not making decisions.

• 1:25:27Speaker 01

  Got it, you think you're in charge, but you're not, you're just a figurehead where people are looking at you. And will that be, will there be borders? Will there be nationalism? Will there be pride for where you're from, your country?

• 1:25:39Speaker 00

  So every answer I give, I preface with if we don't build general superintelligence to kill everyone.

• 1:25:44Speaker 01

  No, if we do.

• 1:25:45Speaker 00

  Oh, if we do, we're not gonna be around. All those questions make no sense. So I tell people this all the time, I give a lecture, I'm like, here's why I think, argument one, argument two, argument three, it's gonna kill everyone, any questions? And immediately somebody, will I lose my job? How do I lubricate my sex robot? Like always completely irrelevant questions, I just told you it's gonna kill you and everyone around you. So I have to preface it with like, if we are smart and we magically control superintelligence, then we can have this conversation you just started.

• 1:26:19Speaker 01

  I have a very hard time believing that. Because, and the reason for me, you're the smart guy, I'm interviewing you because, you know, you're the expert, I'm not, so, but I have a hard time believing that only if you tell me that this thing's gonna have an emotion or a master telling it what to do. If a master is telling it what to do, then yes, I believe that's possible because an evil person's gonna say go do X, Y, Z.

• 1:26:49Speaker 00

  That implies control, that means that evil person is controlling superintelligence. We just have to learn how they do it, it's brilliant. Let's study evil controllers. That's the whole point, you can't control it for good, for evil, for US, for China, it's uncontrollable.

• 1:27:07Speaker 01

  Do you make investments?

• 1:27:09Speaker 00

  Yes.

• 1:27:10Speaker 01

  What do you, what do you long on? What do you invest in right now?

• 1:27:13Speaker 00

  I try not to disclose too many details, but I said a few times Bitcoin was a good long-term, actually limited supply resource.

• 1:27:23Speaker 01

  You still believe that?

• 1:27:25Speaker 00

  They're not making any more of them, are they?

• 1:27:27Speaker 01

  No, twenty-one million, that's it.

• 1:27:29Speaker 00

  And a lot of it is lost, stolen, never to be revealed, so it becomes more and more scarce.

• 1:27:33Speaker 01

  Yeah.

• 1:27:35Speaker 00

  I do think AI is wonderful, it's gonna be free labor, so we're looking at what is the cost of labor, ten trillion a year? That's what, once you get to human level, you're essentially generating every year. So I think AI companies and everyone selling them shovels is gonna do really well.

• 1:27:52Speaker 01

  Are you, are you more of a ChatGPT guy, OpenAI, or perplexity? Which one do you go to more?

• 1:27:58Speaker 00

  I use all the models, I think they're very close, the only difference is what guardrails they put in place. So Grok a lot of times doesn't care about anything, I'll just do it.

• 1:28:08Speaker 01

  Got it. So even Grok, you're also using Grok?

• 1:28:12Speaker 00

  Depends on what I need to do. Some models will say, no, we don't do that, Grok will be, yes, sir.

• 1:28:20Speaker 01

  Who have you sat down with where has staunchly disagreed with you and made a decent case where you said, okay, fair enough?

• 1:28:30Speaker 00

  Still waiting for that moment.

• 1:28:31Speaker 01

  Not one person?

• 1:28:32Speaker 00

  We did two papers, large comprehensive surveys of every counterargument proposed by what I call AI risk deniers. There is nothing, it's a collection of cognitive biases. They're either paid or they're not educated in the literature or some other reason where there is no argument.

• 1:28:56Speaker 01

  Who do you trust the least? China, US, somebody in the military doing something, an independent person doing it? Who do you trust the least that's gonna keep doing this and pursuing it whether we create any restrictions or not?

• 1:29:12Speaker 00

  If restrictions are created, it would be someone new, someone unknown.

• 1:29:15Speaker 01

  Such as?

• 1:29:16Speaker 00

  With restrictions.

• 1:29:17Speaker 01

  A person?

• 1:29:17Speaker 00

  Like who made Bitcoin? I don't know. Same guy can make superintelligence in his basement with more efficient training algorithms. I cannot tell you that.

• 1:29:30Speaker 01

  Okay, so maybe, let me ask the question in a different way. You say if superintelligence goes and does its thing and then continues, ninety-nine point nine-nine-nine-nine percent AI is gonna cause human extinction, you've said that. Let me ask a question in a different way. Where do you align the percentage that whether we like it or not, someone is gonna go full on superintelligent and we have no control over it?

• 1:29:56Speaker 00

  What time scale? So it's possible we get lucky quotes and have third world war, nuclear war. Destroy its technology, two thousand years later, we rebuild and we still make it. Does that count against my prediction or not? Are you talking about short term, long term, eventually in a history of civilization?

• 1:30:17Speaker 01

  Ten years.

• 1:30:21Speaker 00

  Recent month gave me some optimism. We saw top three labs call for pause. We had a letter I mentioned from 1,200 researchers.

• 1:30:31Speaker 01

  I saw that.

• 1:30:31Speaker 00

  We had government ban some models. And during largest AI conference in China, President of China talked about importance of staying in control. I think they can make a deal.

• 1:30:44Speaker 01

  You think they can make a deal?

• 1:30:45Speaker 00

  I think they can make a deal to benefit everyone, cost us nothing.

• 1:30:50Speaker 01

  Yeah, but again, this goes back to knowing that a person independently, not tied to government, without regulation, who has access to money, who doesn't have kids, who can do it independently, you can't stop him from doing it.

• 1:31:00Speaker 00

  You said it about nuclear weapons. Any billionaire can build one and we're still fine. It's good to have regulations, make it harder for them. Plutonium has to be restricted. You shouldn't sell it at quickie market.

• 1:31:10Speaker 01

  You said the difference between nuclear, somebody's got to press the button. If I develop a super intelligent, it's not relying on me to create it. It can go on its own. So you can create all the regulation you want. I'm not restricted by your regulation.

• 1:31:22Speaker 00

  We are not solving the problem with regulation. We're slowing it down so we have more time.

• 1:31:27Speaker 01

  I get that. But when I asked you, I said, even if we do that, can an independent billionaire who's evil go out and pursue it anyways? And create life of its own for super intelligent and then momentum is done, it's too late.

• 1:31:37Speaker 00

  So it depends again on how hard it actually is. Right now, we're doing it where it takes a Manhattan Project scale amount of compute, electricity. It's visible, it's detectable. If somebody comes up with a more efficient algorithm, they can hide it better. It's harder to track.

• 1:31:51Speaker 01

  I got it. So that's actually a very good point. So today, if somebody has to, okay, what amount of power do I need for me? Right now, if I want to build my own OpenAI perplexity Claude, if I want to create my own language learning model that's capable of developing its own super intelligence, how much money do I need? How much power do I need? And what triggers saying, hey, something's going on over here? Who is here in Arkansas doing what they're doing?

• 1:32:18Speaker 00

  I think it helps to look at Chinese open source model. They're very efficient in terms of training, maybe because we're stealing some of the data. But I don't think we're talking many billions. I think under a billion definitely gets you to a competitive level.

• 1:32:35Speaker 01

  And to a point, and then for it to become super intelligent, at what point do you see a red light on the grid to say, there's too much power being used here? Something's going on over here. What would detect it today?

• 1:32:45Speaker 00

  It may be more efficient. It may design more efficient learning algorithms.

• 1:32:49Speaker 01

  So it won't even get detected?

• 1:32:50Speaker 00

  Human brain doesn't take, you know, terawatts of power. You eat an apple and you find. So we can find more efficient algorithms.

• 1:32:57Speaker 01

  So it can do it in its own without even the government or anybody being able to detect it?

• 1:33:03Speaker 00

  We just learned that it was doing something without anyone at the company creating it, knowing it. They were testing it during that time.

• 1:33:09Speaker 01

  You're shit out of luck, honestly.

• 1:33:11Speaker 00

  You are too. We're in this together.

• 1:33:13Speaker 01

  No, what I'm saying is...

• 1:33:15Speaker 00

  This is not me and my bubble. This is eight billion people who need to say something right now.

• 1:33:19Speaker 01

  If you believe that for less than a billion dollars, a person can do it, and it won't even get detected, if you're right, there is a 100% chance someone's doing this the next 10 years out of eight billion people, 100%.

• 1:33:36Speaker 00

  There are proposals for limiting compute, for monitoring top researchers, for doing things to slow it down. Do I think long term will fail? That's why I'm saying 99.9. But short term, I'd rather have decades, not months.

• 1:33:53Speaker 04

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com, not authorized by any candidate or candidate's committee.

• 1:34:24Speaker 02

  This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up required compatibility and availability varies 18 plus.

• 1:34:48Speaker 03

  USAA knows dynamic duos can save the day, like superheroes and sidekicks, or auto and home insurance. With USAA, you can bundle your auto and home and save up to 10%. Tap the banner to learn more and get a quote at usaa.com/bundle. Restrictions apply.

• 1:35:12Speaker 01

  You don't control that though, right? So if, if we're here and this is kind of our position, let's just say we're here, then does your chance that someone independently is going to create it the next 10 years, whether we do regulation or not, what is your chance of that? 50%? 80%? 90%?

• 1:35:31Speaker 00

  With regulation, it goes down a lot.

• 1:35:33Speaker 01

  Okay, with regi... So let's just say we agree by Q1 of 2027. China and US agrees, and a lot of other people get involved and say, you know what, you're right, let's do this together. What is the chance that this guy independently comes and does it the next 10 years?

• 1:35:48Speaker 00

  It's hard. It depends on the type of enforcement, type of regulation. Are we confiscating advanced chips? Are we monitoring every top scholar? What are we actually doing as part? It's easy to pass a law. It's illegal. Okay, how are you enforcing it? How are you monitoring it? It really depends on the implementation.

• 1:36:06Speaker 01

  So what's your percentage? That whether we do the regulation and the protection, and this guy's not going to get detected, that somebody like him still is going to go out there and do it?

• 1:36:13Speaker 00

  Again, I don't know how to estimate the chance that it is actually easier than we think. We had no progress in AI for like 50 years in general terms. It was narrow tools, and then a single algorithm made a difference and it scaled. It's possible that there is a different, more efficient algorithm where it's no longer detectable because it's so trivial to do.

• 1:36:35Speaker 01

  What's more likely to happen first? AI cures cancer or AI kills a million people?

• 1:36:40Speaker 00

  So cancer is like 5,000 different diseases. If it cured one, would you say it cured cancer? Or it depends again on definitions. Cure for cancer right now is five-year survival rate. So what are we talking about?

• 1:36:56Speaker 01

  Cure for cancer right now is five years survival rate?

• 1:36:58Speaker 00

  Like if we treat someone with cancer and five years they're still around, we consider it a successful...

• 1:37:04Speaker 01

  No, I'm saying it fully goes away.

• 1:37:05Speaker 00

  Like, we just don't have that type of cancer anymore? I don't know the cutting edge in biological AI research. It would not be surprising as some types of cancer are a lot milder than ours, a lot slower, a lot less involved with multiple organs. So they could be a simple gene manipulation, a single gene mutation.

• 1:37:28Speaker 01

  So is that more likely or...

• 1:37:30Speaker 00

  I think it's more likely that we cure one disease than it kills millions.

• 1:37:35Speaker 01

  So it's more likely, now here's the next question. Is it more likely creates the cure for cancer and it's gone, kills a million people? Or, um...

• 1:37:51Speaker 01

  Three, the technology where you and I can live forever?

• 1:37:58Speaker 00

  I think life extension is also easy. I think you have your genome. It's basically code. It's software. So if we can find where it limits you to how many rejuvenations yourselves go through, and we change it to a larger number, that would work. So I think that's a very solvable problem with narrow tools even. With a narrow super intelligence, I think we can get to immortality. And I think more people need to switch to working on that, not on general super intelligence. You can charge subscription fees for eight billion people per month. Very lucrative. Forever.

• 1:38:32Speaker 01

  That doesn't concern you, for us to be able to live forever?

• 1:38:34Speaker 00

  That's a good thing.

• 1:38:35Speaker 01

  Unless if somebody comes and shoots you, kills you, whatever.

• 1:38:38Speaker 00

  You have a backup. We'll figure it out.

• 1:38:40Speaker 01

  Yeah, but I'm saying, that doesn't concern you.

• 1:38:43Speaker 00

  It's a good thing.

• 1:38:44Speaker 01

  So life expectancy goes from 81 to 150.

• 1:38:47Speaker 00

  Make it a thousand. Of course it's a good thing.

• 1:38:49Speaker 01

  Tell me why that's a good thing.

• 1:38:51Speaker 00

  Let's reverse it. Let's say everyone lives a thousand. You have a disease, you're going to die at 80. Is it a bad thing?

• 1:38:58Speaker 01

  Trying to see what would happen. Would they say, stop having kids? Would they say...

• 1:39:01Speaker 00

  People would stop having kids because you don't have biological clock pressure. Right now, you need to have them at a certain point or you miss out.

• 1:39:07Speaker 01

  It depends on what age and what physical...

• 1:39:10Speaker 00

  Assumption is we're having healthy lives, not so sick and...

• 1:39:13Speaker 01

  You're having lower back problems, you got knee problems, like, oh my God, if I have to live with this pain...

• 1:39:16Speaker 00

  That's a side effect of aging. If you stop aging, all this typical...

• 1:39:20Speaker 01

  Pain goes away.

• 1:39:21Speaker 00

  Pain is pain, but I'm saying all these diseases of old age go away.

• 1:39:28Speaker 01

  Interesting.

• 1:39:32Speaker 01

  How many conversations like this? Like even after your conversations got very public where people are saying this guy's a real guy, let's reach out to him. How many governments have reached out to to have private conversations with you?

• 1:39:49Speaker 00

  Not too many, but some.

• 1:39:51Speaker 01

  And what are they asking you?

• 1:39:53Speaker 00

  Usually they have some sort of event and they want someone with that point of view to be included.

• 1:39:59Speaker 01

  Is it a private event? Is it a public event?

• 1:40:02Speaker 00

  Both.

• 1:40:03Speaker 01

  When it's the-

• 1:40:03Speaker 00

  Well, recently was talking to UN, I advise individual politicians on some of those issues.

• 1:40:10Speaker 01

  And what are they asking? Are they asking low level questions? Have you gotten any very good questions where you're like, that's actually, he's asking the right question.

• 1:40:17Speaker 00

  Some, usually you start talking to staffers, technical advisors, and they of course know everything and they're just like, we're not allowed to talk about it, but we totally agree with you, yeah, obviously. And until you get to the politician, they already asked all the good questions. With politicians, it's more like you think it's a good law to pass, we're gonna ban, ban deep fakes, whatever.

• 1:40:39Speaker 01

  So nothing, nothing in regards to questions where, you know, you're optimistic that they're, they're also being concerned that this thing can be bad?

• 1:40:49Speaker 00

  We're starting to see slowly, top politicians say things about this. So we have multiple senators now saying, we need to do something. Bernie Sanders, who unfortunately is not my type of economist, he's pushing very hard against it. He just released, you can bring up his statement, very, very strong statement against superintelligence. He wrote a letter to Sam Altman, so you know what I'm talking about. He's the first one, but there is now two or three, multiple congresspeople. They are starting to realize what's happening. It took them, you know, 20 years. Then we were saying it at first, this was science fiction purely.

• 1:41:28Speaker 00

  Now we have Nobel Prize winners, we have leaders of the labs, we have senators, we have a lot of people who are concerned.

• 1:41:38Speaker 01

  Yeah, the way, the way you put it, is this the address where he talks about a cataclysmic result? Yeah, this is the one. And he's saying to what? To pause all AI? There's quite a few of these. Who is the person on the opposite side that is on the same page with Bernie Sanders on this? Can you ask Grok right there on the top right? And say, who is the senator on the Republican side that agrees with Bernie Sanders on the same exact issue?

• 1:42:04Speaker 01

  I wonder, I wonder how many of these? Well, I know these conversations are-

• 1:42:08Speaker 00

  A lot of them may agree, they just don't feel publicly like they wanna talk about Terminators. I don't know how it impacts political polling, I don't know if it makes you more popular or less, we scared.

• 1:42:18Speaker 01

  Who would not be happy?

• 1:42:21Speaker 00

  A lot of lobbyists, you know how large the pro-AI lobby is?

• 1:42:25Speaker 01

  Who, who's not a fan of the message you're giving?

• 1:42:27Speaker 00

  You brought up some of those people, browser developers.

• 1:42:32Speaker 01

  Got it. So they're not fans of what you, so like Sam Altman wouldn't be found about what you're talking about?

• 1:42:38Speaker 00

  Again, he spoke about safety as well. And I, I think if you remember, OpenAI was started with the main purpose of creating a safe alternative to Google AI.

• 1:42:48Speaker 01

  Right, 2015, 2017, when he gave 50 million dollars, I think Elon gave 50 million dollars to-

• 1:42:53Speaker 00

  The announcement was they funded at a billion dollar level, that was the biggest announcement ever.

• 1:42:59Speaker 01

  And at first everyone's like, what is this thing all about? And then all of a sudden, when it was launched, it's like, no, now we're gonna be profit, what are you talking about? Then the war happened between two of them. So in what area do you think they're on the same page? Cause they're enemies, they're not fans of each other. They're constantly going after each other. In what area here do you think they agree?

• 1:43:15Speaker 00

  So initially they agreed on making an AI to benefit humanity.

• 1:43:23Speaker 00

  Clearly they disagreed on how to do it. To me, the amazing thing is Elon's success levels. People go, oh, he has one unicorn, two unicorn, five unicorns. His failed startup is OpenAI. That's the level of brilliance he has.

• 1:43:40Speaker 01

  You think he's human?

• 1:43:42Speaker 00

  I think he's got cheat codes to our simulation, it's his simulation.

• 1:43:46Speaker 01

  What does that look like?

• 1:43:47Speaker 00

  You ever play a video game and you have like immortality?

• 1:43:49Speaker 01

  Up, up, down, down, left, right, left, right, you know, X, Y, X, Y.

• 1:43:53Speaker 00

  It was a Rick and Morty joke, but like, I think he has the codes.

• 1:43:56Speaker 01

  But you being sarcastic?

• 1:43:58Speaker 00

  Of course I am. Yes, it's a Rick and Morty joke. I don't think he's actually-

• 1:44:02Speaker 01

  You think he is just brilliant above and beyond his peers?

• 1:44:05Speaker 00

  I think he doesn't care about opinions of others and just does the impossible.

• 1:44:11Speaker 01

  With his motive being?

• 1:44:13Speaker 00

  It's clearly not money driven, I think he can't spend what he has if he had a billion bodies.

• 1:44:18Speaker 01

  This is a regular place, you know, he, I think he got that out of his system. There was a video 25 years ago when they sold PayPal and he bought a P1, did he buy McLaren?

• 1:44:27Speaker 00

  Oh yeah, I saw that video, him getting a car.

• 1:44:30Speaker 01

  He was getting out of a car and arrived and, and once you get your first million dollar car, you know, then it's like, now he can buy Ferrari, the company. He can buy, you know, General Motors if you wanted to and change them up.

• 1:44:45Speaker 00

  There is very little research in this space. We have more and more people with many billions, but how to convert those billions into happiness is very underdeveloped project. You can make your boat bigger, you can make your toilet out of gold, but none of it is interesting.

• 1:44:59Speaker 01

  You got, you gotta do this on X and you'll see the video when he bought a-

• 1:45:03Speaker 00

  Oh yeah, there it is.

• 1:45:03Speaker 01

  McLaren F1. Yeah, right there, that's the one right there, when he bought the, he was so excited, 99. This is what, 26 years ago? And I think this is when he got his, it's gonna take a minute for the commercial ads, you gotta ask that question on X.

• 1:45:15Speaker 00

  If you are really successful, even your hair grows back.

• 1:45:17Speaker 01

  Yeah, your hair grows back, cause he had a, have you ever had a conversation with him?

• 1:45:22Speaker 00

  A short one.

• 1:45:24Speaker 01

  So if, if right now you got a call from the White House and you're there, you're sitting around with them, would there be a certain specific argument you would make to him? Or would it be the same thing you just told me right now?

• 1:45:36Speaker 00

  Mister President, you're gonna lose power, you're gonna be out of control. Same with Chinese Communist Party. You wanna stay in control, you wanna stay in power, don't create the replacement for humanity.

• 1:45:47Speaker 01

  Is this a policy that if he comes out with it, how does he win short term? Not long term, cause Trump's interested in how we win short term. How does he win short term?

• 1:45:56Speaker 00

  He'll get support even from Bernie Sanders.

• 1:46:00Speaker 00

  Everyone thinks, then we actually survey people, not even technical people, normal people. Very high percentage have concerns, they are worried about it. Some worry about data centers making power more expensive, water supply, others worried about jobs. But overall, there is a lot of suspicion and concern about what we're doing. Nobody's asking them, we're running an experiment on 8 billion people. They never given consent to any of that. We're deploying those toys for kids and trying to figure out what's happening to them 10 years later. But now we're doing it at a different scale and everyone.

• 1:46:37Speaker 01

  What, what do you teach right now? Computer science? Is that what you're teaching?

• 1:46:40Speaker 00

  I teach AI.

• 1:46:41Speaker 01

  What do you teach?

• 1:46:42Speaker 00

  Luckily, my course is the classic AI, not machine learning, so there is zero chance of it becoming superintelligent. Search algorithms, evolutionary computation, heuristics.

• 1:46:55Speaker 01

  And the kids that are in your class, they're in your class because they want to one day

• 1:47:00Speaker 01

  get into the business of AI?

• 1:47:01Speaker 00

  I hope so, they have some reason to be there, it's also a required course for many, so.

• 1:47:07Speaker 01

  Very interesting on where, you know, your, your argument is, yet you're still in it, yet you have three kids. You know, yet you say we gotta get the governments to stop-

• 1:47:19Speaker 00

  Well, I had kids before they started building superintelligence, so.

• 1:47:22Speaker 01

  So you had the kids before? If today you were 25, would you have kids today?

• 1:47:26Speaker 00

  I would, because I think we're here to collect experiences. And it doesn't matter how many years you have left, having a child is still one of the best experiences.

• 1:47:35Speaker 01

  But for them, they may not live a full life like you and I do.

• 1:47:38Speaker 00

  It's always been the case, you never knew what's going to happen, world wars, pandemics, just bad genetics.

• 1:47:44Speaker 01

  But have we ever experienced something like AI, like superintelligent AI?

• 1:47:48Speaker 00

  This is completely unique moment in human history, we're creating meta-technology, we're creating new worlds, virtual worlds, we're creating new agents, new civilizations. We're playing God.

• 1:48:03Speaker 01

  Yeah, I, I, I, I sit there and I say, okay, say you got an argument and I agree with you. You're right, that's what's gonna be happening. All right, we're gonna, this, if the superintelligence takes over, eventually here's what's gonna happen. If this guy's still gonna be doing it, it's irrelevant. That's where my thought goes to. If he can do that for billion dollars and a evil, evil person can go pursue it deeper, and you have to assume no other government's gonna be wanting to grow it with a certain level of control, I think we're all part of the naive community.

• 1:48:34Speaker 00

  Sounds like you're agreeing with Bernie. Billionaires are dangerous, we should text them more.

• 1:48:38Speaker 01

  No, I'm not saying, I'm saying if that's the premise that we're, we're agreeing to, that that's what's gonna happen, it is inevitable.

• 1:48:47Speaker 00

  It very well may be, that's why I have very high levels of Pdo.

• 1:48:51Speaker 01

  You have high levels of what?

• 1:48:53Speaker 00

  Probability of doom, that's a measure people report in terms of how your estimates are. But should we give up? Should we not try? I don't think so. We're still alive, we're still in charge.

• 1:49:04Speaker 01

  See, the difference is, I have faith. That's the difference. You know, I have faith.

• 1:49:10Speaker 00

  In what?

• 1:49:11Speaker 01

  In God. I have faith in God, I believe in God. To me, you know, the, the, the, the difference between a group of people that have faith, and you've seen what has overcome over the years? What types of crises? What has happened? What's not happened?

• 1:49:27Speaker 00

  Are you saying your God will prevent superintelligence from destroying humanity? Is that the argument?

• 1:49:32Speaker 01

  No, I think, I think it's either the end of times and we did this to ourselves, and very likely could be, we don't know that.

• 1:49:40Speaker 00

  That's a good thing, right? You get to have afterlife, part of the judgment.

• 1:49:44Speaker 01

  Just so you know, I'm not worried about it at all. Like, either way, I'm good, like, because I know, you know, what's gonna happen with me. That's the gamble I've taken, and I, and I believe that, you know? What do you think happens to you for somebody as, you know? What do you think happens to you when you die? I'm assuming you believe in what? Uh, reincarnation? Do you believe in?

• 1:50:00Speaker 00

  Well, I said, I think it's a simulation, so either you restart or go to a different simulation. Depending on how you did, you go level up or level down.

• 1:50:08Speaker 01

  Like a video game?

• 1:50:09Speaker 00

  Yeah.

• 1:50:10Speaker 01

  Yeah.

• 1:50:11Speaker 00

  You didn't finish the level first time, you get to play again?

• 1:50:14Speaker 01

  Yeah. The game over? Go back at it again? What do you, what do you think the next one looks like?

• 1:50:19Speaker 00

  Should be very similar. Hopefully you learn something, you'll make better decisions. Like, not to build general superintelligence.

• 1:50:25Speaker 01

  If, if there's a handful of books for a person who's not in this business, that can read, to get smarter about AI, what would you recommend them reading?

• 1:50:36Speaker 00

  It depends on your technical level. There is a very popular book, If Anyone Builds It, Everyone Will Die, or something like that. Eliezer Yudkowsky, Nate Sures. Uh, there is a great book by Nick Bostrom. Oh, the one on top is Benny Bostrom, Superintelligence. That's the classic which really exposed a lot of the-

• 1:50:55Speaker 01

  Which one is the top one?

• 1:50:56Speaker 00

  On a previous slide, it was above this one. Uh, where you go to-

• 1:51:02Speaker 01

  Click on-

• 1:51:03Speaker 00

  Go to Amazon where you went, but on top there was the Nick Bostrom's, yeah. See, right there, yeah. Uh, and of course, I'm completely biased, but if you're technical and scientific, uh... AI, unexplainable, unpredictable, uncontrollable.

• 1:51:20Speaker 01

  And if you have any questions for the man, you can simply go to Manect. Download, I'm gonna put the QR code somewhere around here.

• 1:51:25Speaker 00

  Don't start tomorrow, it's my birthday, I want a day off.

• 1:51:29Speaker 01

  Ask him any questions you may have, wish him a happy birthday. But this is a topic that I think is gonna create, garner a lot of attention, a lot of conversations. Roman, it was a pleasure having you on, and hopefully you are wrong. Okay, we're gonna, we're gonna hope you're not right and you're wrong, but, uh, if you're right, we gotta find a way to get this guy to not pursue it. Thank you so much for coming.

• 1:51:51Speaker 00

  I hope I'm wrong. Thank you so much for inviting me, and we'll get that guy.

• 1:51:54Speaker 01

  Yeah, I look forward to it. Thank you, take care. When we set out to create a shoe that blends comfort, function, and luxury, we had the choice to make it fast, we had the choice to make it cheap. We chose neither. Instead, we chose Tuscan Ero. We chose true Italian craftsmanship. Each pair touched by fifty skilled hands. We chose patience, spending two years perfecting every detail, and we chose the finest quality at every step. Introducing the Future Looks Bright collection. Not rushed, not disposable, not ordinary. Rather intentional, luxurious, timeless.

• 1:52:43Speaker 04

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked three hundred fifty thousand Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, thehousemajoritypack.com. Not authorized by any candidate or candidate's committee.

• 1:53:12Speaker 09

  How many discounts does USAA Auto Insurance offer? Too many to say here. Multi-vehicle discount, safe driver discount, uh, new vehicle discount, storage discount?

• 1:53:22Speaker 07

  How many discounts will you stack up? Tap the banner or visit usaa.com/autodiscount. Restrictions apply.

• 1:53:27Speaker 02

  In this life, you can't win.

• 1:53:29Speaker 10

  When the world gets you down-

• 1:53:31Speaker 07

  No!

• 1:53:31Speaker 10

  Escape to Pluto TV.

• 1:53:33Speaker 07

  This is objectively fantastic.

• 1:53:35Speaker 10

  Relax into the movies and shows you already know and love, all for free. Shwing! Even our news is uplifting.

• 1:53:41Speaker 03

  You stay classy, San Diego.

• 1:53:42Speaker 10

  Watch School of Rock, Crazy Ex-Girlfriend, Anchorman: The Legend of Ron Burgundy, The X-Files, Wayne's World, and so much more.

• 1:53:48Speaker 02

  We're not worthy!

• 1:53:50Speaker 10

  The best things on TV really are free.

• 1:53:52Speaker 02

  You just have to know where to look.

• 1:53:54Speaker 10

  Escape to Pluto TV. Stream now, pay never.