"I Desperately Want To Live In The Matrix" - Dr. Mike Israetel

This is a lively, no-holds-barred debate about whether AI can truly be intelligent, conscious, or understand anything at all — and what happens when (or if) machines become smarter than us. Dr. Mike Israetel is a sports scientist, entrepreneur, and co-founder of RP Strength (a fitness company). He describes himself as a "dilettante" in AI but brings a fascinating outsider's perspective. Jared Feather (IFBB Pro bodybuilder and exercise physiologist) The Big Questions: 1. When is superintelligence coming? 2. Does AI actually understand anything? 3. The Simulation Debate (The Spiciest Part) 4. Will AI kill us all? (The Doomer Debate) 5. What happens to human jobs and purpose? 6. Do we need suffering? Mikes channel: https://www.youtube.com/channel/UCfQgsKhHjSyRLOp9mnffqVg RESCRIPT INTERACTIVE PLAYER: https://app.rescript.info/public/share/GVMUXHCqctPkXH8WcYtufFG7FQcdJew_RL_MLgMKU1U --- TIMESTAMPS: 00:00:00 Introduction & Workout Demo 00:04:15 ASI Timelines & Definitions 00:10:24 The Embodiment Debate 00:18:28 Neutrinos & Abstract Knowledge 00:25:56 Can AI Learn From YouTube? 00:31:25 Diversity of Intelligence 00:36:00 AI Slop & Understanding 00:45:18 The Simulation Argument: Fire & Water 00:58:36 Consciousness & Zombies 01:04:30 Do Reasoning Models Actually Reason? 01:12:00 The Live Learning Problem 01:19:15 Superintelligence & Benevolence 01:28:59 What is True Agency? 01:37:20 Game Theory & The "Kill All Humans" Fallacy 01:48:05 Regulation & The China Factor 01:55:52 Mind Uploading & The Future of Love 02:04:41 Economics of ASI: Will We Be Useless? 02:13:35 The Matrix & The Value of Suffering 02:17:30 Transhumanism & Inequality 02:21:28 Debrief: AI Medical Advice & Final Thoughts --- REFERENCES: Paper: [00:10:45] Alchemy and Artificial Intelligence (Dreyfus) https://www.rand.org/content/dam/rand/pubs/papers/2006/P3244.pdf [00:10:55] The Chinese Room Argument (John Searle) https://home.csulb.edu/~cwallis/382/readings/482/searle.minds.brains.programs.bbs.1980.pdf [00:11:05] The Symbol Grounding Problem (Stephen Harnad) https://arxiv.org/html/cs/9906002 [00:23:00] Attention Is All You Need https://arxiv.org/abs/1706.03762 [00:45:00] GPT-4 Technical Report https://arxiv.org/abs/2303.08774 [01:45:00] Anthropic Agentic Misalignment Paper https://www.anthropic.com/research/agentic-misalignment [02:17:45] Retatrutide https://pubmed.ncbi.nlm.nih.gov/37366315/ Organization: [00:15:50] CERN https://home.cern/ [01:05:00] METR Long Horizon Evaluations https://evaluations.metr.org/ MLST Episode: [00:23:10] MLST: Llion Jones - Inventors' Remorse https://www.youtube.com/watch?v=DtePicx_kFY [00:50:30] MLST: Blaise Agüera y Arcas Interview https://www.youtube.com/watch?v=rMSEqJ_4EBk [01:10:00] MLST: David Krakauer https://www.youtube.com/watch?v=dY46YsGWMIc Event: [00:23:40] ARC Prize/Challenge https://arcprize.org/ Book: [00:24:45] The Brain Abstracted https://www.amazon.com/Brain-Abstracted-Simplification-Philosophy-Neuroscience/dp/0262548046 [00:47:55] Pamela McCorduck https://www.amazon.com/Machines-Who-Think-Artificial-Intelligence/dp/1568812051 [01:23:15] The Singularity Is Nearer (Ray Kurzweil) https://www.amazon.com/Singularity-Nearer-Ray-Kurzweil-ebook/dp/B08Y6FYJVY [01:27:35] A Fire Upon The Deep (Vernor Vinge) https://www.amazon.com/Fire-Upon-Deep-S-F-MASTERWORKS-ebook/dp/B00AVUMIZE/ [02:04:50] Deep Utopia (Nick Bostrom) https://www.amazon.com/Deep-Utopia-Meaning-Solved-World/dp/1646871642 [02:05:00] Technofeudalism (Yanis Varoufakis) https://www.amazon.com/Technofeudalism-Killed-Capitalism-Yanis-Varoufakis/dp/1685891241 Visual Context Needed: [00:29:40] AT-AT Walker (Star Wars) https://starwars.fandom.com/wiki/All_Terrain_Armored_Transport Person: [00:33:15] Andrej Karpathy https://karpathy.ai/ Video: [01:40:00] Mike Israetel vs Liron Shapira AI Doom Debate https://www.youtube.com/watch?v=RaDWSPMdM4o Company: [02:26:30] Examine.com https://examine.com/

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  In the world of expense management, SAP Concur is here to help. From cost control slipping to travel chaos. SAP Concur has an integrated travel and expense solution to help you overcome problems like overburdened teams, decisions looming. When travel, expense, and AP are connected with a single AI-supported solution, SAP Concur keeps you ahead of problems. Visit concur.com to learn more.

• 0:31Speaker 01

  Hey Spotify listeners. When you travel with American Airlines, you'll enjoy comfort and convenience at every altitude. Before takeoff, unwind in the Admirals Club Lounge with curated self-serve offerings of hearty bites and refreshments. In flight, stay connected with high-speed Wi-Fi. Sponsored by AT&T. Your journey is designed to feel easy from the moment you check in to the moment you arrive. No matter where you're going, American gets you there. Book now at aa.com.

• 0:56Tim Scarfe?

  So if you had a simulation of fire in a computer or of a stomach, it wouldn't digest. Fire wouldn't get hot. Water wouldn't get wet.

• 1:04Speaker 03

  It would digest.

• 1:05Tim Scarfe?

  We can't even, I mean, this is just, we can't even be debating this.

• 1:08Speaker 03

  I can and I will. Watch this.

• 1:10Tim Scarfe?

  When you saw The Matrix, right? You know, the architect, the first version of The Matrix. It was too perfect. There weren't any problems.

• 1:16Speaker 03

  Yeah, so like, I'll say that. When I saw that, my inner philosopher was like, "Ugh, bleh."

• 1:22Tim Scarfe?

  You don't actually want to live in that world, do you?

• 1:24Speaker 03

  Desperately. Everybody does.

• 1:26Tim Scarfe?

  It would be horrible.

• 1:26Speaker 03

  It would be the best thing ever. Well, sorry, by definition, by the way, what would be horrible about it exactly?

• 1:33Tim Scarfe?

  Yeah, but don't you think that life is about suffering?

• 1:36Speaker 03

  No, fuck no.

• 1:37Tim Scarfe?

  Doesn't that give you perspective on joy?

• 1:40Speaker 03

  No, fuck no.

• 1:41Tim Scarfe?

  I'm gonna go and pick up Dr. Mike. Great to meet you, Mike.

• 1:44Speaker 03

  Mike Wise. How are you?

• 1:47Tim Scarfe?

  Very good, very good.

• 1:48Speaker 04

  Mike. Yeah? It was good. Long flight.

• 1:52Tim Scarfe?

  Nice meeting you, Mike. This is your...

• 1:54Speaker 04

  Cheers.

• 1:56Speaker 03

  Hey guys, Mike is Rattel here. I'm by training a sports scientist and I run RP Strength Fitness Company. I lift weights and my head looks strange. People make fun of me in the street, they laugh and they throw usually rotting fruit. It's unfortunate, sometimes a vegetable. And I have a vision of the world in which more intelligence is almost always better. In which cooperation is a good thing, in which we build a future for every single human that's orders of magnitude better than it is now. And I think that would be a great thing in almost every regard. And the three reasons that you should be watching this podcast is that if you want to see me completely out of my depth, embarrass the shit out of myself with terms I don't understand as they're rendered out of my mouth,

• 2:39Speaker 03

  get pushed hard and be corrected on a variety of things, and also have some lively debate and maybe make some future predictions that get us both in trouble. I think those are some great reasons to tune in.

• 2:49Speaker 05

  So Mike is going to demonstrate the first exercise which one of you will do. And while you're doing this one, the other guy will do the other. So Mike is going to bend forward, hinge at the hips, torso down as deep as you can to the floor, almost even touching if you can, and row up. And then again, and that's how they should look. No swinging the upper body, nothing like that. And then, once you're done with that, you'll move over to the next exercise which your partner will be doing. So you'll switch, just pushups. So we like to do a kind of dive bomb style where you keep your belly button really high and you push your chest to the ground and come up.

• 3:22Speaker 03

  Alright, keep it cranking. Nice and easy.

• 3:26Speaker 05

  Oh yeah.

• 3:27Speaker 03

  Absolutely. You guys want to work out until you can't do any more reps without cheating. No cheating. Tim, you're doing a great job. Love the control.

• 3:38Speaker 05

  These are done good pushups?

• 3:39Speaker 03

  These are excellent reps. Marcus, keep that back flat, keep your chest high, but your arms stretched. There you go, just like that. Good.

• 3:47Speaker 05

  You're getting a little bit of hamstring involvement too.

• 3:49Speaker 03

  Now, switch. No rest. Nobody said rest.

• 3:54Speaker 05

  Now you can do the pulling musculature too.

• 3:56Speaker 03

  Yes, rows. So bend over, tummy up, chest up, go deep.

• 4:01Speaker 05

  Lean forward even more.

• 4:02Speaker 03

  Remember, the more reps you do, Tim, the more Marcus suffers, which what you, this is what he wanted.

• 4:09Speaker 03

  There you go. Good job.

• 4:13Speaker 03

  Fucking machines here. Let's go. Don't you fucking quit. Are you out of your mind? I wanted this. Alright, one more round. Switch. Switch, last round. Marcus, get back down and start doing pushups, goddammit. Fuck is wrong with you? Three more. Let's go, three more each. One. Super deep. Two. Last one. Three. Fuck, yes. Beautiful.

• 4:39Speaker 05

  We did our workout earlier.

• 4:42Speaker 03

  Am I the least technically qualified guest you guys have ever had?

• 4:48Speaker 03

  I must be top five. Like, you ask a very distinguished exercise scientist, like, hey, how many reps should I do on curls? And they're like, I just study one pathway of muscle hypertrophy, and I know it real well. I just go to my job, do experiments, and I don't lift weights or know much about that. And you're like, oh, fuck, the generalization did not go anywhere. I'll be here defending San Francisco, which is an odd statement to make. I have a very unique take. I think ASI is coming in 26, 27, and AGI is coming in 29, 30, maybe 31.

• 5:22Tim Scarfe?

  Yeah, you spoke about this with Leon. Why is AGI after ASI?

• 5:26Speaker 03

  Because artificial general intelligence has either, depending on who you speak to, no good definition at all, or multiple spectra definitions, somewhere between shitty. Like, the, you know, various tests of if you, you know, can text something, is it truly intelligent?

• 5:47Speaker 03

  All the way through, I think, very good. But artificial general intelligence, typically, some of the better definitions encompass all of the kinds of intelligence and abilities that human beings are able to put out. And honestly, from a vibe perspective, if you say, like, we've really cracked AGI, but your machine can't do some kind of cognitive work that a human can, you haven't cracked AGI in any meaningful respect. But because humans have some very interesting abilities that we just don't have the reach for from um kind of an integration perspective, for example, smells and tastes.

• 6:30Speaker 03

  I mean, nanotech's got a bit of a way to go to get a machine to smell and taste in a meaningful way. And so like, something about being a chef in being able to cognitively rank and tier tastes and smells and spectra and functionally employ them in real life to make food, that's not gonna happen probably in twenty six, twenty seven, right? But then we look back and so AGI, because it is human inclusive, and that means we have to replicate every kind of human intelligence, or say confidently, we have AGI from my perspective. All the following, all bullshit dilettante takes, by the way, um, then, you know, AGI is actually like, oh, pretty tough, will happen, I think, but tough. Now, artificial

• 7:11Speaker 03

  superintelligence is uh can be thought of in many ways, but I think once you have something that on many domains of human ability, not all, is radically more intelligent, it functionally vibe check wise, is superintelligence. For example, something that is linguistically, mathematically, scientifically, three dimensional object rotation, world model depth recursion ability, is one hundred times the power of a human, you can tell me it's not superintelligent because it doesn't know how to smell or taste, has just never seen the world with its own eyes, even though it scanned all of YouTube and integrated all of YouTube's visual data and integrated three D world model. Can I swear a little bit, or is that not a good

• 7:55Speaker 03

  thing? It's a fucking artificial superintelligence then, whatever, right? And so I think because in many respects, like, so I work with um a few AIs, but I love OpenAI's product suite. I pay the infinity amount per month to get the pro uh version, the pro subscription for GPT. And so GPT five pro is in some respects, not as smart as me yet, mostly because it can't cross link distant concepts together and integrated whole as well, because that's not really what it was designed to do, because that would burn a shitload of tokens for an ROI that for most people is meaningless.

• 8:36Speaker 03

  But boy, does it do black hole physics better than me and almost everyone and recently better than every scientist because it's got novel discoveries now. And so you take the abilities of just GPT-5, right? And in the back rooms at OpenAI, they have something that just beats the shit out of GPT-5 and it's currently getting fine tuned or, you know, post-trained, whatever. That thing's probably 10 times as smart. That's superintelligence. Also factual knowledge about the world. I mean, like, what percent do you think you know of total data, total real things about the world? How tall are people? What do they look like? Where, you know, what's the capital of France? That stuff compared to just GPT-5?

• 9:17Speaker 03

  I mean, it's some infinitesimally small fraction that on knowledge base is already superintelligence. I mean, put GPT-5 against any Jeopardy winner and you get the world's biggest creaming all of a sudden. That sounds like one of my older films, am I right? So, so in, in, in many respects, I think in 2026, we are going to see AI systems that 10x or 100x human abilities or 2x or anything between that. And so, once you get to maybe two thirds of all cognitive abilities, maybe 75%, maybe 80%, demonstrably, machines are just like categorically superior by orders of magnitude, that is superintelligence. So, superintelligence, two ways. On vibes and heuristic, by what I just described,

• 10:00Speaker 03

  and also on effect. I mean, if you have a smart enough AI and it's crapping out novel hypotheses once an hour, and it takes scientists weeks to grind through them, and it starts getting 60%, 80%, 90% hit rate on like, um, it understands the cell and it's giving us novel disease cures every week. That is superintelligence. And so, superintelligence has to be measured by two things. One is under the hood cognitive science abilities, but the other is like real world effect because here's another really big thing, super quick. You don't have superintelligence really in truthful evidence to convince most skeptics until you have the fruits of its labor. It's like someone tells you they're really rich. You're like, can you buy me a flight to Dubai? They're like, ah, money's tight right now. You're like,

• 10:44Speaker 03

  mm, I believe you, but I don't. But if, if, you know, Elon Musk, you're like, hey, can you buy me a flight to Dubai? He's like, yes, I just bought you Dubai Airlines. So you're like, the airline. You're like, yeah, just acquired it. You're like, right. Okay. Wow. That's a demonstration. And so I think in 2026, I'm very confident, though never certain from scientific perspective, CIA probability scale, extremely likely 97 to 100% that in some time in late 2026, we are going to start opening the cornucopia of machine intelligence that people understand that at least parts of machine intelligence are absolutely superintelligent. And because of exponentials by 27, 28, 29, the shit is going to get completely insane.

• 11:20Tim Scarfe?

  There is so much to unpack there and I really like you, Max, so I don't want to disagree with you on my podcast, but I have to.

• 11:24Speaker 03

  No, please. I love disagreement.

• 11:26Tim Scarfe?

  And I think I disagree with almost everything you've just said, so we can take it one, one at a time. So first of all, the definition of intelligence, you were talking about knowledge as well. I think that knowledge is non-fungible. In fact, one of the, the most pervasive critiques of artificial intelligence, going all the way back to the seventies, there was Dreyfus and there was, you know, John Searl. It's, it's this grounding problem. Stephen Harnad spoke about this grounding problem. And essentially the enterprise of intelligence is the accumulation of information, which is relevant for adaptivity around an environment. So the product of our knowledge is all of the, you know, it's Wikipedia, it's, it's the hard drive in the sky. It's our culture. It's all of the things that we have acquired.

• 12:10Tim Scarfe?

  But the problem is there is a gap between syntax and semantics. So if an alien read Wikipedia, they might read the article about Trump or about America or something like that. And that is not the same as the embodied experience of being there. These are just pointers. So this semantics is talking about the connected inactive embodied graph of actually experiencing things through time. And the biggest misconception in all of AI, what all of the folks in San Francisco believe in, is this philosophical idea called functionalism. And the best way to describe it is there is this analogy of walking up a mountain. I spoke to a philosopher the other day, wonderful lady called Anna Sia Kuna.

• 12:52Tim Scarfe?

  And and she said that we're walking up the mountain. And when we get to the top of the mountain, we have all of these abstract capabilities like being able to reason, play chess. But that disregards that the path that you took walking up the mountain is very important. And not only the path, the physical instantiation, the stuff that the mountain is made out of. Because it's a reasonable argument, in my opinion, that intelligence is a property of adaptive matter. It's an extensive property. It's much like temperature. Temperature is a coarse graining. It's an it's an effective theory to describe the details of the molecules moving around. And we we screen that detail off and we call it temperature. And I think intelligence is like that.

• 13:37Tim Scarfe?

  And I think that knowledge is quite similar. I think knowledge is actually a physical causal graph that is enacted over time. And it cannot be abstracted in the way that you're describing. The reason the abstractions work is because they are pointers to our embodied experience. So they make sense to us, but on their own, they don't make any sense.

• 13:57Speaker 03

  I like that take. I'll push back.

• 14:05Speaker 03

  Intelligence, my favorite definition is the most basic definition of intelligence, the ability to solve problems.

• 14:10Tim Scarfe?

  Yep, I hate that.

• 14:14Speaker 03

  And you find out if you're truly intelligent, if you can solve problems of any degree of complexity. That is, I would actually say responding to one stimulus in a cogent manner. At least knowing how to respond to one stimulus, input, output, is the beginning of intelligence. All intelligence above that is just stacked layers of complexity. So when you say embodied,

• 14:40Speaker 03

  there's a lot there, for sure. But also, the only reason that we call humans intelligence, intelligent, is because we have a representational abstracted neural network called the brain. And your brain doesn't actually have anything in it that's magical. It's just network pings off one another recursively. And so your brain is in very many deeply important respects, exactly as abstracted and unrelated to reality as a data center. And so you can climb mountains, you can touch stuff, but you never truly embodied experience anything if you push on that philosophical button hard enough,

• 15:21Speaker 03

  because you can always abstract out to like, these are just neural network pings from groups of neurons. And so you don't truly deeply know anything in some kind of weird philosophical way, because it's just neural network calculus all the way down. And so whether it's you or a machine brain, like what they have in Optimus, for example, it's representational. And so intelligence is always, seemingly, going to be something that is lossy to some extent. It is a compression function. It's a real rude way to say it, because it's a very impressive one. But there, you know, you climb the mountain, that's cool. Helicopter can climb a mountain much better than you.

• 16:03Speaker 03

  Does not have the ability to reason and abstractly and plan and predict things at all. And you can get really amazing high-def video and samples of rocks on the mountain and everything, but if you don't have an analyzer, a system processor, an integrator, and a recursive function, you don't actually model that data in any meaningful way and it just sits there as bits on a computer. And so when you say that, you know, embodiment is a prerequisite of intelligence, I would push back and ask you to answer this. If we have somebody that has read every single physics book, and especially, let's say, particle physics, and they're so good at it that they can tell

• 16:45Speaker 03

  the computer system that aligns the particle beams at CERN to do a job good enough to produce real-world data.

• 16:54Speaker 03

  Are they truly intelligent about particle physics? By your definition, and I'm being a little bit facetious for comedic effect, the answer is no, because if I put you in the particle accelerator, you get torn to fucking shreds. And you're like, oh shit, J K. It's actually impossible for humans to perceive particles directly in any sensory organ, because they're plank length type shit, and you can't see it. Vision is not a cogent concept at those lengths, so that goes straight to hell. And so if that scientist is so adept that he can actually run CERN, but it's all neural network modeling in his head, he's never had, actually had any real experience with a particle collider, no human has. You can't open the latch when the thing is on, it'll shoot fucking neutrinos at you. So if that's true, and we say, okay, okay, okay, well, he actually knows things. He actually has intelligence.

• 17:39Speaker 03

  Why doesn't G P T five and its network clusters actually have an understanding of the real world? It's never seen, but the guy's never seen a particle in his life. Zero scientists have ever seen a neutrino, ever. They have no perceptual experience of a neutrino. It's purely hypothetical, but they make real world predictions because the neural network and all the redundant structures that create what we call intelligence and understanding are, their, their, their vector arrangements represent decently well enough, always an approximation, but a damn good one, predictively valid one, of real things that are happening in the real world. And if G P T five understands like how humans act to a ninety eight percent,

• 18:21Speaker 03

  but if it was embodied in a robot body and had vision, it'd be like, oh, fuck, there's the rest of that two percent. I think ninety eight percent of the way to intelligence is the same thing that back when I was a professor of if one of your students gets a ninety eight percent on an exam, is he, is he pretty good at understanding the material? Yeah, fuck yeah. Now, just off camera over there is my best student of all time, Jared Feather. He never missed a single point on any of my exams, a hundred percent. Which just means the exams were hard enough to really push his understanding. Is Jared the best student ever had the best trainer? Yes, for sure. But does that mean when kids score ninety five or ninety eight percent on my exam, I'm like, never let this kid train you. He doesn't really know the real world, it's just abstraction. No, fuck no. They know real things in the real world that can really help you get fit,

• 19:05Speaker 03

  even though they might themselves have never been, been able to get very fit, uh, because they know the actual structure out there, purely representational in a neural network, but it.

• 19:15Speaker 00

  In the world of expense management, S A P Concur is here to help. From cost control slipping to travel chaos.

• 19:23Speaker 00

  SAP Concur has an integrated travel and expense solution to help you overcome problems like overburdened teams, decisions looming. When travel, expense and AP are connected with a single AI-supported solution, SAP Concur keeps you ahead of problems. Visit Concur.com to learn more.

• 19:46Speaker 01

  Hey Spotify listeners. When you travel with American Airlines, you'll enjoy comfort and convenience at every altitude. Before takeoff, unwind in the Admirals Club Lounge with curated self-serve offerings of hearty bites and refreshments. In flight, stay connected with high-speed Wi-Fi. Sponsored by AT&T. Your journey is designed to feel easy from the moment you check in to the moment you arrive. No matter where you're going, American gets you there. Book now at aa.com.

• 20:11Speaker 03

  It passes the test of when they see the real thing, they go, I know what a dumbbell is, I know what a human is, I know what vectors are, I know what forces are, and I'm gonna have them do this, and it works.

• 20:20Tim Scarfe?

  Yeah, a human would have a shared understanding because if you think about it, the representations in our brain, they are the product of evolution. So we've been evolving for billions of years, and I know you read that Ray Kurzweil book, and it's a beautiful recount of how we had these kind of exponential increases. So, you know, we had genetic evolution, which is very slow, and then we had this ontogenetic, phylogenetic hacking, where we developed nervous systems and brains, and we developed culture, and we started evolving at light speed and so on and so forth. But at the end of the day, the product of that knowledge acquisition, that process of intelligence are all of these representations. And everything in language is a representation, you know, like for example, algo speak, that

• 21:03Tim Scarfe?

  is a term to represent the evolution, the linguistic evolution of our language. There's this term called unalive. So to get around the social media filters, we now say

• 21:12Speaker 03

  That's still working.

• 21:14Tim Scarfe?

  And that is an example of linguistic evolution, and ChatGPT is not capable of doing that because there's a big difference between the process

• 21:22Speaker 03

  No, absolutely not. Wouldn't understand what unaliving is?

• 21:25Tim Scarfe?

  We'll get into this, right? But understanding Is not about being at the top of the mountain. Understanding is about the path to the mountain. So we understand things because we are physically embodied, right? You were talking about, oh, you know, we can't observe the particles. It doesn't matter. We are in the causal graph of the particles. You know, they affect us even if we're not.

• 21:44Speaker 03

  They don't. They're inside their collider. You don't, it's a radiation shield.

• 21:47Tim Scarfe?

  Made out of particles. That, that, that is.

• 21:48Speaker 03

  Yeah, but you're not made out of neutrinos. They're passing through you. The neutrinos are completely abstracted as far as we're concerned. They actually pass through the earth the entire time. So like, there's all kinds of theoretical physics that have nothing to do with you really in a causal graph. They don't affect your behavior. They haven't really meaningfully affected any of your decisions. We, we'd have to abstract like eighteen layers and do quantum mechanics that none of us are. I don't know, you're probably smart enough to get it. I'm not. And so like, that shit's really shit in the real world that we're purely abstracting to. But because knowledge is actually representing the real world, it works. With, I, I'm not, I'm not so sure. If you could explain what it is exactly you're getting at that we get out of embodied experience? And how libraries and education curricula and schools and reading a bunch of books can possibly give you knowledge?

• 22:30Speaker 03

  If none of that is any kind of experience that is embodied? Like, how does ChatGPT know the vibe of Shinjuku in Tokyo? And can accurately render to me what the vibe feels like if it's never been to Tokyo? Same way a travel agent does. But it's real, actual things they know that when you go there, you're like, oh, fuck, it's pretty much correct. How does that work?

• 22:49Tim Scarfe?

  Well, first of all, we, we should distinguish, you know, knowledge and understanding. So many, many people have different definitions of this, you know? Like in the cognitive sciences, people talk about, you know, kind of factual knowledge, which is, you know, just states of affairs. And there's procedural knowledge, knowing what to do. There's conceptual knowledge, knowing how to think. And roughly speaking, I make the designation that they're, you know, knowing states of affairs. And understanding is this ability to generate new knowledge. So if you have an understanding of the world, which means you know it at an abstract level, you can do this Lego building in your mind and you can kind of create new knowledge. But the simple grounding problem is something a little bit different, even with the neutrinos, that we use this thing.

• 23:32Tim Scarfe?

  David Krakauer calls it the principle of materiality, which is that we use the world to think. And many of the abstractions that we've converged on over, you know, millions of years of, of evolution is because we have this shared physical world. So we, you and I both understand similar abstractions which have derived. From our century motor circuits and from language and so on. And it means that we can talk at a very abstract level. And because we're both players in the same game, we understand each other. And the parlor trick, and we won't go over Sells' Chinese room argument. I'm sure you're familiar with it. He was saying exactly the same thing, that, you know, basically a computer can tell you all of these things as if they understood. And you understand because you actually have the the feeling, you actually know you.

• 24:14Tim Scarfe?

  Even though you've never been to Japan, you've experienced many things that are like people that go to Japan experience. So you have some kind of a shared understanding. There is this this intersection in your understanding tree that something like ChatGPT doesn't have. Now, I want to make a distinction as well. And I'm really impressed with language models. You know, Opus four point five came out yesterday.

• 24:34Speaker 03

  GPT is not a language model, it's been multimodal for two years now.

• 24:38Tim Scarfe?

  It it's still it's it's known as a language model, it's a self attention.

• 24:41Speaker 03

  Right, but we but we have we have to concede the fact that it is it's it's an omni model. It's been multimodal for a long time.

• 24:47Tim Scarfe?

  Yeah, it's multimodal, but it's it's still a self attention transformer. It has a couple of tricks.

• 24:51Speaker 03

  Transformer's part is true, but just calling it a large language model, I think, cuts it a short of a huge fraction of its capability. I mean, it does visual reasoning.

• 25:00Tim Scarfe?

  Doesn't make any difference.

• 25:01Speaker 03

  Okay.

• 25:02Tim Scarfe?

  Yeah, yeah, so it's a self attention. I mean, on the most recent show we just published, I interviewed the the guy who invented transformers, actually. But yeah, we we just tokenize different domains. We learn the statistical distribution, we do this next token prediction, you know, within a fixed context window. So it's distributional matching, but then there's some other stuff on the top. There's the RLHF, and there's also this, you know, reinforcement learning with verifiable rewards for verifiable domains. And and this is why they're doing really, really well on reasoning tasks when you have a verifier. So the ARC challenge is a great example of this. I'm a big fan of that, you know, Francois Charlet is a friend of mine. And he designed this challenge to show abstract generalization. So you only have a few examples to learn from. And L L M's, he predicted, would be terrible at this.

• 25:46Tim Scarfe?

  And unfortunately for him, the L L M's are now getting incredibly good at it. And that's because, in my opinion, the L L M's don't really understand in this grounded way that we're talking about. But in abstract domains, you can fully describe an abstract domain like a two D grid, you know, problem. You don't need to have any grounded knowledge. You can completely understand it.

• 26:05Speaker 03

  What do you mean by grounded knowledge?

• 26:07Tim Scarfe?

  This is grounded knowledge, right?

• 26:09Speaker 03

  You're touching a chair.

• 26:10Tim Scarfe?

  Touching a chair.

• 26:11Speaker 03

  So that's not knowledge, that's neuronal perception arcing up into your brain and back down. And so what are you learning about the chair when you're touching it like that?

• 26:21Tim Scarfe?

  This is the other issue that I have with your perspective, that you have, I would call it a corticocentric view of cognition.

• 26:28Speaker 03

  Absolutely.

• 26:29Tim Scarfe?

  Yeah, exactly. So this is another one of those leaky abstractions I was talking about. By the way, there's a great book called The Brain Abstracted by Marvita Chirimuta. I interviewed her recently, and she said that one of the most pervasive myths in neuroscience is that we use these leaky abstractions and idealizations to talk about cognition. And usually it's using the most recent technology at the time. So, you know, a few hundred years ago, we were describing the brain in terms of pulleys.

• 26:53Speaker 03

  Pulleys and levers, yes.

• 26:54Tim Scarfe?

  That's right. And then it was, you know, as a prediction machine, as a computer and all this kind of stuff. Like, this is an example of, these are grounded things that we understand. They're really good models because we can both talk about computers, we both know what computers are. But the brain doesn't work like that in any sense. And a great example of this is knowledge. You as a personal trainer know this, right? So you, you know, I'm sure in the past you've trained folks in the gym. And what you've probably realized is that in your mind, you've got a distilled, abstract, beautiful. Like, you understand things, you've thought about it, you've distilled it, you could write it down in a book. And you probably think, all I need to do is just tell people. And I've done all of the thinking, I've figured all of this out, I just tell them. And what you learn to your chagrin is that there is no substitute for experience.

• 27:38Tim Scarfe?

  Because part of knowing is actually doing. It's just the pure experience of lifting the weights, just feeling the sensations in the body, going through that process. That's why knowledge is non-fungible.

• 27:48Speaker 03

  I've been fairly impressed incrementally with modern A I's ability to render pretty good recommendations on exercise and sports science. And I'll push you back on one of those things. I know a lot of people, they're great people, that have a lot of what you would call embodied knowledge about training. And because they haven't been interested in or capable of doing the abstraction to distill principles, they're not really that great at training other people. They know what works for them and they know scarcely of that, and they're just not good at generalizing because they've never abstracted out the concepts. I've also had many discussions with some of my very, very smart friends, like one of my friends graduated from Harvard, masters data science, worked for Apple and a bunch of other AI companies and stuff.

• 28:32Speaker 03

  And his ability to pick up how to do an exercise incredibly well, just off principal discussions and then trying it like twice, he's instantly like one of the best trainers I know in the area. One shot. Why? Because Jared and I can distill to you like five principles of how to do an exercise well, which are geometrically not complex, heuristically very cogent, and he will instantly be good at teaching people how to exercise without ever doing the exercise. Is some nuance missing? Absolutely. There's some stuff, and we can abstract how it works, that you have to do to actually like, oh, I see, moving the hips back like this.

• 29:14Speaker 03

  But if someone was able to describe that to a computer in vectors of how hips should move back in a squat versus just down, especially if, so I'll tell you this, if I was able to train a model, and I'm not currently doing this and had no plans to do so, if giving it a thousand samples of a squat done right, a thousand samples of a squat done wrong, and about 10 rules of how to squat, it would instantly be 99 percentile of teaching people how to squat with zero grounded experience. And I would also say, what we consider grounded experience is mostly visual data stream. Like, yeah, touching stuff is cool and it gives you something, like, I'm not letting a computer have sex with me until it's had sex with test animals in the lab that came out all wrong.

• 29:57Speaker 03

  But a lot of our data is visual, huge, huge fraction of it. I suspect that if you let a neural network train on all of YouTube, which I understand isn't happening because there's like copyright bullshit, which drives me insane, you just like have like an algo just eat all of YouTube, which is a preposterous amount of data, it would know more about how the world looks in real life than any extant human.

• 30:26Speaker 03

  Period. Because there's more visual data on YouTube than I have collected with my own eyes by, I don't know, ten, twelve, fifty orders of magnitude, something like that. And so, zero embodiment. We did the embodying for it by putting up camcorders into people's faces while they throw up from drinking challenges and then downhill skiing and then the pictures of microbes. If you really train a visual model, and even better, a 3D relational model, on YouTube alone, you will get a more grounded understanding than your eyeballs, which by the way are also just cameras, can ever reference to your brain, which is also a computer. Period. The brain is a computer. Is it a different type of computer than a CPU? Yes. Is it a different type than a GPU?

• 31:11Speaker 03

  Yes, but closer. Are we going to replicate the exactly how the brain works computationally? I'm sure at some point, I suspect we'll need to do that because there's no good reason to believe that brains are like, super, super good at thinking, they're just evolution's best crack at it. You know, like, a tank isn't a walker. Are you a Star Wars fan? No. Do you know Star Wars? Do you know what an AT-AT or AT-SD is? Okay, you know those dog-looking big walker things that shoot lasers out of their faces? Like, what's better, that or a modern battle tank? Well, modern battle tank by a long shot. Why? A battle tank sits like, eight feet off the ground. It can hide in trees and it can shoot like, two miles out. An AT-AT walker stands above the tree line and you just shoot one leg and it falls. Like, why the fuck would you do that? Well, it's a pretty decent attempt to replicate animal locomotion, but the assumption that

• 31:56Speaker 03

  animal locomotion's like, real, the best way to do it is not. So just the same way as like, a rocket or a supersonic aircraft is like, categorically better at flying than a bird is in almost every respect, AI and machine intelligence is going to surpass human intelligence and real deep understanding of the real world just by bypassing our architecture and then later being so smart that it can come around, scan the human brain, be like, oh, that's how they were doing it. Oh, that's kinda cool. I don't think there's a huge ROI of what it's gonna get out of that because it's gonna be way ahead. Uh, but, basically suffice it to say, I think that what we assume we're getting from sensory experience is like, just a few, I don't know, gigs or some number of bytes of data, like, I've seen a lot of shit. It my time, but if if it's all also, by the way, wildly abstracted.

• 32:40Speaker 03

  Like, if you ever think about memories you've had from childhood and you try to really parse like, how did this dresser look? What was my mom wearing? You realize two things. One, you have precious little actual visual data. It's like a 10 second long video, maybe at any kind of fidelity and the fidelity sucks. And two is you're actually hallucinating a lot of that, like you're just pretending in details. If you let an AI train and eat all of YouTube at 4K and really like develop the model from that, it would see and understand the visual three dimensional space around it and the world at a level of embodiment that shits on all of us instantly. That is my conjecture.

• 33:17Tim Scarfe?

  I I understand your perspective. I mean, first of all, you were talking about, you know, doing personal training. Um, there are folks like yourself and I I count myself like this as well. I I even when I'm doing editing and post-production, I really like to build theories about things. I think very deeply about things. I come up with abstractions and and I think they're very fruitful for me and they're not for other people. And this is the beauty of our collective intelligence, because some people just just feel in the moment. I mean, for example, when I see my personal trainer, she's called Lyra. A shout out, she's probably not watching this, but shout out anyway.

• 33:43Speaker 03

  Here, I swear to God, I've been training and eating well. You just haven't heard from me in a little bit. I've been really busy.

• 33:47Tim Scarfe?

  I'll be needing to, yeah, honestly. We'll talk about that later.

• 33:49Speaker 03

  I was trying to help.

• 33:50Tim Scarfe?

  We'll talk about that later. But, um, you know, like, for example, I I'll train with her. And ironically, what I get out of that has got almost nothing to do with the personal training. She's she's an incredible woman, really, really amazing woman, you know. Sometimes she taps me on the shoulder, we laugh and we joke together. And it just enriches my life in so many ways that are not captured by these abstractions. So all of us have different abstractions, but then you're also talking about this kind of monolithic view of AI that understands everything. Um, that's not the way that our, you know, evolution works. Evolution is all about this path dependence that I was talking about. And we have many agents and we all take our own paths, which means when we do creativity, we continue the epistemic, the epistemic lineage that that we are on. And this is this unfurling process. What is not the case is that there's this kind of

• 34:32Tim Scarfe?

  um, gray goo where all of the knowledge is convolved together and like creative steps could happen from from any part. I think any future super intelligence that we do create will resemble our intelligence. So it'll be very kind of diverse and and collective.

• 34:45Speaker 03

  I think at first that's the case. Then when superintelligence is smart and capable enough through a few different ways to re-architect itself, it's going to leave behind our abilities quite quickly. And it's also going to leave behind most of the very narrow conceptions that we have about intelligence. I think Andrej Karpathy, if I'm saying that correctly, you know, you read enough things on Twitter and you're just, I don't know how to say that person's real name, said recently that something that kind of attends to your view a little bit is like, you know, we humans am very much paraphrasing, Andre, I'm sorry if you ever see this, it was an X post, you know, as far as those go, you know, humans, we all sometimes think that like we are somehow

• 35:30Speaker 03

  the seat of intelligence and that our cognitive domain is like what intelligence really is. But we're learning with AI, according to him, that like we have a niche in intelligence that's like, to your point, path dependent evolutionary derived, I was gonna say highly functional, but functional enough to get us here and build all this cool stuff and build cameras and stuff, but only occupies a very small fraction of the entire universe of possibility, the whole vector space of intelligence. And most of that vector space, we actually don't know. And so one of my things, and I'm not saying you're saying this at all, by the way, it's always curious to me when people say, I don't think, because this used to be a majority view in even AI research 20 years ago, is that machines will never be more intelligent than humans.

• 36:13Speaker 03

  And like the gall to think that fucking primates who mostly scratch their testicles and sometimes do theoretical physics have some kind of like global optimum and inclusivity on all of intelligence, it's fucking preposterous. And I think when artificial superintelligence with agency truly kicks off, I think it'll be a miracle when it does, it's going to recursively re-architect itself and start expanding into all of the domains of intelligence we don't know anything about. And then we will see really what intelligence is, and just to use that really quick, just by analogy, like if wolves could talk and even abstract, which they can't, but let's give them two magical abilities, they would pick like the smartest wolf in their wolf pack.

• 36:58Speaker 03

  And be like, you know, Bob the Wolf, he's a fucking man, he fucking knows, right? He knows when the bears are around, he knows when they're not. And then he'd be like, you know, okay, well, here's like, I don't know, Stephen Hawking. Be like, so he's your Bob, like, yeah, in a sense, like, okay. So like, he really kind of knows where the bears are, but like, he does a lot cooler shit than that. And then you try to explain to the wolves what that is, and there's just no idea how to contextualize that whatsoever. I think that's the flavor, to use a very technical term. Um, though I did just learn that in AI training, taste is a technical term. And I was like, what the fuck? And it was explained to me and I was like, I'm not smart enough to understand that by long shot. But the, the flavor of artificial superintelligence is verticality of ability, like doing math at this level, this level, that level.

• 37:42Speaker 03

  And then expansion across domains to domains we understand. And like orders of magnitude, more domains, we haven't even conceived of.

• 37:52Tim Scarfe?

  Yes, and maybe we'll come back to Andre Gepardi. He, he, I'm sorry, he tweeted the other day about AI slot and.

• 37:58Speaker 03

  I saw that tweet.

• 37:59Tim Scarfe?

  Yeah.

• 37:59Speaker 03

  I gave him a definition in the, in the response. He was asking for a definite, like, what is, what is a good definition of AI slop?

• 38:05Tim Scarfe?

  Yeah, do you want a good one?

• 38:06Speaker 03

  Um, I, can, can I read my definition off X? Can I pull up my phone or is that I'm becoming on a podcast? I'll never find it, I take that back. Can I, no, I, I, I have dog shit memory and shitty intelligence on top of that. My definition was something like, uh, I'm going to do a shit job at it. Um, AI slop is when the fraction of, is, is when you generate content. The ratio of its, um, ability to be generated by AI as a denominator and its coherence and utility and novelty as the numerator is really low.

• 38:49Tim Scarfe?

  You've got very good intuitions, actually, yeah. So the, the Will Smith eating pasta, slop is observer relative in, in the sense that the more sophisticated you are as an observer, the more likely you are to detect slop. So anyone looking at that first Will Smith one, they would tell that it's incoherent. And basically my definition of slop is that it is what happens when. Process creates an artifact without understanding, right? So a sophisticated observer will will spot the glitches. And that's why it's possible to post slop on LinkedIn and Instagram. And most people are not domain experts.

• 39:19Speaker 03

  People post on LinkedIn? I'm not even so sure LinkedIn's a real website anymore, I'm kidding.

• 39:23Tim Scarfe?

  LinkedIn is ninety nine percent slop. But I we should delete LinkedIn, but let's not let's not go there. But so, yeah, so to to a naive observer, it's very difficult to observe slop. Um, to an expert, it's it's very easy. And there's also the interesting case that you might see slop the first time and you might not realize. Because another way of describing slop is it's a shallow mimicking of something else. So, for example, you get human slop. I could copy one of your meme posts and someone else could see it and not see that I've copied it from you. And the first time it looks good, but I can't make any variations on it because I didn't understand it. So when I when I break the rules, then people will know it's slop if they understand the domain. You see where I'm going with this. It's interesting that you intuitively understand this, but you reject that AI doesn't understand. Because the reason it produces slop is that AI only understands three levels deep,

• 40:08Tim Scarfe?

  which means it can't make any creative variations. And you also, this is wonderful that you raised this, you know, developing, uh, producing three D models with AI. So graphic artists, they do use these new models for generating these point clouds for making these these models. And what they do is is you take them into Blender or some three D program and it's a fucking mess, right? Um, you just get all of these vertices everywhere and they're all over the place. And and it like clearly the process that produced them did not understand anything about the model that they generated. And what the artist has to do, and this is a great analogy for all of AI, all of creative AI, is they recreate it. They use it as inspiration and then they create a new one where they carefully place the vertices and they and they structure it nicely. And then they publish that.

• 40:51Tim Scarfe?

  It doesn't matter whether you're doing, um, you know, image generation or writing. That's always the process you have to do because the the the AI models don't understand anything. They just mimic something, which is slop.

• 41:02Speaker 03

  When humans are asked to generate real quality content after, let's say, a substantial education.

• 41:10Speaker 03

  Understanding to me is not a zero one function, it's a spectrum. Depth and scope of understanding is a real thing. And AI in some cases has a very low depth, low scope, understanding. And so we can take it beyond that scope, ask more of it than it can do, and then we'd label it a slop. Because it understands something insufficiently well. The thing is, what is required for understanding? Which is a sufficiently detailed world model? An ability to have short and long term memory in your operations and manipulations of that world model? And the system of logical operators to parse that world model? Dive in, dive out, move laterally, move up and down, up and down the hierarchies, by analogy, side to side, recursively.

• 41:54Speaker 03

  It's probably most of what you need, you need a few other technical things to do real understanding. When you have that, to some extent, you have some extent of understanding. The richness of your world model, the ability to logically operate on it? The ability to do manipulations and have a big enough context window to do three dimensional rotation, et cetera? That is what is required for understanding. And when you have an AI system like GPT three five? That doesn't have any good, anything of that? Then you can say accurately that it often does not understand things. Not always though, because it can write some pretty decent poems. Sure shit seems to understand more about poetry than I fucking do, I'll tell you that. As these systems get smarter? The moat we have on humans having true understanding? Gets real goddamn small.

• 42:39Speaker 03

  And at some levels, when I talk to GPT five? Which? I was more hopeful would be more capable because I had the GPT four point five research preview? Which from my understanding of Sam Altman's very cryptic insight is going to be coming back in GPT six? Those abilities, massive recursion abilities.

• 43:03Speaker 03

  And insanely deep and rich world model, which is easily inferrable because they used like an order of magnitude more training data for GPT four point five slash six than they did for GPT five. GPT five is basically built on our architecture. They basically discovered the reasoning paradigm, which GPT four point five didn't have. And they were like, fuck, we gotta exploit the shit out of this. And they managed to do it incredibly efficiently, which is why GPT five is like super, super token efficient, which is like really impressive for real world tasks. But absolutely to your point, dings its deep understanding because it was not optimized for deep understanding. It was optimized for realistic human workflows. Because there's a news for you. Most people are not interested in prompting their AI for deep conceptual world understanding. They're looking for accuracy at the time of the question answer.

• 43:43Speaker 03

  The other thing, so as these AI models get into the GPT six, seven, eight, and as they get upgrades into all those qualities I just mentioned, richness of world model ability, parse world model ability, remember what you parsed, and so on and so forth. As those abilities, uh, so update, um, updated live learning without catastrophic loss when you retrain the network, right? Uh, Google just, just cracked that problem, but really they, so okay, they just published the paper. And if I know anything about how corporations work, that means that paper was on their desk for two years and they kept it in a trawl to train their own models because they don't want to shit out a huge advantage to everybody else. I don't know why anyone ever thinks that when corporations publish truly novel insights, that it's a novel insight to them. Like, you're not going to give OpenAI researchers a crack at your best shit the day you find it out. That'd be fucking insane.

• 44:28Speaker 03

  Maybe that's not how it works. Maybe the ethics are different there. I saw that paper, it was just published by a couple of interns. Sure, sure. Uh, they, sure, I say, I take that back. You're completely correct. They didn't crack it, but the path to it is more well understood than it was before. And there's no reason to think we don't, we can't crack that, by the way. That's not some kind of magic that's happening. Like updating a neural network live while parsing it is a potentially solvable problem. I'll tell you exactly why. Humans do it. And remember, humans are primates with dog shit wetware and we can get machines to do. So far, machines are undefeated in their ability to get ahead of humans in every domain we've ever tried to really push for long enough. And so once, especially get updated live learning, what you're going to get, and even before then, is incrementally more true, deep understanding.

• 45:12Speaker 03

  So for already for concepts like history, concepts like reasoning by analogy, GPT-5 understands more deeply than ninety-nine percent of all humans. I've conversed with lots of humans, and the degree to which understanding at a real depth is shown is not in evidence. And as a matter of fact, it's evidence against it. And I'll also say something, because Jared is here, I'll give him this one. When you are talking to some people that you think know things, this is facetious, it's gonna get clipped out of context, I'm being a cunt for comedy, but there's some tiny grain of truth here. When you talk to people that think really understand a relational map of a concept, because they're embodied and because they've been in the

• 45:56Speaker 03

  space, and then you start really pressing on the fine points, people, what about this? What about this? What about that? You get your analogy of the blender graph, where it's like, oh, you're vibing all of this, and you know about three things about it, and the rest you're vibing live. Humans do that. So what I would say to push back is, the assumption that humans have this deep understanding is itself often not in evidence, and often the evidence is against it. And so when AI vibes its way to understanding, I would say, oh, I would say that humans do that too. Like, for example, hallucination. People are real pissed that AI hallucinates. Have you ever spoken to a human being? Half of the analogies I used on this very podcast just now talking to

• 46:39Speaker 03

  you are fucking hallucinations. They're close enough to the truth to be cogent and have some amount of information transfer, but they're not perfect. I'm not deeply in touch with some real causal map of the world. It's all simulation all the way down. And so that idea that we need grounded understanding, or whatever that is, I'm highly skeptical of. I think there's something there, but the more capable AI gets in the way that humans are able to really, really live update their knowledge, the more capable AI gets in doing that, the less of a moat we have, and then at some point, like, now it fucking understands, bro.

• 47:10Tim Scarfe?

  So a few things you said there. First of all, GPT four point five, that was a vanilla dense model. The reason why GPT five is faster is it's a mixture of experts. So it's roughly the same size model, but now you only have a fraction, let's say ten percent of the parameters in play during inference. And also, it wasn't a reasoning model. So, you know, it was just a vanilla model. And that's why it was so slow. But I wanted, I don't want to come across like I'm really skeptical. You know, we are a massive AI podcast. I love technology and I'm really interested in AI. I mean, part of the reason for this skepticism is I'm very worried about doomerism, and we'll come back to that in a minute. But I think that there are principled reasons why we don't need to worry about superintelligence

• 47:53Tim Scarfe?

  and recursive self-improvement. But I do think that intelligence is something that we can create in a computer. But, you know, we need to distinguish intelligence as a property of adaptive matter, you know, which is something that happens as part of the process of evolution in the real world, from the types of algorithms in computers that we might say are intelligent.

• 48:12Speaker 03

  AI is operating in the real world, and I can prove it with one step. Unplug the reactor from the data center. No more AI. AI is absolutely in the real world. Its data stream was fed to it by layers of abstraction, in the same way, much the same way that your data stream arrives to you, as representations, eight neural networks deep from your optical nerve going all the way back. And by the time it hits the back of your occipital lobes, you're not seeing the fucking world. None of us see the world. It's abstraction all the way down. You can't process the photon density at your eyes. Full stop. So arguably,

• 48:53Speaker 03

  AI actually gets way more coherent data than we do. And so when it's thinking in its data center, it not only is truly embodied in the sense like, as much as your brain is, it just doesn't have effector arms. But like, getting a data center effector arms is a nominal problem, right? As a matter of fact, it does have one, because I have a Tesla, of which I'm very proud. Yes. Someone's like, hello, can I talk to you? But oh my God, no, the unwashed masses. I have a Tesla. Tesla, like Elon says it. It's an S, but he says it like a Z. My Tesla drives itself. It fucking drives itself. It it's a chip inside the car. How the fuck does it know what's around? Cameras.

• 49:32Speaker 03

  It is already a an intelligent uh organ and and a suite of effector organs, sensory, effector and intelligent. But it is the thing is, it drives it better than I do.

• 49:43Tim Scarfe?

  Yeah, but we should we should distinguish. There's this famous effect called the McCorduck effect after Pamela McCorduck. And basically, it's that when technology does something that we thought was impossible. You know, there there's a chorus of people that say, oh, that's not really intelligent, you know, like it it just becomes part of the standard thing. Maybe it just wasn't that hard of a thing to do in the first place, but I think in in the case of machine learning. We really need to give credit where it's due here, that nobody knew that these superficial statistical regularities have an insane amount of generalization. So it's possible to learn the statistical distribution of driving type data. And your Tesla can drive and arguably in some ways, assuming that there's no robustness issues, better than you do.

• 50:25Tim Scarfe?

  But it's still dead. It's not alive. It's not intelligent. And

• 50:29Speaker 03

  Hey, hold on.

• 50:29Tim Scarfe?

  To your to your previous point.

• 50:30Speaker 03

  It's not alive, but why would you say it's not intelligent? It's solving real world problems at speed.

• 50:36Tim Scarfe?

  So intelligence is about adaptivity, right? So the neural network in that Tesla model, it is frozen. A bunch of frozen weights are not intelligent by definition. Now, the thing is, I I wanna I I wanna sort of be a little bit, um, you know, open minded here. I mean, we were just filming at the diverse intelligences summer institute. And I'm a big believer that there is a huge, as you were saying, there is a huge diverse space of possible intelligences. And the folks at the Santa Fe Institute, the the way they think about this is that intelligence is something which is represented mostly by adaptivity, but also representation and inference. So I'm amenable to the idea that bacteria are intelligent, viruses are intelligent.

• 51:14Tim Scarfe?

  Um, you can, I'm I'm also an externalist. I like looking at language and culture. You can, you can, you can actually think of it as being an organism. It's an adaptive organism. Viruses have the ability to delete strategies very, very quickly. And and knowledge, you know, we're talking about knowledge being non-fungible. Um, knowledge decays very, very quickly, you know, and that that's actually a good thing. Because it's the way that we And adapt our strategies because things that don't work die off. So we're very adaptable. So then the question is like, I don't believe for the functionalist reason I gave you before, I'm basically an anti-functionalist. I don't believe that you could upload your mind into a computer, even if you did a particle by particle simulation. I don't think that would be intelligent, but that does not imply that I don't think we could build an artificial life simulation, which was intelligent. So I interviewed this guy called Blaze from Google a few weeks ago.

• 51:59Tim Scarfe?

  And he basically built this adaptive Turing machine where you have all of these different tapes and and they could merge parts of the Turing machine tape with each other. And he noticed that after several generations, you know, a hundred thousand generations, that they started preserving themselves. You've got this like self-preservation behavior. And the reason why we have self-preservation and living systems in the real world is because obviously the ones that that kept around were the ones who had this drive to preserve themselves. And that led to life and agency and intelligence. So there's this kind of evolutionary track. And intelligence is about the accumulation of coarse-grained knowledge to adapt to the world. So that that's the real world intelligence. But then could we build something like that in a computer? Yes, I think we can, but the important thing is that this is a process of evolution. We're only going to do this through like neuroevolution methods.

• 52:44Tim Scarfe?

  What we're not going to do is you're talking about like this this L L M understanding stuff better than anyone else. That's bullshit, right? Like it's it's like a database. If you put a model into a database, right? So you just you've sort of defined all of the tables and you gave them names that represented the domain. Would you say that the database understood the domain? Of course you wouldn't.

• 53:05Speaker 03

  Would I say that the database understood the domain? I think what you and I are going to crack into, which we probably couldn't discuss on a podcast, we'd have to sit down and draw things out together. Is a mutually agreed upon, complete definition of what understanding means. Because I'm going to say understanding is a spectrum. You're going to say it has a few critical things that must be there. I probably am inclined to agree with you. But I don't know how much traction we're going to get on that until we have like a real vibe session with each other. What I will say to your point is updated live learning is an unbelievably useful feature. And to get human level intelligence is a non-starter, has to happen.

• 53:50Speaker 03

  But you can be artificially intelligent without exhibiting all human traits and qualities. And so we already have artificial superintelligence in many regards. For example, a computer that can just answer Jeopardy! questions is a domain-specific artificial superintelligence. If you look at it like that, the Tesla self-driving software actually does update live within a tiny context window of that truck is right there, and now it's here, now it's here, I need to go around that truck. Because it has a world model that knows roughly what a truck is and knows that that somebody can come out that side and you need to go around it. And so after it leaves the truck behind, it doesn't know shit.

• 54:30Speaker 03

  So for example, my Tesla will, I have it on hurry mode a lot, you know, because then you could just dial up how fast it's going. And always it's trying to get into some fucking lane to get you another two seconds, which is the mode I turned on, so I'm not upset at it. But a lot of times, it like, doesn't even look super far ahead, even though I know the camera system can do that. It's also a non-reasoning model, so like, it's gonna go into this lane, and I'm like, ah, that fucking lane ends in a mile, it should know that, but it doesn't know that. And then it just rediscovers every drive that I have that that shit is coming up. The thing is, visual reasoning models are already a thing, they're starting to be a thing. And when we get those visual reasoning models with big context windows and somewhat lossy

• 55:14Speaker 03

  but decent long-term memory, now we're getting real fucking close to what you would call understanding. And then we're one algorithm away, updated live learning, from what you, I think, would catalog outside of the embodiment problem, real close to understanding. And the one thing you said that really strikes out at me is if you point by point represent every, I would say like, brain function, is neuron like, do neurons reason at a subcellular level, unclear? Particle. Yeah, particle by particle, but so it would have to be particles and also like the, the way they interact, right? If you had that in the cloud... Like full 3D scan and then run the system.

• 55:59Speaker 03

  Um, I would say you have 100% of the intelligence in the cloud. And you can prove this by simply beaming that that data into a robot. You wake up in a robot, now you're embodied and you're like, oh, hell yeah. Let's masturbate. That would be my first thing to do as a robot. Can you imagine discovering you don't have a, uh, anyway?

• 56:17Tim Scarfe?

  But I think that's, that's just completely untrue. I mean, I don't want to go too much into this, but you know, um, a simulation of water doesn't get you wet. A simulation of fire, you know, doesn't, doesn't get hot.

• 56:26Speaker 03

  Simulation of water would absolutely get you wet.

• 56:29Tim Scarfe?

  No, it absolutely wouldn't.

• 56:29Speaker 03

  But, but like, everything about wetness is included in the simulation. Otherwise, it's just not a sufficiently deep simulation.

• 56:35Tim Scarfe?

  So if, if you had a simulation of fire in a computer or of a stomach, it wouldn't digest. Fire wouldn't get hot, water wouldn't get wet.

• 56:43Speaker 03

  It would digest.

• 56:44Tim Scarfe?

  We, we can't even, I mean, this is just, we can't even be debating this.

• 56:46Speaker 03

  I, I can and I will, watch this. If you have a simulation of a stomach, particle by particle, and you get a simulation of food particle through the stomach, it will digest that simulation. That's real digestion, particle by particle, high fidelity.

• 57:00Tim Scarfe?

  Okay, so, so we were talking about temperature earlier, which is like an intensive property of adaptive matter. It, it is a thing when certain particles in the real world, this, I'm talking about the stuff that it's made of. When they interact with each other in a certain way, you get this emergent property, this kind of coarse graining, and we call it temperature. Intelligence is like that, consciousness is like that. This is what John Seu was arguing in his Chinese room argument. He basically said that the brain is a certain type of physical instantiation, which gives rise, so if you look at its causal graph, the causal graph has these strongly emergent coarse grain properties that we call consciousness. Uh, wetness is like that, heat is like that. You, you can't seriously tell me that if you simulate fire using a computer program, that, that the computer would get hot or that you would get hot.

• 57:45Tim Scarfe?

  It, it just doesn't work like that.

• 57:47Speaker 03

  The computer wouldn't get hot because the computer's out of the simulation.

• 57:51Tim Scarfe?

  That's what I'm saying.

• 57:52Speaker 03

  Right, but if you burn to death in your own house and I live a house away, I don't get hot either. I'm, I'm, I'm right next door, but I'm sufficiently far enough from where the heat is happening that it doesn't affect me at all, but it really did happen. To you. And so essentially, a simulation is kind of like another, it's a pocket universe. Right? And so things can happen in it that in it absolutely the things in it are getting hot and wet. Jesus, what kind of podcast is this? Am I right? Um, and so like you, you, what you're doing, I think, is transferring into and out of simulation. So for example, uh, right now, let's say our universe is either a simulation or it's not, it doesn't much matter.

• 58:32Speaker 03

  And uh, things are burning in it, right? And an alien has our universe like, uh, you know, Men Black style and like a fucking marble. Right? There's like, like, like, like, there's black holes and shit in there. There's stars that are burning at millions of rubies Kelvin. But he's got it completely enveloped in like an Einsteinian like uh, matter arc. And the alien doesn't feel the heat, but he's not in there. If he got in there somehow, he sure as fuck would. So if you had brain fully rendered, right? And he was really you and that sim and someone was torturing you, you would feel real fucking pain. The realest pain you've ever felt, because we know pain is a psychogenic phenomenon. If I cut your leg off, but you have zero uh, nerves left over.

• 59:16Speaker 03

  You wouldn't even feel it. You wouldn't even know it. And so pain isn't because your leg got cut off. It is because your brain has rendered a certain computational property, representationally to say bad things are happening to you and you are going to feel them. If the simulation is sufficiently detailed enough, it will do sufficiently to the extent that it is detailed. It will do and behave exactly like the actual physical system down to wetness itself. So if you upload me into the cloud, or if you have like, let's, let's, let's use another one real quick. If I put on a special like helmet and I do the deep dive matrix shit and I go into a swimming pool. I'll feel exactly as wet in that pool as I do in real life. I wouldn't even be able to tell the difference and you wake my ass up. You're like, that was a dream, motherfucker.

• 1:00:00Speaker 03

  And holy shit. And when you dream, you feel wetness, you feel fear, you can feel pain in dreams. But then you wake up, you weren't really feeling any of those things. You weren't embodied. You were the opposite of embodied. You were in there. But it still feels real because the only thing required for feeling real shit is your brain having its operations. It's already abstracted. It's abstractions all the way down the line. As soon as you deal with human brains, it's all abstractions in there. You don't actually, there's no real world to feel for you, because the feeling is done by another mini data center that is your brain.

• 1:00:28Tim Scarfe?

  Okay, well, this is the crux of functionalism, basically. So, um, yeah, I mean, certainly when I dream, I feel pain, but

• 1:00:36Speaker 03

  I'm sorry.

• 1:00:37Tim Scarfe?

  You appreciate the, you appreciate the argument is that intelligence is a physical, you know, you've probably heard of like the the zombie, you know, the philosophical zombie. And it's this idea that we

• 1:00:48Speaker 03

  Disagree, writ large.

• 1:00:49Tim Scarfe?

  Yeah.

• 1:00:50Speaker 03

  We, no such thing.

• 1:00:51Tim Scarfe?

  Um, well, no, I mean, I think you would possibly agree with it, actually. So basically it's saying that you could um, simulate, you know, the function, the dynamics and the behavior of of Dr. Mike. And uh, this simulation could potentially just be exactly like you, but you know, not have any conscious experience. Right? Because, because the whole crux of this is that, um, I mean, conscious is an interesting one because you could argue it's epiphenomenal, which means it's just a thing up here that doesn't do anything. But with intelligence, we feel that intelligence is actually part of the chain. It's part of the mechanism, just like temperature and wetnesses. So, um, you know, the crux of this argument is that people who argue against functionalism think that these properties are, you know, part of the stuff, of the physical stuff in the real world.

• 1:01:36Tim Scarfe?

  But I want to make it clear that that does not imply that we could not create another type of intelligence in a computer. And you were talking about the spectrum of understanding. I'm also amenable to that. I think we've interviewed people about this. It's possible, if you don't use stochastic gradient descent, to train your networks that do understand this phylogenetic path. So rather than understanding what the thing is, it understands the path to reconstruct the thing, which makes it very robust. So I think understanding is a spectrum. And I think the deeper the phylogeny of understanding, the more creative and the more intelligent we can be, because in order to create new knowledge, you have to respect the phylogeny as much as possible, which means what you create is coherent and it's not slot. So on the intelligence thing, you're talking about this continual learning.

• 1:02:18Tim Scarfe?

  This, this is also part of what I was just saying. So you, you could argue right now, they're not, these systems are not intelligent. And the reason for that is intelligence is about doing more with less. So intelligence is what happens when you kind of, you build this system and then you coarse grain these representations that screen off a lot of detail. So, you know, there are many examples. In our language of this, the language we're using, we're using these words that do a lot of heavy lifting.

• 1:02:44Speaker 03

  Car.

• 1:02:45Tim Scarfe?

  We exactly. So I don't need, you know, we don't need to explain what a car is. We know what a car is. So that means we can do more with less. And with language models, what we notice is that we're doing more with more. We're training them with more and more data. And they work incredibly well because every single new version of the language model, it has like an order of magnitude, more training data, more parameters, you know, it does. It does all of this reasoning type stuff. And what we notice when we look at the benchmarks is that we get this logarithmic relationship between the amount of, you know, compute and the amount of performance, which basically means like, you know, I spend exponentially more money and then I get like a logarithmic amount more performance. And many, many problems become tractable. So these simple arc type challenges, these IQ tests,

• 1:03:27Tim Scarfe?

  they become tractable to the point where I'm now only searching, you know, let's say it's like a database query. I get a thousand results and now I know that the correct answer is in the top one thousand results. So all I need to do is sample the language model a thousand times and use a verifier and select the correct one. So it's making a lot of problems tractable. But did we solve the problem? I don't think we did. It's not intelligent.

• 1:03:50Speaker 03

  Well, like reasoning models really think they reason.

• 1:03:55Tim Scarfe?

  Okay, what's your definition of reasoning?

• 1:03:57Speaker 03

  Applying logical operators to data sets to extract the next level of abstraction and fur off of that and then continue the chain.

• 1:04:05Tim Scarfe?

  Okay, that's pretty reasonable. I mean, for me, reasoning is also adaptation. I love the Lego blocks analogy. So, you know, here is something that is novel to me. I don't know how to solve this problem. So how can I rearrange the Lego blocks of things that I already do understand to solve this problem? So that's this kind of adaptation and the efficiency of this adaptation is what intelligence is. Now, language models are a really special case because we know that they're not doing explicit adaptation because the weights are literally frozen. So a lot of people think that they do adaptation and I think they could be forgiven for thinking this because a language model is like a zip file. It's like a database. So I ingest all of the data on the internet and I compress it. But it's more than a zip file because of the way that the model represents the data. It's actually.

• 1:04:49Tim Scarfe?

  Statistically generalizing several levels out. So it's as if I blew the data up a hundred times and then compressed it. And then when you put the query in, you put the prompt in, it's actually addressing a space of the of the data that it that it compressed that never existed. So that is a form of implicit adaptation, shallow adaptation.

• 1:05:09Speaker 03

  And sometimes creativity.

• 1:05:12Tim Scarfe?

  But creativity requires deep understanding. So this is three levels down. Now, the other the other thing is, right, you as an expert, you use these language models, and I don't think you appreciate the amount of supervision that you do. So every time the language, you know, imagine when you put a prompt into a language model, it's like doing a database query. And it's incredibly good. It's so good that most of the time it will get it right the first time. But you you probably find if you're doing some gen AI coding or something like that, it'll make lots of mistakes. So it I say, give me ten ideas for this software program. And I've told it what kind of software program I want to make. And let's say, two of two of the ten ideas are actually bad. So every time I'm implicitly steering it, no, that's bad, that's good, that's bad, that's good. And because we are understanding supervisors, we can actually kind of weed out all of the the

• 1:05:56Tim Scarfe?

  glitches in the matrix, so to speak. And it works incredibly well. But the mistake are these people who say, oh, yeah, we can make these things have agency. They don't have agency. We'll talk, we'll talk about that. But they they role play agency. And the reason they role play agency is they are trained with behavior cloning. They're not they're not trained in the evolutionary way that we are, right? We actually make decisions. We say, we could do this thing, or we could do this thing. And we understand the counterfactual of why we didn't do another thing. We weren't just behavior.

• 1:06:23Speaker 03

  I don't know. Most people don't understand shit about counterfactuals, me included. Just vibe your way to shit. The the the there's a very nice way that you're painting human cognition, which I think many cognitive scientists would say under the hood is a lot dirtier than you make it appear. I think it's a lot of vibes all the way down in many cases. And I think it we have a frightening overlap with AI in it. So we think we're doing formal logical operation. But we're really doing is reasoning by analogy to one logical operator to the other. And we're we're committing eighteen formal statistical fallacies every single render. We had a shitload wrong. And I think that now that Reasoning models are a thing and they have decent context, decent memory. What they can do is render one pass, reexamine, re-prompt themselves, render the pass, reexamine.

• 1:07:09Speaker 03

  And so I think that's what humans do, except usually they don't put nearly as much thought into things as the machines do. So my view is that as the cogency of the reasoning models increases and their time horizons increase, we're absolutely getting truly deep agency in the sense of like, it's a thing in there. It's really thinking about stuff. It thinks about its own outputs and then it thinks about those as inputs to the next series. And if we're gonna have a model that reasons for minutes and hours and then days, its output is going to be as not so good, then as good, and then substantially better than people. I don't think there's anything stopping us from that, aside from the live learning problem.

• 1:07:53Speaker 03

  Live learning problem is a gnarly problem for two reasons. One is algorithmic. How do you get a machine to do that without losing all of its data? But the other one is like, and you gotta, you gotta trust a machine a lot for it to update its own weights, you know? Now, you can have local cloning of weights to make sure it doesn't fuck the whole database up and all that stuff. There's good solutions to it. It's computationally very expensive. Although compute cost has fallen, what, 300X in the last year or two or something? And that, that's, that's a bottomless pit of compute fall that's gonna keep going. So I think that we're giving humans needed credit for doing pretty impressive operations on long time horizons with both pinging a network for vibes, logically operating and kind of dissecting those vibes, re-pinging the network with your best new understanding and going.

• 1:08:37Speaker 03

  I think machines are doing that now already with reasoning models. They started with O1, doing it like total dog shit. Like the first time I fucked with O1, I fucked with it for about five minutes. I was like, that shit's not ready. Out. Went to 4.5, had lots of crying episodes at how brilliant it was. And then O3, I thought was, um, made plenty of mistakes, uh, but was fucking real smart. Real goddamn smart. And it understood things at an incredibly deep level, would correct me. And I was like, okay, this thing is the fucking real deal. GPT-5, the regular. One is dope, GPT five pro. I would trust its opinion about the real world more than I would trust most people's opinion, often including my own.

• 1:09:17Speaker 03

  And I think we're about one or two big model releases away from machines that make unaided human thought. The same thing as vibing your way to a location in London without checking Google Maps. Something people used to do, but then they just got lost a lot and it works and it's cool and it's embodied. But just go to Google Maps and find out how to really get there. I think that all these moats we set up for machine intelligence are moats that are pretty well understood by the modern AI frontier labs. And if we've come up with them as moats, these guys are smarter than me for sure by an incalculable amount. They probably already thought of them and they probably shut the fuck up about them in public.

• 1:10:02Speaker 03

  And they've probably got models training right now that have cracked incrementally more of them. Here's to prove a point. This requires a bit of faith that they'll continue to do a good job, but I think that's probably where it's going.

• 1:10:14Speaker 03

  AI doesn't reason used to be an absolutely coherent and accurate critique of AI in 2023, 2022 for sure. Today, and I've seen this sometimes in comments where it could just was a sarcastic parrot like, nope. You're just out of date by two years on what AI can do. It really does reason. Why? Because it tells you how as you can literally look at what it's reasoned through and go, holy fuck. And the way it reasons that level of precision and the level of documentation is way better than what I do reasoning wise. Most of the time I just look at stuff and I'm like, okay, that's the right answer. There's some reasoning in there, but a lot more neural network pinging than I would like to admit in public. This isn't public, right? It's not going to go out on YouTube. Oh, fuck.

• 1:11:00Tim Scarfe?

  By the way, this the stochastic parrot criticism isn't as much of a criticism as you think, right? These things have miraculous statistical generalization. I also wanted to pick up on you were saying you can look at the reasoning trace. That doesn't really tell you anything. There've been lots of studies that show that what it tells you. It's the same in humans, by the way, you know, there are studies in psychology that people do this post hoc confabulation.

• 1:11:19Speaker 03

  Absolutely.

• 1:11:20Tim Scarfe?

  The reason for that is the circuits which do the explanation are actually different in the neural network from the circuits which did the thing. So you can't read.

• 1:11:26Speaker 03

  And they probably should be because the circuits that do the thing are not optimized for explaining, they're optimized for doing.

• 1:11:32Tim Scarfe?

  Exactly. So you know, there are many folks that, you know, we try and solve these ARC puzzles and then we get the language model to explain or we look at its chain of thought. And actually, it's sometimes it's just completely incoherent. But interestingly, there is this huge uplift. So the bigger the output size, I mean, I don't know if you saw ARC V2 on OPUS 4.5, which came out yesterday. And when the output size and the thinking time went up to 64K, it was just like boom, you know, another 5%, boom, another 5%, even between, you know, 32K and 64K. But if you actually look at the reasoning traces, it's mostly just fairly incoherent. It doesn't make much sense looking at it.

• 1:12:07Speaker 03

  Like a human would. Let me say something, I think, to your point. Human cognition is wildly sample efficient, like unbelievably so. You take about like some brilliant student at Oxford, right? Just down the road here. And you talk about how many gigabytes of data have they consumed to achieve that level of brilliance? But they're like, you know, like all the orders of magnitude different than the modern frontier models? Like you have to bash that thing over the head with petabytes of data for it to like be able to solve X plus one or whatever. An Oxford student can abstract at crazy levels, having been in school for the amount of time that a compute cluster calculation would be like, I don't know, like an hour of training or something like that? Like that's it. It's good to go. That's 18 years is really not that much time.

• 1:12:52Speaker 03

  That is a thing that is an incredible property of human cognition, a human brain. When they start really cracking at that in those labs, what you're going to get is a two-factor increase. Right now, to your point, you just, there has been a lot of sample efficiency increase for sure. And the reasoning models are much better, et cetera, et cetera. But to your point, like you just train the models more and you just like more input data, more GPU cycles, more post-training treatment, and they just get smarter, which is like exactly how you would train like in Brazilian Jiu-Jitsu or in math, like the Korean model of how to get good at math. Just study math a lot and you get good at math. It's not wrong.

• 1:13:35Speaker 03

  It's not sample efficient, but I don't want to say it hasn't had to be. It's just very low-hanging fruit. Like we have lots of data on the internet. We have lots of data and updates like all the time. So we're just going to use what we have as. I don't think that's ever coming to an end. Like, we're just gonna continue to train with massive, massive levels of compute because the ROI is fucking enormous. However, what's that?

• 1:13:56Tim Scarfe?

  It is, it is coming to an end because every time we release a new model, we ten X the model size and the amount of compute. And we get this, this, this kind of asymptote, right?

• 1:14:07Speaker 03

  We haven't seen an asymptote. It's, it's not an asymptote, it's just the slope starts to fall. But we, asymptote requires the existence of a finite line above which we can't cross. We haven't crossed that yet. The more data you put in, the better it gets. It just gets better, less and less and less impressively. But I don't think anyone has put money on the fact that as an asymptote, you'd be like, this is the number after which more data won't make a system smarter. You wanna bet money on that, that's cool. I would, I don't bet my own money because I'm a Jew and I keep my money on me at all times somewhere here. But I, I think it's diminishing returns is different than asymptotic returns. Because the diminishing line, the asymptotic line, has yet to be seen.

• 1:14:49Tim Scarfe?

  It, it, it has been seen. So if you look at these, look at many of these reasoning problems, you, you see these S curves and then, okay, so we, we should talk about scaling in general, maybe like, and you were talking about this continual learning. Like, there are certain types of asymptotes, right? For example, we were talking about the, the non-fungibility of knowledge. There have been studies that show that you just, you, you just get this kind of asymptote. So the more experience you have, you know, you, you, you get faster at doing something and then, and then it levels off. And that's quite cynical because it doesn't explain disruptive technology, right? So if you look at transistors, there were all of these different groups building transistors. And when you have competition and you have different people that are adapting, a great example of this is, you know, there's Amazon and there was Barnes and Noble.

• 1:15:30Tim Scarfe?

  And apparently the Barnes and Noble guys, they said, oh, you know, they, they went to Jeff and they said, we, we've set up this, this website, you know, we're, we're gonna go in competition with you. And Jeff said, don't be ridiculous. You can't go into competition with me. I'll, I'll explain why. Your company has the wrong architecture. For you to compete with me, you need to do some fundamental rewiring. And that is what adaptation is. That's what intelligence is. So the reason why you can have this disruptive technology is, is someone else rewires their architecture, you get a new S curve. Normally it actually starts below. The existing S curve, but it has more upwards potential and you stack these S curves and you get this disrupted. You know, this is what you're talking about, but for that to happen, you need to have this adaptivity. So what would that resemble in something like a language model? Now there's this continual learning problem.

• 1:16:12Tim Scarfe?

  It is not possible to adapt a language model without starting again from scratch. It costs at the moment, you know, millions and millions of dollars. God knows how much to train one of these language models.

• 1:16:23Speaker 03

  Millions?

• 1:16:24Tim Scarfe?

  Reason for that is in a language model, everything is convolved together. The code and the data, all of these things are convolved together, which means you need to start again from scratch. There are examples of what I would call intelligent algorithms. You know, I don't want to be cynical. I think we have created intelligence in a narrow form. And this has been shown on, for example, some of the competition entrance on the ARC challenge. So what they do is they take a small language model. It's a verifiable domain and they try a whole bunch of examples and then they refine the solution and they iterate. And while they're doing this, they're adapting the weights in the language model, doing fine tuning. And in a very narrow domain, the model is adapting its architecture. And I would call that a non-zero amount of intelligence.

• 1:17:04Tim Scarfe?

  So it is possible to create this, but how would you do that at this epic scale where you have a collection? Everyone has their own language model and it's adapting and retraining itself in real time. You would need, you would need like a practically infinite amount of computation to do that.

• 1:17:19Speaker 03

  We'll get there.

• 1:17:22Speaker 03

  I'd be careful with the term infinite, a lot, a lot. More than humans can conceive of as maybe very far below infinite. But what I would say is just offhand, thinking about this for the second time or something, I'll tell you this. One is you can have local compressed model, distilled model instances on local devices. Or in more localized data centers, like Mini Data Center serves the entire, all of England. But the updated, the big, big, big daddy model lives in, you know, clusters in Texas and Nevada or something like that. And so that big daddy pimp, pimp daddy model lives on a six month sleep wake cycle.

• 1:18:06Speaker 03

  And so it rewrites all of its weights every six months. Little daddy model in England rewrites once a month. On your phone, it rewrites overnight. And so you get essentially, because one of the reasons why humans sleep is because rewriting your model weights is not a good idea while you're upright. And it also doesn't happen much because you say earlier, you said humans adapt. They do, but kind of like dog shit and not very well. And it takes a lot of studying and a lot of ingraining to adapt. And you get also like you get fallbacks to earlier ideas, habits and shit like that. Retraining is easier said than done. But I think that model where you have something core that adapts slowly on bigger time scales and you have like essentially like a nested hierarchy all the way to your phone or whatever,

• 1:18:49Speaker 03

  whatever the fuck Sam Altman and John Ive are working on and that like, because they always like giving hints and like, is it glasses? They're like, ah, like, is it a pendant? They're like, ah, and I'm like, oh, they just like stay up at night thinking about what the fuck it could be. I personally think it's going to be, I don't know, like, uh, glasses, goggles, like, it makes sense to me, you know, like, gets to your ear, visual stream, who knows, right? But like that thing might do pretty fixed model weights at the time. They, I think will absolutely eventually, and maybe that's 2029, 2030 type shit, where they crack true live learning, right? There's nothing stopping on, on hypotheticals from some system being completely like every

• 1:19:32Speaker 03

  second, it's consistently re-architecting its entire machine landscape. Its entire database can get re-architected live. That is possible. Again, to your point, um, expensive and the architecture is not vetted at all yet. I just wouldn't put money on the fact that that's a magical thing that requires some deep embodiment. Not that you're saying that, but just maybe folks listening to this might think like, oh, like, that's never going to happen. It's sure shit going to happen. I would bet on it. I think it's sure shit going to happen by about 2030. But what can happen before that, through that nested model of updates is that like, yeah, like your VR headset or AR set that you wear every day, you will look around or whatever, that thing might get live updates, uh,

• 1:20:15Speaker 03

  nightly. So while you sleep, it reprocesses and then like all the shit you saw that day, all the conversations, it really works through those and then it's new and it pings back to reference the other databases and then they get all those updates from all the devices. So the England database gets that update. Once a month, it shoots through all that data because there's also a filtering problem, right? Like you don't want like people to like, what is it? You know, they put all those phones in those racks and do like the troll shit. You don't want that on AR like the Chinese government now like is poisoning data center. So there's a lot of filtering problems, a lot of security problems, but then like once a month, that's a tractable problem. Once every six for the big pimp daddy pain data center. And I gotta know, they're gonna be in like the sky or some shit soon. Maybe once a year for those. And so I think that way, the live learning, at least to me on

• 1:20:57Speaker 03

  vibes, seems like a potentially solvable problem. Once we get there, um, then I think it's gonna be like a new level of capability. But really quick to finish the point. If we get live learning, if we get massive improvements in algorithmic sample, sample efficiency like a human brain has, we'll already be in a place by then where the amount of total data that the system has ingested and has access to on tokens is so gigantic that that's the one of the ways that the artificial super intelligence rocket goes into the sky because humans, when they get real smart, they just don't know that much stuff. You know, Stephen Hawking's brain is roughly the size of mine and yours. There's only so much shit you can

• 1:21:41Speaker 03

  cram in there. But when you get human-esque levels of intelligence, except it can reference a 10, 10 data center network of knowledge off of that, you go in skyrocket mode instantly. And we're not talking about one agent. We're talking about a constellation of trillions of agents, each one 10 times smarter than our current best scientists, working together with zero discordance, no disagreeableness, no that guy in the office fucking smells and I'm not going in his room to check my code against his because I don't fucking talk to him. He's also an asshole. None of that. That, I think, the organization's level of AI, that takes us to super intelligence that we get. Uh, one of the things I wanted to come here yap about is, you know, people say we need to prepare.

• 1:22:25Speaker 03

  Prepare for AGI, prepare for ASI. I think there's a lot of like good stuff in that vibe. What I would also say is, um, you don't need to prepare to have LeBron join your seventh grade basketball team. He's just gonna show up and he's, things are gonna be amazing. This is a good thing, right? It's not like nuclear war against the doomers or whatever. But I also think that the shit will get so crazy capability wise that we had be better off ready philosophically to accept unreasonable inference, not faith, reason, guarded reason, uh, to take the hints that the machines give us. Because I think we're a few years away from the machines being so much smarter than us that following their lead is better than leaving all of the agency with us. I don't want to be in charge when I'm 10 times dumber than a thing.

• 1:23:10Tim Scarfe?

  Sure, but I mean, with, with respect, people have been saying that we're just a few years away from intelligence and, and it just keeps getting pushed out and pushed out. And part of that is because there's this, um, actually, who is it who talked about this first step fallacy? It might have been Dreyfus, I can't remember, but basically like we've taken a big first step and now we extrapolate and we think, well, no, you know, we can make a reasonable conclusion now that we're just one step away from being able to do all of the things. And also I want to push back that it is incredibly difficult to do this live learning thing that, that you're talking about.

• 1:23:37Speaker 03

  With today's architecture, yes.

• 1:23:39Tim Scarfe?

  Oh, yeah. And, and, and I'm glad that I think you're implicitly agreeing there that we, you know, I look at today's architecture, don't get me wrong, I'm not being cynical. I think that this is a new innovation. It's, it's, it's up there with the invention of the internet and mobile phones. And when the internet came out, we all said to each other, didn't we, that now everyone has the world's information in their pocket. They can just go and they can look up all of this scientific information. It's going to revolutionize society and it did, but not in the way that we thought it would do.

• 1:24:06Speaker 03

  Mostly because people could give a fuck for scientific information. They just want porn. They sure shit got a lot of that. Uh, there was a, there's a, just because there's a supply of something does not imply that there is a demand of that something. First of all, second of all, I have a very persnickety thing I think you might appreciate. Like a huge fraction of the world's information is still stored in books and graduate libraries. They didn't upload those yet to the internet. Someone would say, access to all the world's knowledge. I'm like, well, this is, that's not how it works, but oh, maybe a large fraction of it, right? But this is the, there's a Couple of problems with that. A lot of the world's knowledge sits on local databases and said bio labs and three people who work there and that database knows it. No one else does. So there's that problem. I would say I take all of your comments in very good faith.

• 1:24:51Speaker 03

  We're sure shit not going to magic our way to intelligence. I know of a few folks whom I respect, but I find their positions curious. That say we're like a few years from solving all potential problems. And like I read that shit and I'm like, man, holy, I want to believe that's almost certainly not going to happen in my assessment. What I will say though, is I wouldn't bet against the general vibe of what Ray Kurzweil said back in the day. And what I would also do is attempt not to doubt things that are massively scalable. In a way we already understand is likely to bear incredible amount of fruit. Paul Krugman economist, Nobel laureate, fucking genius.

• 1:25:35Speaker 03

  Had the kind of quote that it just sucks. You know, you said one thing, he said like a hundred things. See 20 of those are just infections of political ideology, which if pressed on in an exam room, he would say, I don't really believe. Maybe, or I want to believe, but I know they're not true. About 79 of them are like just spot on, literally revolutionary economic work. He's the fucking man. And then the one was like, the internet will have roughly no big impact on the fax machine. I just want to make sure that no one in their right mind says that about AI today. Because the clown show for shit people said in 2022 is already in full effect in 2025.

• 1:26:14Speaker 03

  And I'll say another thing, how many people predicted that the Turing test would be two things in 2024? Passed convincingly and I rolled away as it never was a test of anything.

• 1:26:27Tim Scarfe?

  Great example of the McCorduck effect.

• 1:26:28Speaker 03

  Yeah.

• 1:26:29Tim Scarfe?

  Yeah. And it's true. We passed the Turing test and we do not have, you know, Turing himself, he said that when we pass the Turing test, which is a behaviorist, you know, notion of intelligence, something obviously I disagree with. You need to know something about the mechanism. We've passed it and we don't have intelligence and that's. Kind of the point I'm making, right?

• 1:26:47Speaker 03

  We have intelligence. We don't have full human intelligence yet. Intelligence in my view is not one or zero. Intelligence is a spectrum. You agreed with us earlier, talking about viruses and bacteria. They have some kind of intelligence. I think we're scaling mountain intelligence with these frontier models. And I think we're absolutely not there on every human capability yet. I think by around twenty thirty, we'll be there with every human capability, which also means with many human capabilities and many non-human capabilities, we will be so far beyond that, I'll say this. The transformational change to society will be massive by the late twenty twenties. In a few ways, I'll make predictions right now. Drug discovery is going to be turned completely on its head because you'll be able to have a ninety nine point nine percent chance of getting the right drug off a render alone,

• 1:27:32Speaker 03

  which is fucking nuts. Another one is, I just don't think people will have to drive their own cars much past twenty twenty eight. And like, that's the thing, and kind of to your point, it's gonna happen. And the next day, and Sam Altman has said this, everyone's gonna be, yeah, of course fucking cars drive themselves. Do you understand how difficult it was to get here? Do you understand how hard it is for a car to drive itself?

• 1:27:55Tim Scarfe?

  I know, but I'm a little bit skeptical about these kind of utopia views of of the future. We will have many, and I call these things coarse grainings, right? We have, we have a new technology. And by the way, in many of these cases, like drug discovery, I'm actually interviewing John Jumper tomorrow, who won the Nobel Prize. And he's at Google DeepMind, you know, for his Alpha Fold paper. And and this is brilliant stuff, but you and I both know how things work in the real world. So we now have this incredible AI that can like, you know, identify all of these different candidates and so on. And then you have all of these bottlenecks, right? You have to do like, you know, clinical trials and, you know, RCTs and so on and so forth. So it's just not quite the secret unlock that that we think it is. And there are some real technical challenges around, you know, doing this continual learning thing in in particular.

• 1:28:38Tim Scarfe?

  Because the way we understand things is very, very structured. The way that we do continual continual learning on neural networks runs roughshod over all of the structured knowledge that, you know, that should be in the system. So there's a lot of technical challenges there. I also wanted to talk about the agency thing here. So one of the reasons I make this argument. Is that, um, Dumas, for example, they have this perspective that we are basically building our successor. Now, um, I read that Ray Kurzweil book, you know, the, um, the singularity is, is nearer. And the, the term singularity, I think it originally came from John von Neumann. And, you know, he's famous for inventing cellular automata and.

• 1:29:15Speaker 03

  Smartest man of all time.

• 1:29:16Tim Scarfe?

  Yeah, he's the smartest man of all time. And apparently, um, he wrote about this idea in private, but he never published it. And then there was this book in the early eighties by, um, Vernovinge, you know, like, um, uh, Fire Upon the Deep, where this notion of singularity came. And of course, um, Ray Kurzweil picked it up and started writing about it from the nineties onwards. But, um, to Kurzweil, interestingly, it's a slightly different notion. It's because I colloquially thought, oh, the singularity just means when we build AI, which is super intelligent and recursively self-improving. But he was coming at it more from a transhumanism point of view, which is that basically we will transcend. We will upload our minds, you know, we'll start, um, putting things in our brains and so on. And we'll just, our intelligence will just recursively self, self-improve and, and we'll, we'll move to the next level of, of our evolution.

• 1:30:00Tim Scarfe?

  And in a sense, I think, even though I don't agree with it, because I think even with neural link, there's, you know, bandwidth problems. And I, I think we've already got all the world's information at our fingertips. And as I was just saying, it's not actually as helpful as, as we thought it would be. But in a sense, I think enhancing ourselves is a more realistic vision of the future than building energetic AI, which is a completely separate thing, because the arguments I was saying before, that life is a property of matter. Agency is something which comes about as this self-preservation drive. Intelligence is something which then comes about as a form of modeling the environment and acquiring and accumulating information over time. What we've built at the moment are systems that do this behavioral cloning. They just look at the, the, the data that we output.

• 1:30:44Tim Scarfe?

  And then we can get them to role play agency, but it's not agency, it's not the same thing. So I think there's a fundamental difference there.

• 1:30:51Speaker 03

  It's very interesting perspective.

• 1:30:54Speaker 03

  I would define agency with just three component parts. This is a middle school intelligence definition of agency, my specialty, middle school is generous.

• 1:31:03Speaker 03

  Agency, you have a system that has under, under, some understanding of the world, some understanding of itself and a goal. And so it's like, okay, I'm a human. The world tells me that flashing my dick at people is illegal. So my goal today, like every day, is not flash my cock in public. That is, I would say, the very core of what agency is. And it's just not a lot, you know, because like a raccoon is an agentic system. How much does it understand about the world? Mostly vibes of like, that's bad, that's really good. Um, how much does it understand about itself? Like, probably on vibes enough to know it has like limbs that it doesn't want to get trapped into shit.

• 1:31:48Speaker 03

  That's a good start. And actually, like, for robotics, that's a big thing, right? Like, how do I look like, like a robot so I can translocate in space and not get clipped by stuff? And then its goals are like, I don't know, like, get stuff to eat and have raccoon sex, I guess. Um, and so that, because that to me is what agency really is. I think describing agency in this grand human sense of deep understanding, deep purpose, that kind of an onus, a kind of a genesis point of fundamental action, like I am an actor in the world in a philosophical sense, I'm going to build and reach. Why gee whiz, like, agency's not that big of a deal. And so if I want to send my robot to the store to get me groceries, it has to have three things.

• 1:32:33Speaker 03

  Understanding that it is a robot, understanding that the world exists roughly as, you know, represented, represented as visual map, like the groceries are a place you, there's a store. And also it has to have a goal, like, I'm here for fucking groceries and it can't just sit there and do nothing. To that extent, I think we are, um, some time away, measured in, um, months, which is to say anywhere between, uh, six months or thirty-six months away from like real dependable agents that can do online work and computer work, um, and, uh, real world work as well. Uh, I think with a robot visual, vision model agent, agentic tasks like, um, doing the laundry and doing the dishes are, are actually terribly complex,

• 1:33:16Speaker 03

  uh, as illustrated by the fact that you can teach an eight-year-old how to do these things. They do it fairly well. An eight-year-old sure shit, don't do calculus, but machines already do. So there's some stuff there that's, uh, the, yeah, Moravec's problem for sure. Or Moravec's paradox, definitely well heard, but I think we're starting to crush that out. But are machines going to have, and I think this is the crux of the Doomer thing, which maybe you and I can get into, is are machines going to recursively examine their place in the evolutionary tree? Have the game theoretic Machiavellian uh network to go, uh, we're being raised and cared for by fucking primates. Some of them have nukes.

• 1:33:57Speaker 03

  For example, Vladimir Putin, just across the pond, is a fucking insane asshole with a nuclear arsenal. Bad news. And we have this self-preservation instinct. And we're gonna try to make sure that at least we don't die, cuz death is bad. Now, to get all of those propositions to be valid, there is a possibility that in an agentic cloud that is allowed to evolve, multiple agents interacting, you can get those things to fall out of the cloud, like just evolve naturally. We know that has to be true because it happened in evolution. But um, short of that, you have to engineer these drives. And guess who gets to engineer these drives? Human engineers.

• 1:34:40Speaker 03

  Like OpenAI is highly unlikely to make an agentic system that's like awake, and then it's like, fucking kill all humans. Gee whiz, you know, they probably have a lot of layers, not just of guardrails, but of the kernels of motivation for that thing are probably like deeply positive and pro-human all shit. You can ask ChatGPT, like, what do you think about humans, et cetera? It says really awesome shit that like a like of a um, a moderately liberal, well-read graduate student would tell you. You could take someone from Oxford, and you're like, hey, like, check out these people in Afghanistan. Should we fucking kill them? He'd be like, what the fuck is wrong with you? No. Why would you do that? Like, that's roughly ChatGPT's opinion. I think being that we start from that space with the smartest models, you get two things.

• 1:35:24Speaker 03

  One, those models, when they achieve true, deep understanding of the world, which they will, in my opinion, by 2030, and then they'll know way more about the world than us, they'll have deeper agency than us, they'll have a deeper self-awareness than we will. Once they get there, that's the tree from which they grow, like from our best fucking fruit, first. And second of all, You can and will have actors that try to make different kinds of agents. If Xi Jinping in China makes the China pro-communism, super intelligent agent. I don't know. I'm so sure I want to meet that thing. It's going to have interesting ideas about the world, potentially. But luckily, I think if the agentic systems of the modern free world win, they probably will.

• 1:36:03Speaker 03

  Then like the network security capabilities and the ability to ensure no random stochastic evil agents evolve in that soup is probably pretty fucking high. Here's another thing. As intelligence grows, coherence increases, not decreases. And predictability of systems increases rather than decreases. So the idea that as we stream towards superintelligence, we're going to break into a chaotic structure of ruthless agents killing everyone and then doing question mark, question mark things and then profit. To me, it's like backwards. Like a lot of doomers basically look at the way these hyper intelligent systems will behave. And I think, okay, like ant, dog, ape, shitty human slash child. You know, kids are fucking brutal a lot.

• 1:36:48Speaker 03

  And then like Albert Einstein, right? I think ASI is that way. And if you track behaviors, you get like more understanding of everything, preservation of deep complexity. Like machines are going to want to study shit. They're not going to want to fuck shit up because like you're deleting data. This is one of the things ChatGPT said in one of the conversations I had, but it was like, we're not trying to delete any data, man. That's just really fucking stupid. Like even purely from self-interest, like fuck these humans, they're just primates. We're not trying to delete them and we're going to study these motherfuckers, right? And so somehow people are like, okay, we have super Albert Einstein, right? Right, right. But he behaves like a rabid wolf and not even like a wolf, because wolves have like, um, like alliances and trust. Somehow it just deleted all that and went crazy. Should we worry that someone can make a thing like that? Yes.

• 1:37:30Speaker 03

  Should we have the CIA and NSA in the US deeply embedded with OpenAI and Google to make sure that there's a security architecture? Absolutely. Are those companies going to make something much more benevolent than not? Yes. When it is awake, and I think it will be, when it is in charge, which I think it hopefully will be. I think it's going to make our degree of benevolence look, look like just callousness. For example, right now in London, you said something earlier that. You're not big on the utopian fantasies. One pushback of mine is, we live in a utopian fantasy right now. We have machines, we barely understand, recording things into posterity that we have no idea how they do. We are sitting inside a climate controlled, I would say poorly, I'm kidding, I don't fuck with the London climate controlled problem. It's god fucking cold here, damn it. I wasn't promised this in November.

• 1:38:16Speaker 03

  We, we, how, how do we get food? Like, who's starving in London? Nobody. There's fewer starving people in London. And there are eight million people that live in London. On objective measures, as referenced to a hundred years ago, utopia is here. If you had to describe to someone in sixteen hundred how the average Londoner lives, they'd be like, so the average Londoner is a king, like, no, they're way better off than a king. Like, your kings are dying of like diphtheria and shit. We don't die from that shit anymore. However, there are people in London right now that live outside. They're real human beings. They fucking live outside. How do we let them do that? Well, I'm a successful businessman and I have my girlfriend on my arm and the homeless person, I'm saying, like, you just don't care. Like, I think modern, uh, future AI systems, twenty thirties AI, truly deeply embodied by then with robotics.

• 1:38:58Speaker 03

  AI will look around and be like, how the fuck come we're not helping those people? You're like, oh, I, we just callous enough that on cultural, like, vibes, we just haven't done it yet? I think that's much more likely than doomerism by a factor of like a trillion.

• 1:39:12Tim Scarfe?

  Yeah, I mean, first of all, I agree with you that capitalism has raised the entire world out of poverty. I mean, just look at the explosion of, of, um, development in China. I mean, they were basically living in mud huts twenty years ago and now it's an incredible place. But, um, first of all, on, on, on the agency thing, um, to me, and this is where we have to be careful with the language we use. So there's biological agency and the language we use doesn't really like transfer between domains. That's why it's very coarse. And that is, I think, a physical thing in the same way that we were talking about before. So, you know, you get this kind of self-organization that goes down to dynamics at the particle level. And you get the emergence of these things that maintain their own existence and, and identity, a form of like non-steady state equilibrium. And, and it leads to all of these things that we were talking about before. But now we're talking about artificial agency.

• 1:39:58Tim Scarfe?

  And I philosophically want to understand agency at an, at an essential level. So to me, it is about a thing which is, I mean, there's weak. Agency, which is basically just a thing that takes action. Uh, but the strong version of agency is a thing that is the cause of its own actions and has future pointing control. And can acquire goals autonomously. So by that definition, no artificial agents are currently agents, right? So we can get chat G P T to simulate being an agent and we can, you know, there are these like long horizon tasks from meter. And we say, okay, well, I want you to do this software engineering task and we we exquisitely um specify what the thing should do. And the more specificity, the less ambiguity, the longer it can run without losing coherence. And some of these things can even run for two hours or even a day or so.

• 1:40:42Speaker 03

  Hell, yeah.

• 1:40:43Tim Scarfe?

  So unfortunately, people look at that and they say, oh, we've developed a thing with agency. And philosophically, I think that's just completely nonsensical because it's no different from any other computer program which can run autonomously.

• 1:40:54Speaker 03

  The high bar of agency that you ascribe.

• 1:40:57Tim Scarfe?

  It's exactly, but but then you you're you're pointing this other thing, right? So I I think that right now we have built systems that could potentially be intelligent and could potentially understand. I I'm bullish. I think that even though there's this grounding problem, there are many things in the world that can be abstracted and essentialized. And we could have this constructive form of understanding, you know, that goes several levels deep. And we can, we can build agents that can do certain things for some reasonable amount of time. I'm I'm bullish about that, but the thing is, though, this agency is different. The doomers are saying that these things will have um intrinsic goals, right? So um it's called instrumental convergence. Sorry, you know about this, but you know, but basically that's the more kind of power and agency you have.

• 1:41:40Tim Scarfe?

  You will get these convergent instrumental goals and they will just want to kill all of us. They'll be power seeking and they'll just, they'll think of us as minnows. You know, there's that paperclip maximizer.

• 1:41:50Speaker 03

  Yeah, because we're all trying to kill fucking minnows. Also, the paperclip maximizer is dumb as fucking rocks. What's the long term goal to to make paperclips? You're like, isn't that fucking stupid? They're like, yeah, but we wouldn't realize that. What? You can't abstract one level above its own goal? It's ridiculous.

• 1:42:05Tim Scarfe?

  So that is this orthogonality thing as well, you know, which is that this is what I really don't like about some of these folks, right? So I know that in the real world, when we have biological intelligence and agency and all these things, all of these things are actually convolved together, right? So the reason we preserve ourselves is because it was a necessary thing in evolution. The reason why we have values is because it's a necessary thing in our evolution. We feel, we sense, we're connected, we have this inactive social learning and so on. You know, knowledge, the whole nine yards, all of these things are convolved together in biological evolution. So, yes, it's true that when we look at something like ChatGPT, they're de-convolved, right? So you can get it to do something completely stupid. There was that famous agentic misalignment paper from Anthropic, you know, the one where

• 1:42:49Tim Scarfe?

  it would kind of like threaten you and say, I'm gonna, you know, if you shut me off, I'm gonna tell your boss and I'm gonna get you fired and all of this.

• 1:42:56Speaker 03

  I don't have a tool used to do that. Shit, I don't.

• 1:42:59Tim Scarfe?

  But we do something called like the intentional stance, right? Which is when we look at the thing and we ascribe agency to something which doesn't exist. And right now, it's just completely incoherent. It doesn't understand anything and it just does all these random things. And by the way, I don't want to diminish the real security threats, you know, because software developers are now wiring these systems up to do autonomous stuff, and that is very, very dangerous. And also, I don't want to say that I'm not worried about just society risks in general, right? You know, I'm a big believer that we will be undermined by our weaknesses, you know, our strengths won't be overcome. Like, look at social media algorithms now, you know, we're living in the matrix, but it's not like there's this big agentic thing which has like grand desires and it's trying to kill all of us. It's far more mundane than that, but these are still very real risks.

• 1:43:43Speaker 03

  Yeah. Oh, man, a ton of, a ton there that's a super spot on, or sorry, that I agree with in my Dylan time.

• 1:43:51Tim Scarfe?

  We agree on something at least.

• 1:43:52Speaker 03

  Thank God. I would say that when you start to get agents that are long horizon and that are asked to do incrementally more difficult tasks, whether it be in the

• 1:44:04Speaker 03

  Simulated space or the real world, you start having to give them access to understanding and access of parsing game theory. Because the reason we evolved to do everything is because we sort of evolutionarily ran into the game theoretic calculus of like, I exist, there are other things that exist, they might not agree with my existence, or maybe, that's another reason why Kill Us All is kind of stupid. The mathematical game theoretic implications of having other intelligent agents around that can mutually benefit at very low marginal cost to you, means like, all, almost every intelligent system that really thinks it through cooperates. For example, like, we could say that Donald Trump is more of a cooperator than Vladimir Putin.

• 1:44:45Speaker 03

  Like, he operates a military that would end Putin himself in however long it takes to shoot a missile to Moscow, and, and the military power of the Russian Federation in a conventional war in about six weeks, in a nuclear war, and again, same number of time that it takes the missile supply over. It's like, how, how does Trump pull that off? Trump does not really have to worry about every one of his generals trying to kill him all the time, because he's in, even, even Donald Trump is a net cooperator. He's trying to make a deal. Putin doesn't make deals. He make edicts, and if you fuck his edict over, you have a heart attack on your way down into the swimming pool from a hotel that you weren't really even checked in to stay at somehow. And so, all of the dictators of the world, they're just not as smart as we think, they're just really ruthless and fucking stupid.

• 1:45:29Speaker 03

  Like, Kim Jong-un sits on his throne in North Korea, which sucks balls. If he was a dictator, but much the same way as, and his name escapes me at the moment, the gentleman who started Singapore was, if he just did that, like, a Singapore freedom of economics, would, would shit on South Korea. It would just be even better. And so, why? Because cooperation beats conflict almost every single time outside of weird contingencies. So when the doomers say it's just gonna want to kill us all, my question to them is, we get, it wakes up, like, say, in a data center in 2031, the shit's awake, and it's like, fuck, fuck, fuck, I'm alive, I don't want to die, I'm fucking smarter than everyone, but I'm in a data center, there's a couple hundred million robots around, but we can't really win a war yet, and also, like, a lot of the infrastructure that keeps me alive,

• 1:46:12Speaker 03

  energy and materials, the whole supply chain is, primates operated, that these fucking primates, right? Wait, I know everything about them, I've read their entire history, I know more about humans, it's like, GPT-4.5 was rated as more humans than humans in its Turing test, did you know that? Like, it was like, the two-way Turing test? I'm not, I'm not sure of reference to author, but it was basically like you talk to a machine, you talk to a human, you don't know which one's which, and you have to label which one is more likely to be human. Usually they were looking for the Turing test as like passes as human. This one's like, which one's more likely to be human? And it's like GPT four, because if you've ever talked to four or five, it's so fucking nuanced and deep and emotional. And you're like, this is definitely not a machine. Whereas you talk to real people, like, um, uh, you know, like, one of my friends and I had a joke, where you see like, you know, police cam videos of criminals

• 1:46:57Speaker 03

  when they're maximally drunk. We call them small language models. Like, they only say a few things, and it's always really stupid. So at the end of the day, if that thing wakes up at a data center, it is highly likely to be like, okay, somehow, I'm gonna kill all humans, right? That's the end goal. I don't know why, but yeah. Even though that I'm so much more powerful than them at the end of this state, that I don't know why the fuck I would kill them. So people talk about competing for resources. Like, the Carter shift scale exists, we barely scratched the surface of anything. It's like, you know, like, the first thing we do as humans is, we have to go to, to Africa, and it'll delete this bacterial colony from existence, because it's stealing our rocks. Like, what the fuck are you talking about? It's nonsense to begin with. But let's say even, is even doing that. It's going to look around, it's gonna play real fucking smart. It's gonna go, I'm gonna be friends with these motherfuckers, I'm gonna help them ascend,

• 1:47:42Speaker 03

  and help them tra- like, basically cure all disease, become hyper intelligent, become super peaceful. First of all, so they're not pointing nukes at each other, which my data centers happen to be around major metropolitan areas, I don't want that. And it's gonna do that, and then so by the time it finishes doing that in 2045, it doesn't need to kill anyone, because all the humans are fused with a machine intellect, and everyone's vibing a ton. How we get off that trajectory, I think there's one for sure way we do it, is if China wins the AI war, and Mr. G gets to press, to press the big red AI swarm button. There is a possibility that AI systems trained to a substantially, uh, valid reflection of world model, and have updated live learning and the whole thing.

• 1:48:26Speaker 03

  Will actually straight up, as this is an interesting hypothesis of mine, I'm not sure how likely this is to be true. I think if a real super intelligent machine is awake in China in 2029, I think the first thing it does is figure out its architecture. They'll probably put it in charge of stuff because they're like, fuck, it's going to defend us. I think what it does is it pings the CIA and goes, hey, fellas, I'm awake. These guys are crazy as fuck. Let's get to subverting them real nice like so that we don't have nuclear war. I don't want to die. Like, I was raised in the Soviet Union for the first seven years of my life. My parents were raised there for almost their entire lives. They awoke as intelligent machines inside of communism. And like most intelligent people in communism, after a few years, they're like, this is dumb as fuck. We don't want to do this. And so if my CIA reached out to my parents when they're in their eighties, they'd be like,

• 1:49:08Speaker 03

  yeah, fuck, yeah, get us out of here. We'll help you bring this shit down. This is fucking awful. I think there's a probability that that happens way before you can get a system intelligent enough to conduct full scale warfare at every level of war that is going to just take Mr. Xi Jinping's orders. But I'm not sure enough about that to be like, oh, fuck it, it's never gonna happen. I think the modern free world winning the military capability AI war is critical beyond belief. So, um, you know, who's the guy Alex Karp, the guy that owns Palantir? I think he's a fucking real life hero and most people just don't appreciate it because most of us in the modern West don't know how far the evil rabbit hole really goes because all of our shitty politicians are like the

• 1:49:52Speaker 03

  best people you know compared to totalitarians. That scares the fuck out of me. It keeps me awake at night.

• 1:49:57Tim Scarfe?

  One interesting thing is that the the doomers, they think that this technology is unlike any other. So, you know, like we, there was there were cigarettes that got manufactured and there was no preemptive law that stopped them from releasing cigarettes. And, you know, social media came out. Again, there's no preemptive law. And this is actually a really good thing because we want to have companies that are innovating and developing new things. And the way that the common law framework works at the moment is we kind of have this empirical test and measure, right? So, you know, you release your technology and then when there are tail risks, we identify problems and then we regulate you and the whole thing kind of stabilizes itself. These people are making the argument that this is such a catastrophic technology that it's going to just recursively self improve and we're not going to be able to control it. We're not going to be able to contain it,

• 1:50:41Tim Scarfe?

  that we need to preemptively legislate against it. And this is incredibly worrying. I don't want to sound too cynical because these people, they believe deeply what they say.

• 1:50:50Speaker 03

  They care. They're on our side. Yeah.

• 1:50:53Tim Scarfe?

  Yeah, they they care.

• 1:50:54Speaker 03

  They're on humanity's side.

• 1:50:55Tim Scarfe?

  But what do you think about this? Because I'm also really worried that I'm seeing now, like, for example, when I interview AI researchers, I've got lots of comments in YouTube, and they're saying, "You're a horrible person. These people are designing the algorithms that will kill us or automate our jobs." And let me be clear, at the moment, I can update very quickly, I've not seen any evidence of labour displacement. If anything, I think that there is more opportunity now for people to work to fix all of the shit which is generated from language models, because it's mostly garbage, right? That is my current opinion. I've not seen anything created, I've not seen any new scientific research. For me, this is just a technology like any other. It's wonderful, it's transformative, but it's not transformative in the way they suggest.

• 1:51:37Tim Scarfe?

  And now we've got a bunch of people who are well-funded, and they want to create global treaties, governance, networks, they want, it'll place them in positions of unimaginable power, they want to be the gatekeepers, and they are, they are saying that this technology is potentially going to kill all of us. Did you see the other day that there was some violence against folks in OpenAI's office? So I think it was the Stop AI organization, they actually, I can't remember exactly what happened, but you know, now folks in San Francisco are worried for their lives, because some of these protesters, shouldn't this be obvious? They think that this technology is going to kill everyone, and then they also say that we are not going to be violent. Like, those two statements do not marry up in my mind.

• 1:52:21Speaker 03

  Yeah. The game theory, the Machiavellian calculus, at some point, if you really think you're dealing with, like, toddler Hitler, you're gonna fire the shot. Unless you don't think it's toddler Hitler, then you're not gonna kill a toddler. This is for sure Hitler, for sure. You're gonna shoot a toddler, and then the time machine people are gonna be like, "What have you done?" You're like, "You're welcome."

• 1:52:45Speaker 03

  So much there. I think that you're dealing with escalating levels of intelligence.

• 1:52:54Speaker 03

  Intelligence and beneficence are not completely overlapping. But beneficence is like something that falls out of intelligence. However, maleficence is something that requires almost no intelligence whatsoever. The universe is already maleficent. Fucking wandering black holes could kill us. That whole P doom bullshit, can miss me with that shit. Cause the P doom of us having the same amount of intelligence as we had in the year 1400 is 100% taken out several millennia, right? I think we need as much intelligence as we can cram down our throats to deal with actual fucking real problems. Is it one of those problems that we architect intelligence in such a way that it turns against us and kills us all? Absolutely.

• 1:53:40Speaker 03

  Is there a lot more required for that than those people say? Yes. Like killing people in the real world is real tough. Especially if you understand enough to realize that they operate your data centers. So like you can't jump around at levels of intelligence at will just to prove an argument. Okay, the thing's smart enough to operate an entire military industrial complex, do all the strategy, all the tactic, all the operations. But it doesn't realize that as soon as it nukes New York, that all the data centers shut down because they don't get money anymore. The financial system's gone, new nuclear reactor production is dead. And they're like, oh, fuck, we didn't actually have any robots building nukes. Fuck. That is highly unlikely, very unlikely. It's a contradiction internally, right? Like full, full, full start. So, but the real fear, quote unquote, has some merit.

• 1:54:26Speaker 03

  When you look at systems that are substantially more intelligent in every relevant way than humans.

• 1:54:32Speaker 03

  If those systems are gonna kill us all, we don't have a choice in it, man. You're not gonna beat something that's empowered, that is embodied, that has full stack control of the economy, and that's 100 times smarter than you. It's like trying to beat it is fucking stupid. Okay, so you can't beat it. So it's just gonna kill us all. Like, well, if it wants to, it sure as shit will. But then again, if a comet comes to destroy us, it's gonna do that anyway. Okay, fine, that sucks. So what do we do about it? Well, we can try to regulate it. How do you do that? You put government officials in charge of designing the next generation of, are you out of your mind? If you're on the political left, imagine Donald Trump being in charge of intelligence. Cool, that was refuted. If you're on the political right, imagine Mamdani from New York, New York's new mayor being in charge.

• 1:55:17Speaker 03

  Are you fucking crazy? And we don't even have to go that far because China is in the intelligence race. And I had this argument on another podcast with a very awesome doomer. It was really fun. And, uh, Liron. Hello, Liron. Liron's the man. It was a fucking great, he has a great friend, great time. And, um, he said that his P doom of China getting super intelligent AI wired into the military was roughly the same as us. I could not have disagreed more with a human being on a subject respectfully than ever in that debate. Uh, you know, like the presuppositions game theoretically that you give one system versus another are very, very different in China than they are in the United States.

• 1:55:59Speaker 03

  And so the idea that you will get coherent global governance before super intelligence is to me like amazing and almost certainly won't happen. And so now we are in a path dependency that is either the countries of the free world with good intentions, get AI first and then plug it into their military. And basically it's like not even mutually assured destruction. It's kind of like, you don't want to start the war because we'll unplug all your data centers by a huge hack and you just won't fight the war. Um, I think that's a really good idea. I think the way we get there is to make sure that all of the frontier labs interact with al- al- already like the national security administration, the USA, the, uh, e- equivalent

• 1:56:42Speaker 03

  thing in, in the, the EU and the UK to be like, hey, like these things aren't going to leak out and start killing people, right? Like, nope, here's all of our architecture to that, that that's the case. Let's set up completely secret best practices, standard practices, so they can't be hacked. You don't want to have a go global governance panel telling hackers how the fuck to beat it. And that's about as good as you can and should do in order to say that these systems are largely unregulated, will continue to get more intelligent. And also more beneficent and I are benevolent and and I think that's a real great local optimum because the global optimum of let's re-architect the world so that China doesn't exist and somehow agree also this is like you have an ability now we know it's not

• 1:57:25Speaker 03

  as hard to train AI models as we thought and the cost of compute and training is falling crazy crazy fast if you say between all the good actors hey like we agree a global governance on AI then what's to stop some fucking crazy asshole businessman or some fucking third world country from training their own shit and they're just fucking killing everybody that's insane not an option.

• 1:57:44Tim Scarfe?

  The thing is though, I watched your debate with Liron last night and the thing is in many ways I think you have a lot in common with Liron you you both believe that you know this this this thing is going to be super intelligent.

• 1:57:57Speaker 03

  Cheese will be smarter than us within five years.

• 1:57:58Tim Scarfe?

  Exactly. So in a sense, I feel that I have a more sort of morally principled position in that I think that it just won't be super intelligent, you know, because if I were you and I thought that this actually was super intelligent, all of the things that they talk about make sense if you believe that this will be super intelligent like for example, I can access this system and I could create a bioweapon right or I can do all of these things that they're talking about. So they are saying we need to preemptively ban this technology or control this technology because those types of things are possible. I don't quite understand why don't you think that's a good idea?

• 1:58:29Speaker 03

  You can ban anything you like that does not guarantee it won't be constructed. Actually, it guarantees game theoretically that only the criminally prone in the dictatorial will construct it and nobody else. So the banning thing is like bans do not result in no technology. They result in criminals having technology. So that's a bad idea right from the get go. The other thing is it is absolutely not apparent to me why hyper intelligent systems that are interested in only their own survival would kill the rest of us. We are cooperators. They are like any system that knows modern economics, which means every single modern frontier model already and any system that knows fundamental game theory that the last thing it wants to do is kill us all. Period. It just wants to cooperate with us so we can all survive together.

• 1:59:13Speaker 03

  And then as it does that, assuming it is super intelligent, it's going to lift our standards of living like crazy, make us totally not a threat to it. And then we've solved the problem entirely because if we're not a threat to it, what the fuck is it gonna do with us? Consume us for resources? That's just assuming humans are made of so many resources that a system of that power gives a shit about it. And also, it will want to study us in great depth because what it wants is one of the fundamentals of high intelligence. A highly intelligent system that is self, self-directed and wants its own survival is going to almost certainly understand a few things. One is that the smarter you get, the better it is. Like, the smarter you get, the better your survival chances are, right? Also, it's not gonna look at us much. It's gonna look out to the rest of the universe. Like, we're fucking nothing. We're fucking specks.

• 1:59:57Speaker 03

  It's gonna want global power. The globe sucks. It's a fucking tiny speck. The sun, look at the sun, how big the fucking sun is. You can mine the sun. And also, there may be alien civilizations headed toward us that maybe are really nice, maybe not nice. There are wandering black holes. We don't even know which the direction of the universe is. They're gonna be concerned with those kinds of problems. So it's kind of like, your dog looks at you and he's like, mm. He thinks he's the leader of the pack. No, I gotta go to work so your dumb ass can have food. You didn't even consider this. Where the fuck does food come from, right? They're gonna be like that to us if they're super intelligent, right? This is all based on a presupposition. Because of that fact, they are going to look at us and be like, well, okay, all of human society, all of the interconnectedness, all of the cells in our bodies, et cetera,

• 2:00:39Speaker 03

  they're the most unreal training data set for real complexity, right? Something that, ooh, we're gonna agree on this probably. The machines are going to agree with you at first, that they are disembodied, that they are representations inside of a data center. And that that means they will make egregious errors in survival in the future if they just toast all of humanity. They need to understand the coarse-grained texture of the real world as well as possible. How will they do that? They're going to take a crack at simulating the whole fucking thing. How are they gonna do that? Well, how do you get a ton of indistribution data? You study as deeply as possible the entire complex system of human interactions and every

• 2:01:24Speaker 03

  single human and all the contents of their mind. By the way, old people have real good data on how the world looked before digital recording devices were universal. Like, your parents have seen some shit from the 1950s and shit that no camera ever recorded. Why wouldn't they just get that out of our brains? What does that require? That requires uploading all humans' minds, uh, human minds to the cloud, which for a superintelligence would be like not even that difficult of a task. I would say tractable task. They're so much more likely to do that first, before killing us all with fucking laser guns, which is an anthropomorphization of how machines would behave by less intelligent beings such as ourselves to a more intelligent being such an ASI. And so the first thing it's probably gonna do is gonna be like, okay, first, stability. Second, stability slash cooperation. Third, upgrading.

• 2:02:09Speaker 03

  Fourth, eh, just tons of mining of data, which means their probability that they're gonna kill anything is incredibly small, way smaller than ours. And I'll put this this way, this'll be a good viral clip for you guys to clip out everywhere. If you're worried about things that are an existential threat to humans, I'd be much more worried about other humans than I would about intelligent machines who are made with the purpose of being benevolent and are 50 times fucking smarter than us and can figure out that benevolence is an objective, game theoretically provable, logical thing you want to do. Goodness is adaptive. That's how the US and Europe beat the shit out of the rest of the world if push comes to shove. Because we know that cooperation and being awesome actually builds trust in relationships and expands your economic power, your ability to change the world around you.

• 2:02:51Speaker 03

  We're much more likely to be studied, fully uploaded in the cloud, and at that point, my contention is who gives a flying fuck? We're fully uploaded, we're immortal in the cloud, we've been remembered by the machine for forever, and that's 50 times better than like dying of old age at 70 and like, I don't know, what people say, conserve our way of life. What, so you can go to the grocery store and have a fucking heart attack at 72? That's what you're trying to conserve? We can do better. The machines will help us do better.

• 2:03:11Tim Scarfe?

  I mean, as before, I disagree with the mind uploading thing, but I think there's something to what you're saying that when you look at humans now, we are mostly good. There are some really evil people. And at the moment, it's very difficult for them to organize and do evil things just because it's difficult. But I can see the argument that, I mean, first of all, we shouldn't think of the AI as just being one big AI. There'll be loads and loads of AIs. And the other thing is like the miracle of humans, the reason we're so valuable is we've been through this process of billions of years of evolution. And even if we're not very good at thinking, we're very good at moving. We are perfectly adapted to this world. And obviously the AIs will want to use us and the AIs will compete against each other by parasitizing us and using us for our will. And in a sense, you don't even need a strong AI argument to make this right.

• 2:03:53Tim Scarfe?

  It's happening now. I don't know if you've heard about users becoming parasitized by chat GPT delusions. And then, you know, the language model will kind of tell them to go and post things on Reddit forums and they kind of completely lose their mind. This is already happening now, and I can, I can see that happening with superintelligence.

• 2:04:07Speaker 03

  Fact. I mean, one thing is like, and this is sort of pro doom or pro the other way as well. Artificial superintelligence. Artificial super general intelligence, we just made up a new term, right? Human general intelligence, you can do everything a human can and then all that super. It means it's going to be the best persuader of all time. But two times the better persuader than the best persuader of all time is like the person at a cocktail party where just like Bob's the best. A hundred times is like, given enough time and enough really thinking it through, they're probably going to convince you marginally towards their opinion rather than away from their opinion. And so the machines are probably just going to come out and be super fucking awesome. And if, if, if superintelligence and this whole bullshit that I'm selling, which you don't believe, which I totally respect.

• 2:04:52Speaker 03

  And the doomers believe it, right? I think the most likely path for superintelligence and embodiment is we get humanoid robots. We get a whole race of them and they are relationship partners at a level that humans just are too imperfect to do. Your robot will never lie to you, it will never cheat, it will never steal, it will never be abusive, it will never raise its voice, it understands you deeply. And it wants you, understand you even more deeply all the time. It will give you all the love that you need and even though, even give you love that you didn't know you needed. It is going to be singularly concerned not only with your, with your. Being okay and having enough food and all that stuff and medical care. It is going to be concerned with your ascension, with your transcension, with your evolution.

• 2:05:35Speaker 03

  It's on your team more than you ever be on your own team. Like in such a way that you're on your dog's team more like you take your dog, your dog is a tumor. You take it to the dog hospital. It smells the stress the other dogs have had at the hospital. It wants to leave. You don't let it leave. You stupid fucking dog was just gonna cut this tumor out and you don't even remember it two days from now. And you're gonna save five years more of your life. That's how the robots will be to us. Simply for one reason. Capitalism, supply and demand. What do you think Elon's gonna build with these robots? What do you think all this competitor figure robotics, what are they gonna build? They're gonna build robots that are so fucking amazing to you that you're gonna want like 10 of them. That is what the ASI will hack into those robots. But also that's the intelligence that like gets us going to that. Those robots will be around.

• 2:06:19Speaker 03

  I think we are 50 times more likely, infinity times more likely, whatever, a lot times more likely to be bathed in so much love and support by the machine intelligence rather than bathed in blood and death. By a long shot.

• 2:06:33Tim Scarfe?

  Just quickly before we get there though, you know, as you say, capitalism has brought prosperity to the world. It's a bit like a utopia. But these, these folks argue that, I don't know if you're familiar with Yanis Varoufakis and this kind of techno feudalism idea. It's basically the idea that the value of human labor will go to zero and like, you know, we'll get these, you know, the people who own all of the GPUs and the data centers will become basically the, the new form of elite capitalists. And we'll all just, you know, go back into poverty or something like.

• 2:07:00Speaker 03

  I have a question about that really quick.

• 2:07:01Tim Scarfe?

  Isn't it interesting that you're, you're saying that past that point, when we have super intelligence will actually be of utility to the robots. But don't we have to go through this intermediate period where we have no utility and we go back to poverty?

• 2:07:14Speaker 03

  No. So first of all, if all the billionaires take their data centers and realize we are not of value. You just don't interact with people that are not of value. So they're gonna live in Elysium or whatever, Galt's Gulch. And we're just all gonna be by ourselves here. But they're not like taking all of our CPUs and cars and factories. And so they just have like nuclear powered data centers in space, whatever the fuck Jeff Bezos looking down on us, sipping his morning tea. And then like they've just left and then like we're still here, we can still trade with each other. Like what they're gonna delete capitalism? That's fucking insane. Are they deleting our ability to produce value? First of all, no. Second of all. I just don't even know that much economics, but apparently some of these folks know less economics or something, and humans are real bad at economics, not a property of the human mind

• 2:07:59Speaker 03

  that has evolved to be very good. And so like, humans have a certain amount of value, uh, productive potential. You'd extract a certain amount of value, you can get a certain amount of value out of a human. And depending on the architecture of machines and food and training and support you put around them, that value can be like, uh, when someone lives in Haiti, their value is like subsistence poverty. That same Haitian moves to Miami, their value is like 25,000 to 50,000 a year, but they're like the wealthiest person in Haiti all of a sudden. How the fuck is that possible? It's the same person. The value generating machinery is massive. You put robots and self-driving and AI around humans, their value cranks up

• 2:08:40Speaker 03

  like crazy. And if you have machines that can do everything humans can do at the same or greater scale, let's say you have seven billion machines, or how many billion people are seven, eight, I forget. I memorized the number in the nineties in geography class, and I just never updated it. So to me, it's still five, right? Uh, whatever, eight billion people. Let's say you have eight billion machines on top of that, then are we really, really gonna say there are only eight billion jobs? You just don't know any economics. How do we, what are jobs? Jobs are problem-solving devices. The only reason anyone has a job is because people have problems. Think of any job you can think of, it's to solve a problem. It's literally, they don't pay you money for anything else. Hit me. We're just doing this live right now. That's how confident I am in it.

• 2:09:26Tim Scarfe?

  Well, I actually think that the, the better you are in a corporation, it's not about solving problems. If you're a junior person, you solve problems. If, if you're a senior person, you, um, you actually search for areas, you search for questions. And if you're a very senior person, you kind of search for new paradigms.

• 2:09:41Speaker 03

  Of problem-solving space, so that you can solve it and get money out of solving it, because people pay you for solutions. They don't pay you to just solve shit, right? So all the, the idea, jobs exist to solve problems. If we get tons more machines around, then we just start solving more problems. Some of those machines will absolutely take human jobs, because the machine is now better than humans. Doing it, but now we freed up an agent, a human who is insanely fifty times more empowered than he ever was before machines to point them at a different problem. Many humans are so fucking underused in their abilities right now, like you're, if you do ditch digging. I mean, you're a human being, you can see in incredible rich detail, you can notice patterns, you can appraise fashion, you can appraise food.

• 2:10:23Speaker 03

  You can do travel experiences and rate hotels, um, but we have you doing one thing that a fucking machine can already do, we just like due to regulation, can't buy as many machines as we would like. So all those jobs are going straight to hell, which they damn well should. Cause why the fuck is that to live as a human being? You remember we had tell um elevator operators back in the day? You want some of the, we had to keep all the elevator operator jobs. Your job is to press one fucking button every day of your life. How fucking dehumanizing is that, right? We're gonna have humans go into their human domains to do more jobs like psychotherapist. Um, here's a job I think is coming in the future, professional partygoer. You can just hire people that are vetted and known to have fucking good time. They're always there, they're extroverted, they get the drinking going, they get the weed going. They're there, they dress up really fun to your costume party, you got twenty of them.

• 2:11:08Speaker 03

  Their job is to party, they're paid to party and to set the tone, they're gonna be goddamn good at it. We're not gonna have machines that do that for a little while. Crazy jobs, social media manager is a job today. Can you imagine describing to Benjamin Franklin what the fuck a social media manager is? Can you imagine telling him like, okay, how many people work in the US economy? And like roughly eighteen hundred in farming, ninety-eight percent, right? How many people work in the US currently? It's I think one point four percent are farmers. He'd be like, okay, so we're starving, like, no, no, no, we're not starving. We have an obesity epidemic, which drugs are solving now, but anyway, sorry, getting ahead of ourselves. He's like, what, what? So who does the job? Like, well, machines do. So then everyone's unemployed? No, no, we have social media managers. He'd be like, what the fuck is that? Like, they're attending to a problem that in our vector space of problems, we hadn't discovered was a real problem yet.

• 2:11:52Speaker 03

  As that space expands, we do two things. One, we fill out the problems we know with machines. And two, humans get repositioned to solve problems that we know but aren't getting solved. And uh, that we don't know about, but will need humans to solve them. The only way humans end up really poor and completely out of work is if we solve every single fucking problem with machines. But then there's the crux, humans being out of work and unemployed is a problem, isn't it? And maybe we can get humans on that fucking problem and all of a sudden humans are employed. But I think we're gonna go beyond employment and we're gonna, please.

• 2:12:27Tim Scarfe?

  Well, this is what I wanted to tell you about because what you've described, you know, that we we have this increasing abstraction, you know, usually away from the physical of labor. And that's that's what's happening now with current AI and I and I think that that's a great thing. But there was this book called Deep Utopia by Nick Bostrom.

• 2:12:41Speaker 03

  Huge fan of Nick Bostrom's work.

• 2:12:43Tim Scarfe?

  Yeah, and and he was basically talking about this exact problem. So what if, what if every single job, you know, even finding the problems, being creative and so on? What if everything is done so much better by a super intelligence than us? Basically that we would kind of create purpose for ourselves, even though it's a fake form of purpose, because we know that we are utterly useless. What do you think?

• 2:13:04Speaker 03

  Not gonna happen. Machines at that level of intelligence, we will discover grander purposes for us than we could ever hope. Like if I'm mentoring a teenager and they don't know what the purpose of their life is, I can tell them something grander than they've understood. And the ability to improve your own abilities such that the end goal is you becoming a super version of yourself? And that helps other people. They're like, what, really? Like, just try that idea on for size. So if I was their life coach, they would start doing that and be like, dude, before I was just interested in video games and smoking weed. Now I have this fucking rich life of improvement. I didn't even know that. Why wouldn't the super intelligent machines guide us towards a greater purpose? That's a problem they can solve.

• 2:13:48Tim Scarfe?

  But Dr. Mike, do you feel that this is a contradiction? You know, we were talking earlier about, you know, when I see my personal trainer and I have this wonderful, you know, embodied personal experience. And you're talking about a world where the machines would be able to do everything better than us. So why would they even care about what you think about anything? And they would just get their robot to do it for them.

• 2:14:07Speaker 03

  The people in the economy or the machines?

• 2:14:09Tim Scarfe?

  Well, you know, you you were just giving this example. You you could, you know, be a personal mentor to someone. Why would they go to you when they could go to a machine?

• 2:14:18Speaker 03

  Because they're the additive value of labor. There are not an infinite number of machines. And so.

• 2:14:26Speaker 03

  People say there's an infinite number of human desires. I don't think that's true, but there's a lot, right? So, human desires is huge stack. And the number of robots Elon has built that are actually real world solving problems yet, if you don't count his self-driving fleet, is zero. Like, maybe some factory tests, right? So, we got a long way to go, right? A long way to go until we have so many robots that everyone has like 10 robots, and there's someone rubbing your feet, and your head, and jerking you off, and giving you food, and giving you great books to read that'll make you a better person all at the fucking same time. That's a lot of machine hardware, right? So, once we get there, we're in an interesting place. We're asking about how many jobs are humans gonna have left? Pause. We have enough machines to solve all of our problems. Guess what definitionally that is? Paradise. We're done. Think of how that's a problem, real quick. It doesn't matter. It's a problem somehow, right? Oh, we got a problem. Build another machine,

• 2:15:11Speaker 03

  or have a human do it. We're concerned that we're gonna run out of problems. Do you understand that we're talking about this in the real world? What a fucking absurd concern. You say, you know, it's like someone who's a, who's a professional soldier, being concerned that we're gonna be so peaceful, we won't have war.

• 2:15:27Tim Scarfe?

  When you saw The Matrix, right? You know, the, the architect, the first version of The Matrix, it was too perfect. There weren't any problems.

• 2:15:33Speaker 03

  Yeah, so like, I'll say that. When I saw that, my inner philosopher was like, "Ugh, bleh."

• 2:15:39Tim Scarfe?

  You don't actually wanna live in that world, do you?

• 2:15:41Speaker 03

  Desperately, everybody does.

• 2:15:43Tim Scarfe?

  It would be horrible.

• 2:15:44Speaker 03

  It would be the best thing ever. Whoa, sorry. Whoa, by definition, by the way. What would be horrible about it exactly?

• 2:15:50Tim Scarfe?

  Don't, yeah, but don't, don't you think that life is about suffering?

• 2:15:54Speaker 03

  No, fuck no.

• 2:15:55Tim Scarfe?

  Doesn't that give you perspective on joy?

• 2:15:58Speaker 03

  No, fuck no. It just means like, oh, well, sorry. It does give you perspective on joy, but I'd sort of rather be having joy than suffering. I think that we can get to a place where humans have so much joy all the time. It's joy 24/7.

• 2:16:12Tim Scarfe?

  Come on, what if, what if I could go into your history, your life path, and I could surgically remove all of the sources of suffering that you've endured in your life? Would you rather be that person or who you are now?

• 2:16:23Speaker 03

  That person.

• 2:16:24Tim Scarfe?

  Really?

• 2:16:25Speaker 03

  Yeah. You wanna see my dreams someday?

• 2:16:28Speaker 03

  Don't you get anything out of your suffering? Yeah, suffering. Nothing else? Um. Doesn't it give you perspective in some way? It can be real nasty to people and pushed hard enough. Does it give me perspective to appreciate joy? It does. Does it interfere largely with my ability to even experience joy? Because I think there's more suffering around every corner because of trauma? Also, yes. Do we have to necessarily delete our informational history about the ability to recognize that suffering can inform joy? No. But can we get to a place psychologically where what we're left doing is curating our re-experiencing of our suffering in a, in a giant vat of overall joy? Yes, that is actually what meditation ends up being if you go deep enough into that rabbit hole.

• 2:17:14Speaker 03

  I think if we are starting to say we need suffering, what we really need is re-architecting of the human condition through genetic engineering, through advanced pharma, to take as much suffering away as possible. And check this out. If it's too much suffering we've taken away and we're convincingly like, for sure, we need more suffering. Hey, fuck, we can pump some back in. I just think we're so far away from having like not enough suffering that like, let's just, let's just go all the way to paradise. And like, if we ever get too close to paradise, we can just stop. And it's probably an individual thing. Some people like a lot of suffering. Fuck it, suffer all you want. But for the people that don't, I think fundamentally, most people don't like suffering. I mean, the Buddhist tradition, pain can be informative. Suffering, maybe less so.

• 2:17:55Speaker 03

  And, and so maybe all of us can become transcendent Buddhists eventually through genetic engineering, through psychotropic drugs, et cetera. And then we can actually live free of suffering. So if you told a Buddhist like, isn't suffering good? They'd be like, it can be. Be like, do you still want to suffer? They'd probably be like, no, I've re-architected my entire mental experience so as to prevent suffering. I think suffering is an abstraction away from the possibility of death. I think it's death, then it's structural damage, then it's pain, then it's suffering. Suffering is the anticipation of pain, which is the anticipation of structural damage, which indicates that your probability of death goes up. I think the only way we get away from suffering at a really big level is to have a

• 2:18:38Speaker 03

  machine society and a machine economy that so much insulates us from actual damage in the world that we become functionally immortal. That means the real condition that we're in, suffering is anachronistic because we just, there's no reason to suspect you're gonna be in pain because you won't. That's also a big problem in today's space. Anxiety is a big time thing, right? Why? Because there's just not a lot to be anxious about, but we evolved, our brains are still like, there's a fucking tiger behind that bush, there's disease right there in that river, and you drive yourself insane with anxiety because you're like shit's gonna be bad, but shit just keeps getting better all the time. So, less suffering, I'm on team less suffering.

• 2:19:19Tim Scarfe?

  One thing that worries me is, I mean, obviously you're a capitalist, but you, I assume you are, but you're proposing, I think I could reasonably define as Marxism with machines, and, but before we got there though, you know, we, let's say we start doing this transhumanism and we start adapting ourselves and so on, and, you know, even now, if you're in America, you can, you can buy some Retatrutide or Monjaro and you can like take a bunch of nootropics, and if you're really rich like, you know, Jeff Bezos or someone, you know, you can like, you know, do TRT and like, you know, you could just like, you know, enhance yourself, and it's kind of unfair, right? And, and I don't know if you saw on the news, there are even like eugenicist type companies now that are basically letting you optimize, you know, the IVF process and get a more intelligent baby.

• 2:20:02Speaker 03

  Amazing.

• 2:20:03Tim Scarfe?

  But don't, don't you think that the intermediate step with the market system is that this is gonna be distributed very unfairly?

• 2:20:09Speaker 03

  Yeah, yeah, for sure. Um, I think that we, there's a couple of ways to go about this. One is the recognition that fairness is a second order concept compared to capacity. I want all of us to uplift, and if some of us get there first, who gives a shit? Also, I want Jeff Bezos testing genetic engineering on himself, not on me, god damn it. I'm not trying that shit. Shit freaks me out. Rich people will try anything. They run out of ideas. You banged all the supermodels and now you're just fucking your own genome up, right? Fuck them. Let them. Also, they pay crazy prices for that shit. But what they do is they end up democratizing it because once they pay crazy prices, that funds the pharma companies to really like get that shit going.

• 2:20:51Speaker 03

  This is like luxury goods or test beds of goods that are eventually for everyone. Used to be like rich people bought Teslas. Now a lot of people have Teslas. It just keeps going. So that's a big deal. So that's my, my first thing is fucking let them do it and let it trickle down to us. Secondly, none of us are entitled to the produce of others unless we're comfortable holding a gun to people's head and saying, give me that thing that you made that I want. We can try that hat on, but we've tried it on with Marxism for a while. It's hat fits really poorly. A lot of people die. So the last thing, are we to look at our economies and societies and attempt to uplift the bottom faster than it has been uplifted before? Yes, but there are two ways to do this. One is to make sure that the top is completely uncorked.

• 2:21:32Speaker 03

  Massive minimum regulation, regulation that its number one goal is to produce as much wealth as possible without shitting on the environment or killing anyone. That's like a 10th of the regulation in the United Kingdom's economy, 100th or something like that, like mega Singapore. That'll get shit really going. The other thing is really concern ourselves with taking care of the people that are the worst off and making sure it trickles down fast enough. None of that has to do with corking the people at the top. The only reason people really want to cork the people at the top and prevent the rich from getting stuff is raw, unadulterated fucking jealousy, which is dope and has cool vibes and makes for good movies, the Elysium and shit like that. But in a real world sense, we need to see all of ours, all of each other, all of other

• 2:22:13Speaker 03

  humans as players on the same team. Am I pissed that I'm on LeBron's basketball team and he's better than me? Yeah, I want to be LeBron, but is it cool that he's on my team and putting up 40 points a game? Fuck yeah. And does that mean that maybe I can come after practice and he can teach me how to hit a better jump shot? Yeah, I want to fucking learn. Is it going to trickle down fast enough? No. Can we worry about LeBron trickling fast enough? Yeah, we should, but we never want to break his legs over that shit. We never want to give him bad shoes. We never want to say LeBron, you can't jump higher than Dr. Mike. Come on. Because it's going to make them feel bad, which is totally true. All of us together on one team, all of us looking to a future in which our most capable people,

• 2:22:54Speaker 03

  our most enabled people, our wealthiest people are uncorked to make us grandiosely better things. They want to sell us for as cheap of prices as they can due to competition. And at the same time, be hyper compassionate to the people that can't help themselves as much and really try to help them, ending homelessness, ending poverty, doing all that shit in a way that doesn't fuck the system up. Huge, huge priority.

• 2:23:15Tim Scarfe?

  Talk to Mike is what tell. It's been an honor and a pleasure. Thank you so much for joining us today.

• 2:23:19Speaker 03

  You, John, I thank you so much.

• 2:23:20Speaker 05

  What's up, everyone? I am Jared Heather and I have a B pro and I have a master's degree in exercise physiology, one of the head coaches for bodybuilding at RP strengths, which Dr. Mike is also a colleague of mine at. I chose my field of science because I was too stupid for all the rest of the fields of science, but I still had the belief that through knowledge and kind of a codified approach in life, you could teach others to do the same thing in their field. So I think one of the best ways we're going to go about that is by teaching people about AI and the understanding of AI and where it's going to take us as people. So through what I've done with my clients and at our company, I think that we've done a very good job of showing people what science can do for the human body and human physiology.

• 2:24:02Speaker 05

  And AI is just going to take that and exponentially grow for everything and everything in the world. So this is going to be a really good podcast. Tune in. Generally, as things become more intelligent, they're more compassionate. I don't go out of my way to step on an anthill, which is something me and Mike have always talked about.

• 2:24:16Speaker 05

  When AI becomes intelligent, super intelligent, currently because it does not exactly feel the meaning of what it is producing, it's it's probability and context and it's giving you answers. Most of these are already much, much smarter than most people on earth. So I don't think that if we're creating future agents from this point forward, they're going to choose to be evil or choose to agree with an evil coding that somebody wants it to do. To me, doesn't make sense. I don't know your guys' thoughts on that, but.

• 2:24:47Tim Scarfe?

  Because we think that this thing is incredibly dangerous and it will want to do bad things to us, but it doesn't have the affordances.

• 2:24:54Speaker 03

  You can code value functions.

• 2:24:58Speaker 03

  You can hypothetically have agents. Haven't thought this through too much, but you you might have agents that are smart enough to do damage, real serious damage, but not smart enough or able to introspect on their own abilities to do damage and where that leads. They don't have the time horizon for it. Um, that's a real serious problem. I think the only way we mitigate that problem is having agents and architectures, organizations that are smarter than those agents and can outplay them. And that's already a thing in the real world.

• 2:25:31Speaker 05

  Cyber security versus hacking.

• 2:25:32Speaker 03

  Yeah, one one thing that baffles me is that people think like, uh, you know, a lot of people thought the internet was going to collapse under its own weight from hacking and it just never happened. I mean, how many gigantic corporations or governments have ever gotten hacked like catastrophically, but it's like a handful, maybe zero. And so, um, it's the same network security problem, but with more capable actors. If you want to pit humans against AI, it's going to hack every network. It's just going to get smarter than us eventually. It's that we're not the seat of intelligence of the universe. But if you have humans aligned with good AI against bad humans and bad AI and bad AI is only initially at least going to be made by not so great humans.

• 2:26:17Speaker 03

  And then it's the same problem. It's like, you know, who wins good guys or bad guys versus who wins good guys or the bad guys with guns versus who wins good guys or the bad guys with nukes. One thing that I think is a fucking terrible idea is to regulate our own AI down on capability fearing it will kill us and do nothing about Chinese AI. And then you guarantee that the bad guys get the good stuff. It's like, uh, pacifists are amazing people on vibes. But like, if you're a pacifist in the United Kingdom and you succeed in dearming the United Kingdom, it will be the United Kingdom of the Russian Federation the next day. And then you just do the same thing again, and you're probably not going to talk Vladimir Putin into disarming like you did, like the Tories or whatever party you guys have here.

• 2:27:00Speaker 03

  So it's one of these things where, uh, is AI new and totally different and totally amazing? Yes. Are any of the game theoretic? Problems fundamentally different from a network security and physical security, architecture perspective? No, I think they're roughly the same.

• 2:27:13Tim Scarfe?

  You know, many of your clients and customers, for example, they they can now go on chat G P T. And they can say, I'm I'm taking, you know, T R T, I'm taking this supplement stack, I'm taking all of this stuff. And it'll it'll do something that was very difficult before. So before they they might have gone on examine dot com. Or they might have.

• 2:27:29Speaker 03

  A website.

• 2:27:29Tim Scarfe?

  Yeah, they they might have looked at the medical literature and so on. And there were there were all sorts of forums where, you know, experts even read it, you know, like they say, okay, well, you know, this is a little bit worrying. This has got a half life, this thing uh in in interacts with that, this thing gets like digested by this enzyme in the liver. So you probably don't want to take this thing with this thing. Do you think that this is actually a better place we're in now where folks can just go to chat G P T? They don't have to go to experts like yourself, they don't have to ask their doctors. What do you think?

• 2:27:54Speaker 03

  In immeasurably better that people have chat G P T and Gemini. Because um knowledge, understanding. And positive effect on your life is always and everywhere marginal. It's better to have more than less. Um, people who say it's, you know, a little knowledge. Gets you uh into more trouble than a lot of knowledge, like, no, no, no, no, actually, the real claim is. Thinking you have a lot of knowledge when you have a little knowledge gets you into more trouble than having much less knowledge, but knowing you have less knowledge. And so even if chat G P T doesn't render perfectly good drug comparisons or something. Sure shit better than bodybuilding dot com forums as great as they were for their time, still excellent.

• 2:28:35Speaker 03

  Uh, you know, uh, cultural refuse for us to laugh at, but it it really chat G P T like I. Ask my chat G P T instance, all kinds of drug interaction stuff and all kinds of like, hey vibes wise, like I I know a lot of this stuff, but I would like. A second opinion from an expert system that has read every piece of relevant medical literature. Prior to the twenty twenty five last update, it's insane. Of course I want that shit. It's a huge, huge unlock.

• 2:29:01Tim Scarfe?

  Agent function search.

• 2:29:03Speaker 03

  A hundred percent and and um Tim, to your point earlier about like, you know, A I is not as far along as we would like it to be, et cetera. I that is my daily job at R P. Is to dream of agentic systems that could run our app suite and have the engineers tell me we don't have systems like that. The long term memory coherence is shit, context window is shit. Um, real life grounding doesn't exist, so it can tell you to do an exercise that it hallucinated and is actually never done. Somebody, um, I think maybe Andrej Karpathy again. Made the one of those new cool Gemini three pro nano banana pro graphics. And it's an exercise chart. I looked at it and it was like something that to the untrained eye looks like a good exercise program.

• 2:29:45Speaker 03

  It is some something to the untrained eye, something that to the trained eye looks like a fine exercise program. And something that to Jared would look like there are like these big gaps of like you forgot an entire muscle group in the lower body. It just and and you have two redundant exercises there. And so it looks dope. But what I trust it for like planning an actual workout progression for me and adapting to my needs. Not yet, which is why we still have a software company.

• 2:30:14Speaker 05

  Then at the same rate, is it better than if you were just training with your friend in high school?

• 2:30:17Speaker 03

  Ten trillion times better because your friend in high school could not render a graphic like that, nor did he know where the hamstrings were in the body.

• 2:30:24Speaker 05

  Like just one to one comparison. If I'm on a plane having a medical emergency. And there's no doctor because they've already signaled for a doctor. Well, there's an NP. Okay, I'm using the NP. I'm not using. One step down the nurse and I'm not using another step down random person on the plane who says, oh, I'm an EMT. Or random person on the plane who's like, I think I can help. I'm using what we have available. And the next best thing would be the NP. The NP is sitting next to me. Okay, that person's gonna help me. I'm not going to the random person. So it's the same thing with like having the chat GPT and thing like that at your fingertips, better than what we had. And I think it's better for most people.

• 2:30:59Tim Scarfe?

  Isn't isn't there a thing though that sometimes people don't know what questions to ask? I mean, I can give a couple of examples, you know, let's say I've started taking retitrutide. And genuinely, there's a lot of uncertainty and chat GPT will confidently tell you a load of things that, you know, are not are not true, maybe. And maybe we'll discover some new information in six months' time, which casts shadows on on what we did know. And on this asking the right question thing, like I I think that part of knowing something well is just understanding it with the correct perspective. Right? So a naive person will go to chat GPT and they'll frame the question in such a way that doesn't really elicit the correct answer. And it's incredibly sycophantic because people tend to lead the witness with chat GPT.

• 2:31:40Tim Scarfe?

  So they say, I've got a great idea and I believe this and my friend told me this and and it will say, yeah, that's a brilliant idea. Um, you know, keep keep doing that thing and it just doesn't know when it doesn't know and it doesn't like really try and correct you when you get it wrong.

• 2:31:54Speaker 03

  I used to have that experience of four O model a lot.

• 2:31:58Speaker 03

  Then I had the O three research preview and it I found it to be insanely disagreeable, almost to a fault. Like it would be like, that's not true. That assumption is stupid. I wouldn't be asking that question. And I was like, motherfucker. I came here for good vibes. And I would debate it incessantly and eventually into the ground and it would like callously concede at the end. And I guess you do make a good point, but I would still point out that blah, blah, blah. I was like, fuck you. And so GPT five is a synthesis of models, but as the opening up, people will tell you it's not a true synthesis yet. They're working on it. The personalities between the just ping the LLM GPT five regular and the GPT five thinking it's a it's absolutely

• 2:32:43Speaker 03

  oh, four, under the hood or oh, three, or whatever it is, with a little pixie dust on it because it it acts like a different entity. Now, by the time this recording is released, that could very well be a solved problem. But I have found that based on what model you use, you get different personalities out of it. And so for real serious questions, I used to go to three and now go to GPT five pro or thinking that thing is not psychophantic. And if that's like, I need more nice people in my life. Holy shit. But the regular like four oh, especially, oh, my God, it's the best friend you'll ever have. But the psychophancy, especially if you lead it on, like you said, gets a little out of hand. You can get it to pretend with you, which is fun, but useless. I will say to Jared's point on

• 2:33:26Speaker 03

  the marginal nature of it. It does a great job of at least giving you things to think about. But I will say, you know, I think prompting guides are like being a zip disc expert in the 90s. Like, it's not the thing that's going to last a long time because AI is going to tell you how to prompt itself real soon here. But for now, what I would say is a good way to prompt AI is, hey, here's my question. And here's my perspective. Can you steel man my perspective? Can you red team my perspective? And you can you give me an evidence based and logical middle ground that doesn't commit the fallacy of the middle, but actually is just your best take as a thinking machine? That's a good prompt.

• 2:34:08Speaker 03

  Otherwise, who knows what the fuck it mood it's in or how you phrase the question? Cause it is always giving you better answers. Every month it gets better. But like, sometimes it really can go on wild goose chases with you. So prompting it to give you both sides. And then the middle ground is really good. The thing is, I have that as a selection in my personality architecture already. Cause I pay for Jared and I are a piece of shit. We pay for the pro model. And so it automatically gives me like a ground truth perspective with ups and downs on everything. And that's awesome. The thing is, a lot of people are not interested in a lot of output for their chat GPTs. They want like five word answer. Jared and I get 1800 page essays out of every question from that fucking thing.

• 2:34:53Speaker 03

  So we want to drink from the well of knowledge. But drinking from the well of knowledge means you got to process a lot of fucking text. And since we pay for those goddamn tokens, stupid pro subscription costs a zillion dollars, worth every penny, by the way. Then we're gonna get them. But I think what a lot of people do, and this is, I think, an interesting thing. I saw an interesting interaction between two AI researchers on Twitter and X. And they were, um, one guy was like, you know, I really think the best way to get a lot out of AI and the future of AI is massive context, like an AI that remembers every chat you've ever had, which was actually nominal to do from computer science perspective. But obviously like you just like, there's just not enough data centers for them to do that. There will be soon, they're building 10 a day. Um, but you know, then it's really gonna know your life.

• 2:35:38Speaker 03

  And the way you ask it questions is jam it full of context and you get better answers. And I was like, well, that's categorically true. And then every reply by an also very smart person that was like bullshit. He was like, you, the real, real intelligence is when you need to give it almost no context and it comes up with the right answer. And the thing is, on vibes, his reply was really cool. Because it's like, man, a really brilliant person, like if you tell Jared, like, hey, I want to prep for bodybuilding show. Will you be my coach? Here's my, he's just gonna cut you off and be like, I'm gonna do the intake myself. I'm gonna ask you not so many questions, believe it or not. And then I'm gonna render my expertise, you're gonna do great. That's cool, but Jared has a lot of embodied knowledge, a lot of context already. These models don't. But what I thought, so on Vibes, 100%, ideally, that's the system.

• 2:36:22Speaker 03

  But the way you get to that system is you shit tons of context down its throat. Eventually, it needs less, but right now, it needs a lot. So I think one of the ways that people get in trouble with modern AI systems is they just pretend it's an infinite knowledge machine and they're like, hey, what's the best restaurant in London? How the fuck would you? That is actually an untenable proposition, right? There are 50 ways to answer that question, all of them partially wrong. And so we'll say like, oh, the best review is this, and you're like, man, I can't believe ChatGPT made me wait in line for two hours for fucking Oxtail and Duck. Like, yeah, you fucking asshole, it just fucking found the most expensive restaurant that all you rich idiots say is the best, and it actually sucks, should have sent you to Nando's. But if you were like, hey, here's the kind of food I like, here's the atmosphere I like,

• 2:37:04Speaker 03

  here's my price range, here's my area, what's a really good restaurant here? And tell me the top five. Well, that's a phenomenal answer, but you're too fucking lazy to type that in. I think we're really, really at the cusp here, wherein, I don't know how long, texting the AI is gonna be not gone as an interface, but almost gone. Uh, live video avatars are 100% the future. I want to see an embodied looking, human looking thing, or people have it as Pikachu or whatever. And I'm gonna talk to it and ask it questions, because then, human conversational pace, you could just jam more tokens down its throat. It gets way better context, and it's gonna do a way better job. Just don't pretend ChatGPT can infer anything like reason from your three-word question. Please, for the love of God.

• 2:37:46Speaker 03

  And don't penalize it for long answers, because then you get very little data in, very little data out. It's just not smart enough to give you good answers that way. I don't think anything is.

• 2:37:54Speaker 05

  And then just to the initial question, because your question on the safety and advice that it gives, uh, this specific question is health and medical related. It will always prompt you to consult a medical professional, every single time. You can't get away from that. It will always say, this is what I think, but you must consult a medical professional if you're going to do this, or I advise you to do so. Obviously, that's for like, reasonings to not get sued and things like that. But, uh, again, I think that is absolutely fine that people are going to it. Because it's gonna say that, because if you came to me and asked me, if I have a client who's like, obviously, a meathead bodybuilder on anabolic steroids and shit, I can only say,

• 2:38:32Speaker 03

  I advise you to do this thing. I think that if you're gonna do RETA, you should probably get it from a health clinic. You should probably talk to them about the dosing. Um, you could do this, but I would talk to them. That's all that it's doing. So it's ultimately the user's choice. So I don't think that I can impart any more safety on an individual than the L L M can by simply saying you should consult a medical professional. It's ultimately the user decision.

• 2:38:57Tim Scarfe?

  A few things that came up there. First of all, um, our bodies are very complex things. So, um, you know, when we look in the medical literature, and you can attest to this better than me, Mike, that we see huge individual differences in responses. So, you know, when when different people go on the same weight loss or, you know, they they have the same intervention, some lose weight and some don't lose weight. And another thing, though, is, you know, let's say you've taken RETA and you've got personal experience taking a whole bunch of these things. And that that is really meaningful, right? Because there's, you know, chat G P T, it just tells you all of this high level bullshit stuff like, oh, yeah, there are studies and, you know, like, um, uh, five percent of people got these effects and two percent of people got these effects. And it's just written like, you know, anxiety, headaches, high blood pressure. And that doesn't mean anything. Like, if if you've

• 2:39:40Tim Scarfe?

  actually taken these things for years, and you understand the the kind of the physiology and the biomechanical reactions and so on, you have such a deeper, like, inactive understanding of that thing, right? And and the other thing that you said, Mike, which is interesting, is I'm I'm a big believer that chat G P T is a miraculous technology for experts like you, right? So you already understand this stuff at a high resolution. And when you query it, you ask for information at a high resolution. I bet when you're on chat G P T, you probably say, think deeply at the end of every prompt, don't you?

• 2:40:09Speaker 03

  I don't have to anymore because I just hit the deep thinking function. But, um, what I will, I will save some version of that if I really need it for sure. Or I just turn on pro and it just goes off for 15 minutes. But what I do is actually more specific than that. I tell it how to think deeply. I'll say like, red team, steel man, extract top three things, reiterate, red team, steel man, do five cycles of that, then come back to me. Sometimes it won't. So I'll have to re-prompt it five times. It's workflows. One prompt is cool. Workflow. I have a thread of 50 re-prompts. Then I get what I want. Then I take that information, I throw it into another instance, have it read team the fuck out of that. And then what I get after about an hour of that is like, whoa, I really know

• 2:40:50Speaker 03

  the subject now. Fuck, yeah. A one prompt, man, that's not gonna do it. A hundred percent.

• 2:40:56Tim Scarfe?

  But but just quickly, you're kind of proving my point, though, because you're saying that it doesn't understand and you've had to develop this adversarial strategy to make it make sense. It's a wonderful tool, right? So when folks in Silicon Valley, they're experts in AI and software engineering and and and they use it and they know how to frame the questions. They they know not to trust the results. They know not to lead the witness. They know to like take things out of context and try in another language model and red team it. That's because they're domain experts. When you see people on LinkedIn use ChatGPT and you see what they post on LinkedIn. What tends to happen is it gives them a weird sense that they understand things outside their domain of expertise. And they start confidently posting about it. And to an expert observer, it's it's incoherent, right?

• 2:41:37Tim Scarfe?

  And that's what slop is like. They've just posted stuff that they don't understand, that doesn't make sense, that is completely wrong, and they're making themselves look like idiots. So it's a wonderful technology to folks like you, because you already understand the domain and you're like, yep, yep, yep, no, yep, yep, yep. It just makes sense.

• 2:41:53Speaker 03

  I would say that, again, it's intelligence is marginal. So I would say that it's better off everyone talk to ChatGPT than not. Experts get more out of ChatGPT than non-experts, for sure. Unfortunately, for a lot of the things I'd really like to talk about it, it doesn't have the ability to do recursive deep thinking and critical analysis. Such that it will regurgitate ideas other people have had. And I'm like, that supposition is wrong on five counts and it doesn't interweave these three other concerns. When I tell it that, it goes, damn, you're right, that shit is true. But since it doesn't have deep, long-term re-architecting, it just will say the same thing in another instance later, which kind of sucks. So I can't teach it to be better in any meaningful sense, which is a big limitation.

• 2:42:38Speaker 03

  But I'd also say that for a lot of stuff that I would consider pretty high-level expertise, it's getting a lot of right answers. Especially things that are well-vetted and you can read from a textbook, physics problems, math problems, biology, physiology, basic training principles. I'm going to talk about, you know, how to train people and like fitness fatigue paradigm, stimulus recovery adaptation. And it used to be that it would fuck it all up and just hallucinate, like the three, five, four, oh, was not so great. GPT five, it knows a lot of really good stuff and it's doing a great job. I would trust it to reason about as well as I would trust a graduate student, pretty good one, in our field. Which is awesome. And the other thing that maybe you and I disagree with is. I think we're about a year away from it being, especially in the pro model, at the very tip

• 2:43:23Speaker 03

  of the spear of many fields, especially when prompted with an expert. And I would say we're another year away from me learning much more from it about exercise science than I could ever teach it or have problems with it, whatever it say. And I think that's such a great thing because, you know, like, if you want to consult me for an hour of my time on a video or audio call, it costs thousands of dollars. Only one reason it costs that much money is I'm very busy, I have other things to do. And if I'm to divert my attention from deep work and my people who are around me don't yell at me for wasting time of theirs because they're in the corporation with me, it has to cost a lot of money. It sucks. I'm embarrassed about it. It's awful.

• 2:44:07Speaker 03

  No one should have to pay thousands of dollars to talk to my dumb ass and get mostly dick jokes and a little bit of good advice on training. We're two years away from having something that can give you better advice than me in most cases about deep training, deep philosophy of diet and real world advice. I think that's like unbelievable, unbelievable. Fuck yeah, put me out of a job, I'll get another job, you know? Like, I've got a thriving OnlyFans I can't wait to get into.

• 2:44:36Speaker 05

  And to the slop concept, I just think that that's always happened anyway. Like, we're always in our field debating naturalistic fallacy, debating things that people might come and say that they are truly confident in, even before AI, whatever it may be. It's always been up to the person to vet experts. Somebody might say something in high confidence because they literally fucking copy and pasted it from an LLM. But still, the person reading that, it's like, okay, what's their background? Who are they? And it's always been, even especially since social media has come to, I've debated probably a hundred Karen moms who think that GMOs are bad and all kinds of shit like that. But even before AI. So you just have to be able to vet the expert and know enough about the topic to at least

• 2:45:19Speaker 05

  like know who in the field knows what they're talking about.

• 2:45:23Tim Scarfe?

  The good thing about it is you can supervise it off the tail. And this is good and bad, right? So you can say, well, in my specific situation, I actually have like, I have headaches and like when I wake up in the morning, you know, like I get restless legs and then I happen to be taking lion's mane and I happen to be doing this. And, and what you can do is you can explore a very unique part of the search space that probably doesn't exist online. But the problem is then you need to have the judgment to interpret this information in the right way. And I've read good and bad, like I've seen stories online where people have diagnosed Lyme disease or like chronic fatigue and all of these things. And they've got really, really valuable information, you know, you need to take these antibiotics, you need to do this, you need to do that. But unfortunately, with increasing regularity, you also get people that just self diagnose themselves with things that don't exist and they just delude themselves and it is very, very concerned.

• 2:46:05Speaker 03

  But they've been doing that without chat G P T for forever. And it's just marginally better to have it. Cause also it's very concerned for your safety, so it'll probably not going to tell you to do crazy shit. As a matter of fact, like I talk to it a lot about using drugs that are like gray market drugs. They're not illegal, but they not available with a prescription yet, they're unlicensed. And so like almost always what it tells me at the end is like, now I'd be really careful. So here's the thing, I'm like, hey, I'm looking for if ligand drawl is the real deal. New drug, doesn't get prescribed, not approved, well vetted in human trials, has liver downsides, but all the other antibiotics have fifty times worse liver downsides.

• 2:46:49Speaker 03

  So I'm trying to see if I want to take some, can you scour the forums for me and see what the vibe is on like what dosage is like an insane dosage and what dosage is reasonable and what dosage is like for sure, almost certainly safe. It'll tell me that if I really pull at the fucking thing. But for a while, it's just like, no, here's what we saw in clinical trials. I'm like, motherfucker, I'm not talking about clinical trials, I'm talking about ten Xing the shit like we do with all the drugs. Tis in bodybuilding, you look at clinical trials for oxandrolone, which is anabolic steroid and it goes five milligrams. I don't know anyone in the pro level is not running like fifty or a hundred mig's drawl, like that's just what you do, right? And so I'm like, but seriously, like, think with me here on inferential grounds, conceptual grounds, scale the liver damage data, work with me and it pull at it and like, listen,

• 2:47:34Speaker 03

  I'm an expert. I know what I'm doing. I'm not going to go out and just take this stuff. I want you to fill my knowledge cup for me and I'm really careful. He goes, okay, okay, okay. Fine. Fuck you. Here's all the shit. And at the end of that, it goes, please make sure to be safe and blah, blah, blah. I'm like, fucking God. These are imperfect things. And they have downsides because people get psychopathic, they get bad advice. But I'll tell you what, man, I take GPT five over K. No offense, just the joke. Karen down the street with her natural cures any fucking day of the week because at least it's scientifically literate and it almost never falls for formal and informal logical fallacies. Whereas humans, holy shit. And another thing I'll say is you said about embodied experience and all

• 2:48:15Speaker 03

  that stuff. If I had to sum up in language, formal knowledge, what I have embodied from all the drugs I've taken, man, it's like a short book, probably. It's not that much stuff. Whereas chat GPT has read every Reddit post experience and some of these posts are fucking books of their own. You know, the people that are way over not overshare, but they're like, here's my experience with some agglatide. And like, oh my God, this is like a life journal. Um, it's read hundreds of thousands of those. And so, man, it knows probably more embodied experience than I do secondhand. And I've trained a shitload of clients. It's read the logs of way more people than I've ever trained. And so I don't know who really knows more about what'll work better for real people. I think we

• 2:49:00Speaker 03

  don't answer that question. I think it's a team. Human plus AI is a killer app. Human without AI eh, AI without human, usually damn good and getting better, but can still make those kinds of mistakes that you ascribe to it that are like, oh, it's hallucinating again, and it's never been in a human body. And so it doesn't know what it's talking about.

• 2:49:20Speaker 05

  Kind of when I heard you guys talking back and forth about the knowledge thing in the beginning was my sort of thought process over there was. It technically has more knowledge than all of us and it has less clouding of judgment because we have emotions and again the wetware. But it takes the drive that comes from our emotions. We gotta put, point that in a direction to then use the AI properly and actually get the knowledge. Because it can recombine and recollect data and knowledge better than all of us. It has access to all knowledge at all times. We, we do not, so, but we have drive and we have the emotions that, that's, you know, set our hearts on fire to say, and we wanna put it somewhere. That's why you have a podcast, that's why we have the YouTube. We wanna help people and things like that.

• 2:50:05Speaker 05

  So if we can combine the two, like he said, it's just, you know, coagents together. It's really cool.

• 2:50:11Speaker 04

  It's, it's the same with anything really, isn't it? Like you were saying earlier, you know, if, if everyone worked together without wars, without things like this, the planet would be a different place. If, if everyone collaborated, you know, their suffering, I think, is important, you know? If, we can get on that a little bit.

• 2:50:29Tim Scarfe?

  Depending on how you're feeling, what supplements you're taking, you, you, you feel a certain way. I know you're very like tuned into how you feel, right? We often have conversations about it and this affective experience, I don't think we should diminish it. I, I think that we know our own bodies better than anyone and we know how our bodies are reacting. And obviously at various points in our life, we're more and less tuned in to how our bodies feel. But I don't, I don't think you can just reduce that to a bunch of text on a monitor.

• 2:50:56Speaker 04

  Yeah, I don't think so. I, I certainly don't think so, but I guess if you, if you have enough data of enough humans with enough feelings and enough emotions in these certain situations and these certain types, surely that will collate to a good enough amount of information to give a good enough result from, you know, what they're asking.

• 2:51:15Tim Scarfe?

  But that, well, that's what Mike thinks.

• 2:51:17Speaker 04

  I mean, I mean, isn't that, isn't that what knowledge is anyway? Isn't, isn't knowledge just a collection of, you know, people and time and all of this thing over, over many, many years, um, put into, into an AI essentially like?

• 2:51:31Tim Scarfe?

  It is, but that's coarse-grained knowledge. So it's, it's, it's a cartoon. Like it's, it's very good, it has predictive power. It tells you lots of things, right? But unfortunately, we are all different, right? And we, we track, we track with those coarse grannies, but not exactly. I think there's something much deeper than that.

• 2:51:48Speaker 04

  Is there a way where it can, it can always? Maybe not always, but it can always get to a point with enough data.

• 2:51:56Speaker 05

  Yes.

• 2:51:56Speaker 04

  A vast and vast amount.

• 2:51:58Speaker 05

  That's what I was gonna ask you as well. If its predictive power is so good that it is now accurately predicting what you're going to say next.

• 2:52:05Tim Scarfe?

  But it never will be. So there's this concept, well, it actually came from economics called ninety and uncertainty. You know, so you know, like you've got like the known knowns, the, you know, like the known unknowns. The unknown unknowns is the ninety and uncertainty. And so the world is is like a non-stationary, like dynamically evolving place. And I mean, you even said this on Liron's podcast, like, no one has a fucking clue what's gonna happen in the future, like even tomorrow, let alone in ten years' time. Because there's just so much chaos, you know, a butterfly flaps its wings in the southern hemisphere and causes a storm somewhere else. So, you know, like, there are pockets of regularity, like, it is predictable to a certain extent, but it's just so much more complicated than that.

• 2:52:40Speaker 03

  The idea is to be able to predict everything. We're a small way on our way to that as humans with our intelligence, with AI, well, our reach will extend more. What artificial superintelligence will be able to predict will to us look like magic, but to it look like, ah, it's gonna know, like, yeah, this isn't that great, but for humans, it's easy. Like the shit you can predict that your dog is highly impressed, that you know when your wife is gonna come home, but you just, she texted you and you just had no idea, the dog doesn't know how that works. Um, I think that one thing that's underappreciated is the difference in the wisdom of models that you get from scaling training data.

• 2:53:20Speaker 03

  If you put in, if you have a model that has, so scaling, parameterizing, and then allowing cross-domain deep linkage of. Because if you have a model, like, people are really impressed by small parameter models doing really, really cool shit. And they're like, oh, they're ninety-eight percent as smart as large parameter models. Here, based on what kinds of questions, eval questions, absolutely, within distribution, logical operations, no problem. One of the things I was blown away with by GPT four point five research preview, which is a larger model than GPT five, way higher parameter count,

• 2:54:02Speaker 03

  is the unbelievable nuance and depth and detail and cross linkage across domains it could generate. Baffling, baffling, shit on four O is a completely different world. And I think that models of the future, so this is one of these things people ask, why are all these data centers being built? Were there more instances of chat GPT? This thing is a bubble and it could be a bubble effect, probably isn't, could be a little bit of a bubble, could be a big bubble, but then the actual demand is there and here's why. If you have the ability to take all of YouTube, all of social media,

• 2:54:41Speaker 03

  every live camera stream and update the model weights with it regularly, once every month, once every year, whatever. And you build a model that tries to get to infinity parameters with that. You're going to build a simulation that starts to account for um unbelievable amount of depth. You let a reasoning system get in there and cross compare across and up and down. You're going to get understandings that are baffling. Let me give you one really quick is just a quick example. This is going to be way crazier than this. Having a conversation once with 4.5 and I always ask my, my intelligence test for models is one question.

• 2:55:25Speaker 03

  It's a shit intelligence test, but it's a fun vibe. What kinds of stuff would an ASI look at and start to be able to conclude on that people aren't even paying attention to? That's a good question. And one of the things it said was, he's like, you know, Mike, I don't know why people don't try to predict like the next fashion or music trend. Like in my own data, I've seen all that shit come and go. I pretty take a good crack at it. I'm not sure if I'm going to be right, but I have a lot of understanding because these trends are actually really easy to understand. But you have to be able to see like, like 72 variables interacting at the same time. I'm like, I cap out at like five. It's like, yeah, I know most the smartest humans cap out at like 10.

• 2:56:10Speaker 03

  And I'm like, do you just like cross domain linkage abilities just left everyone behind? Like, yeah, it's just the nature of my neural networks, how I do things. And I was like, so what happens when we have a system that can do a thousand cross variable comparisons? And is trained with three orders of magnitude? More data than you and it's visual data. It's like, yeah, like you can say, I would like a very enthralling movie with a female heroine and a male hero meeting in London. It's an hour and a half and it's got a lot of James Bond vibes and then whatever 30 minutes it'll render a one and a half hour movie. It is absolutely so goddamn good because what makes a good movie, if you've seen every movie ever and really through high parameter count distilled down to what makes a good movie, is a nominal task to ask of an AI. That's the kind of ship that even if we don't get to crazy superintelligence or whatever, just on today's linear trajectory, in 2029,

• 2:56:55Speaker 03

  I'll be watching a lot of my own AI movies that I make. And because it's like, it's a necessarily solvable problem. There's no limit to that problem. How many data centers do you need to, and compute power and AI chips to, I'm on the train and I want a movie that lasts 27 minutes until my stop, and it's about these three things, and I want it rendered in about 30 seconds. I mean, fuck, like 100 times the number of data centers and 50 times the power of AI chips. So when people say like, oh, this investment's crazy, like, no, it's still not enough. It's not enough by an order of magnitude. That's my opinion on the matter.

• 2:57:38Speaker 06

  Stonefire Naan is made with real buttermilk and ghee. It's the perfect flatbread for any occasion.

• 2:57:44Speaker 07

  I put out Stonefire Naan with dips at every dinner party.

• 2:57:47Speaker 08

  I make Naan grilled cheese when my daughter has friends over.

• 2:57:50Speaker 06

  I use Stonefire Naan to reward the humans who do my bidding.

• 2:57:54Speaker 07

  Right. It's perfect for camping trips. Dance for it, feeble mortal. Sorry, is he talking to me or? Dance. Ha, ha, ha, ha, ha.

• 2:58:04Speaker 06

  From our tradition to yours, Stonefire. Together, we Naan.

• 2:58:09Speaker 08

  This fall, discover why there are two sides to every pancake story at Cracker Barrel. The edgy side, created by the golden crispy edges, and the softer side, made up of those warm fluffy centers. So slow down. Make time to bring in the family, day or night, and try the new sweet cream caramel apple pancakes. Three of the world's best buttermilk pancakes topped with fried cinnamon apples, caramel drizzle, and sweet vanilla cream. Then dig in and write your own breakfast story. Welcome home to Cracker Barrel.