What Is Intelligence? With David Krakauer

Why did intelligence evolve in the universe? Neil deGrasse Tyson, Chuck Nice, and Gary O’Reilly explore the nature of intelligence, information theory, consciousness, and whether AI is actually intelligent at all with David Krakauer, evolutionary biologist and president of the Santa Fe Institute.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Meu Deus, és linda. Obrigado por mi vida. Translation: My God, you're beautiful. Thank you for listening. Yes, I'm talking to you, but I'm talking to you in Português. That is correct. You never know when you're gonna find yourself in Portugal, or when you're gonna talk to somebody from Brazil, who also speaks Portuguese. You know, if you're gonna travel anywhere, you'd be surprised how much it helps to know the language, even if you don't know it fluently. But let's say you do wanna know it fluently. Well, I have an answer for both. Rosetta Stone. That's right. Rosetta Stone has just launched Rosetta Stone Sapphire, featuring Sapphire Studio,

• 0:43Speaker 00

  which lets you personalize your learning around what you actually care about right now. Now, we all know that I use Rosetta Stone so that I can understand what my mother-in-law is saying about me behind my back. But maybe you're traveling and you just wanna know, hey, what do I say when I check in? How do I talk to the taxi driver? Rosetta Stone Sapphire is gonna allow you to talk to people in real-world situations in their own language. And the great thing is you can do it on your phone, on your tablet, at your own pace, on your own time schedule. So what are you waiting for? Go ahead and take your language skills to the next level. Don't wait to try Rosetta Stone Sapphire. StarTalk listeners can get 20% off their Rosetta Stone Sapphire subscription

• 1:27Speaker 00

  when they sign up today. This gets you access to Rosetta Stone Sapphire in 25 languages. So go to rosettastone.com and sign up today.

• 1:40Gary O'Reilly?

  StarTalk Radio is presented by ChatGPT. ChatGPT Work, a new way to work in ChatGPT, helps you move from goals to real outcomes. ChatGPT Work can use info from your apps, files, and plugins, like Slack, calendars, and meeting notes, to help produce finished materials. Plus, it can stay with bigger projects for hours while you still remain in control. Put ChatGPT to work on your most ambitious ideas and projects. Get started at chatgpt.com by selecting Work Mode, available on Plus and Pro Plans.

• 2:21Speaker 02

  That was quite the overview on what it is to be intelligent. It went places I didn't even know intelligence went.

• 2:27Speaker 00

  Deep. Yeah, and for some reason, I still feel stupid.

• 2:32Speaker 02

  Coming up, StarTalk Special Edition. Everything you need to know about intelligence.

• 2:38Speaker 02

  Welcome to StarTalk, your place in the universe where science and pop culture collide.

• 2:47Speaker 02

  StarTalk begins right now.

• 2:51Speaker 02

  This is StarTalk Special Edition. Neil deGrasse Tyson, you are a personal astrophysicist. And if it's Special Edition, it means Gary O'Reilly in the house.

• 3:01Gary O'Reilly?

  Hi, Neil.

• 3:01Speaker 02

  Hey, former soccer pro, soccer announcer. Of course, we got Chuck Nice, baby.

• 3:06Speaker 00

  That's right, tempest in a teacup.

• 3:07Speaker 02

  The Lord of Nice.

• 3:09Speaker 00

  Lord of Nice.

• 3:10Speaker 02

  All right.

• 3:11Speaker 00

  Exactly.

• 3:12Speaker 02

  So Gary, what did you and your producers stitch together for today?

• 3:17Gary O'Reilly?

  Well, with all this talk of artificial intelligence, has everyone stopped to think, what is intelligence in general? That's what we're doing today.

• 3:27Speaker 02

  Everybody thinks they know what it is, and no one can really define it.

• 3:29Speaker 00

  That's because they're all dumbasses.

• 3:32Gary O'Reilly?

  To Chuck's point, how did intelligence evolve and why? What chances are there of beings far smarter than us? That's a low bar. Are we? If we are the universe's way of knowing itself, is the universe screwed?

• 3:51Speaker 02

  I mean, if we're the best it's got to figure itself out.

• 3:55Gary O'Reilly?

  You know, maybe I want to know that answer, maybe I don't. But we're gonna ask it anyway.

• 4:00Speaker 02

  So, and that quote, I think, is traceable to Carl Sagan. It might have been in one of the episodes of Cosmos, the original Cosmos. Well, let's find out what our guest has to say. You talk about did intelligence evolve and in what way? We have an evolutionary biologist. Yeah, you need them every now and then, straighten things out. And he's also president and the William H. Miller Professor of Complex Systems at the Santa Fe Institute. Of course, that would be in New Mexico. Yes.

• 4:31Speaker 00

  I'm sorry, you mean New America?

• 4:36Gary O'Reilly?

  We're spotted, Chuck.

• 4:40Speaker 02

  And he specializes in complexity and intelligence. Join me in welcoming David Krakauer. David, welcome.

• 4:47Speaker 03

  Wonderful to be back in person.

• 4:49Speaker 02

  In person. We can verify that. Poke him, make sure he's real.

• 4:53Speaker 03

  I'm here.

• 4:53Speaker 00

  Not a hologram.

• 4:54Speaker 02

  Not a hologram. So, I'd like the fact that you study complex systems, but you have roots in evolutionary biology. Because that's where we are life forms, and that's where it all begins. So, could you just shed some luminosity on the evolution of intelligence as we think about it?

• 5:16Speaker 03

  Yeah, well, okay. So, so the first question is, what is the relationship between life and intelligence? Right? Can one conceive of a life that wasn't intelligent? Can one consider or conceive of an intelligence that's not alive? I think the latter, yes, and we might talk about that. But the former, probably no. And so, as soon as you have life, you have some form of intelligence.

• 5:45Speaker 03

  So what is that? And I guess the canonical, simple argument would be, these are some form of organized matter that can sense the environment and act in a way that increases its representation in the world, either through replication or what have you.

• 6:04Speaker 02

  In the interest of its own survival.

• 6:05Speaker 03

  In the interest of its own survival. And so, any life form that can sense and behave in a way that's adaptive, we would have to grant, has some form of intelligence. So the interesting theoretical problem here is that life feels a bit binary. You don't say that a flea is less alive than an elephant. Right? So that's kind of an interesting fact. Whereas intelligence is not. So how does a kind of binary system give rise to a continuous one? And that's kind of the theory challenge for the origin of intelligence.

• 6:37Speaker 02

  But tell me again what you mean by the binarity of this?

• 6:40Speaker 00

  Yeah, I mean, you're either alive or you're not. But that doesn't mean that the level of intelligence you have is dependent on you being alive or not.

• 6:48Speaker 02

  I see what you're saying.

• 6:48Speaker 00

  It's a gradation.

• 6:49Speaker 02

  So intelligence has a continuum. You either alive or dead does not. Got it, okay. Just to clarify that.

• 6:55Speaker 03

  So that's, okay, so that's the first question, right? So, over the course of time, what are the forces that have created higher levels of intelligence? If they have? That's an interesting question in itself. Maybe it's entirely context-dependent. Maybe there's a sense in which a bacterium in the world is more intelligent than a human destroying a world. You know, one could go there if one wanted.

• 7:16Speaker 02

  In other words, you would say humans with high intelligence who destroy their world are not acting in the interest of their survival.

• 7:21Speaker 03

  I would say they're stupid.

• 7:22Speaker 02

  And that would, that would...

• 7:23Speaker 00

  Definitely stupid.

• 7:24Speaker 02

  There would be a demerit against their intelligence scale.

• 7:27Speaker 03

  Absolutely.

• 7:28Speaker 00

  Yes. And what about this? Because when you said bacterium, is there any kind of collective intelligence where the whole is greater than the sum?

• 7:39Speaker 03

  For sure. I mean, in a sense, that's our own brains, right? So we have billions of nerve cells, each of which is quite simple on its own, and they connect and they create thought, mathematics, music, all the things that we have.

• 7:50Speaker 02

  Chuck, that was a stupid question.

• 7:52Speaker 00

  Well, no, I didn't mean, I didn't mean as a part of one entity like our brain. I meant like our gut, where those bacteria are all different types, but they work together to cause us to have emotions and cause us to think certain ways. They're operating in tandem with our brain, but they're all separate bacteria.

• 8:15Speaker 03

  Absolutely. I mean, there's this slight paradox that happens. So this is the world of collective intelligence, like swarms and shoals and schools.

• 8:22Speaker 02

  Right, because normally when you think of a collective intelligence, is the ensemble of organisms operating in a way that one individual would not otherwise understand, but the ensemble does.

• 8:37Speaker 00

  Right.

• 8:37Speaker 02

  I mean, this is emerging things like flocking.

• 8:40Speaker 00

  Flocking.

• 8:41Speaker 02

  Of birds, I guess.

• 8:42Speaker 00

  Or, yeah, or when you see fish change all direction all at once. Even though, I don't even know if that's like electrochemical or not, but who knows? You know?

• 8:51Speaker 03

  No, but there's an interesting thing that happens here that's surprising, right? Which is that brains made of very simple cells produce extraordinary thoughts. Whereas human societies, kind of made of extraordinary individual thoughts, produces a total shitshow in the aggregate. So there's an interesting relationship between parts and wholes. If you have very smart parts, you don't necessarily have a smart whole.

• 9:14Speaker 02

  That's true. Because there's nothing less intelligent than a mob of people with pitchforks and tiki torches.

• 9:21Speaker 00

  True.

• 9:23Speaker 02

  It seems to be inverse correlated with the number of people who are part of the mob.

• 9:27Speaker 00

  Right, yeah.

• 9:28Speaker 02

  They act less and less intelligently.

• 9:30Speaker 00

  Less intelligent, but more and more as a unit.

• 9:34Speaker 02

  Yes.

• 9:35Speaker 00

  Isn't that funny?

• 9:35Speaker 02

  Okay.

• 9:36Speaker 00

  The less intelligent that mob is, the more...

• 9:40Speaker 02

  The more monolithic. The more monolithic it is in intent and action.

• 9:45Speaker 03

  Right, and that, that is in some sense gets back to this question of evolution. Because in evolution, you call these major evolutionary transitions. So you go from a single cell, like a prokaryotic cell, like a bacteria, bacterial mat, kind of a collective, a single eukaryotic cell, to a multicell, to a tissue, to an organism. So that is the story. And in every case, it's not clear whether the aggregate is somehow more restrictive, less intelligent than the parts.

• 10:17Gary O'Reilly?

  So if you're talking about our own evolution here, when it comes to our human intelligence, was the turning point when we get complicated language? Or was it something else?

• 10:29Speaker 03

  Well, I mean, that, you know...

• 10:30Speaker 02

  Wait, let me, let me prepend that. Because you're saying, well, we have language and, you know, other animals don't. Or do they not have language? I mean, so let's broaden that question to when organisms communicate complex information.

• 10:50Speaker 02

  Do we call it language when a bee communicates to another bee in a hive where where to go or where to move the camp? So and on the limit that we would think of as a limit as our complex language.

• 11:03Speaker 00

  Yeah.

• 11:04Speaker 02

  Do you distinguish as an Evo Bio person? Not evil bio, but Evo.

• 11:09Speaker 00

  Evo.

• 11:09Speaker 02

  Evo Bio person. Do you distinguish dolphins squeaking at each other from humans squeaking at each other? Or not?

• 11:19Speaker 03

  Yeah, it's a difficult one. So I think the jury is out. But one quite useful distinction I quite like. So in the biology community, people talk about signaling and communicating as being somewhat distinct from language.

• 11:34Speaker 02

  Sure.

• 11:34Speaker 03

  And so I guess with human language, Alexander von Humboldt had a nice phrase that I like here. He said.

• 11:40Speaker 02

  Tell us briefly about him.

• 11:41Speaker 03

  So he was a great 18th century naturalist, kind of invented ecology as we know it, was a great explorer, took extensive measurements, worked on weather systems.

• 11:51Speaker 02

  And he was German, French?

• 11:52Speaker 03

  German.

• 11:52Speaker 02

  German, German.

• 11:53Speaker 03

  Yeah. And actually, my friend Andrea Wolf wrote his biography, The Invention of Nature, beautiful book. Plug for the book.

• 12:00Speaker 02

  And if I remember, I think I own some of his books. Didn't he write about?

• 12:04Speaker 03

  Cosmos.

• 12:04Speaker 02

  What's that?

• 12:05Speaker 03

  He wrote a whole series of books called Cosmos.

• 12:07Speaker 02

  Yes. And I think one of his books tries to connect our understanding of science to make a better society. I mean, I think he was a civic scientist as part of his intellectual efforts.

• 12:19Speaker 00

  Oh, he'd be so happy if he saw us today.

• 12:22Speaker 03

  I know. Well, but that's an interesting point. I mean, that romantic movement where they were all in some sense entangled with each other. You didn't get to just be a scientist or a good citizen. You should be both.

• 12:33Speaker 02

  Entangled in a good way.

• 12:35Speaker 03

  In a good way.

• 12:36Speaker 02

  So tell us more. I'm sorry, I interrupted you.

• 12:37Speaker 03

  So, yeah, so he has this phrase where he says, human language is making infinite use of finite means. And what he meant by that is if you have a grammar, like a rule system, you can construct almost any thought out of it.

• 12:52Speaker 00

  Absolutely.

• 12:53Speaker 03

  And that flexibility that you get with language is not as evident in simple signaling systems like chemical ones or even gestural ones, actually.

• 13:02Speaker 00

  And how about, now, we know whales have a certain type of language. They don't, we don't know how extensive it is, but humpbacks call for help when killer whales or orcas are around and the other whales respond. So clearly.

• 13:20Speaker 02

  They call for backup?

• 13:21Speaker 00

  They call for backup, you know? And they'll even attack the orcas when they're going after seals. They're not just going after the other whales. So that's language. Howler monkeys warn each other about jaguars who are in the forest.

• 13:41Speaker 02

  Just to be clear, jaguars, if memory serves, can climb trees in ways that lions cannot. So a monkey is not entirely safe in a tree.

• 13:48Speaker 00

  And that's exactly why they warn each other.

• 13:50Speaker 02

  They have the concern.

• 13:51Speaker 00

  Right. So what I'm saying is that can be considered signaling, but it can also be considered language.

• 13:57Speaker 03

  Well, okay, so, all right. So, yeah, so people, so there's an American philosopher, Charles Pierce, and he made a distinction. So I'm going to get into the signal thing.

• 14:05Speaker 00

  Okay.

• 14:06Speaker 03

  He made a distinction between what he called an index, an icon, and a symbol. So the point is this, so vervet monkeys, Chaney and Seifert famously worked on these different signals, birds. So if you say falcon, or they emit a sound, they crouch down. So there's a correlated behavior, right? And if there's a kind of terrestrial predator, they'll climb up a tree and so on. So they have those. And in the kind of signaling literature, that would be described as an index. But there's a kind of mapping, one to one. That sound always means watch out, there's an aerial predator.

• 14:43Speaker 02

  Pavlov knew all about this.

• 14:45Speaker 03

  Completely, absolutely. So you could say that's how it's acquired, maybe, or innate and so on.

• 14:49Speaker 02

  Or that's where it is.

• 14:50Speaker 00

  It doesn't even have to, it can be just a sound, like the bell.

• 14:53Speaker 02

  Anything.

• 14:54Speaker 00

  Right, okay.

• 14:55Speaker 03

  Now, in the human case, you can substitute any sound for any appropriate response of almost arbitrary complication.

• 15:04Speaker 00

  Correct.

• 15:04Speaker 03

  And that's something you learn in your lifetime.

• 15:06Speaker 00

  Wow.

• 15:07Speaker 03

  And I think that, so it's not that they don't have something that's embryonic of our ability, but what we do is quite extraordinary.

• 15:14Speaker 00

  Oh, yeah. I didn't think of it that way, but you're absolutely right. Because now I'm thinking about in all the movies, they're like, and here's the signal. Cuckoo!

• 15:22Speaker 03

  Right. And that means take out your math textbook.

• 15:25Speaker 00

  So, yeah. Okay, all right, well, that's very good.

• 15:38Gary O'Reilly?

  You may have heard the best voice in show business, Morgan Freeman, talking about a serious and underdiagnosed heart condition that's often missed, A, TTR, cardiac amyloidosis or A TTRCM. It's a condition that can greatly disrupt your life with symptoms like severe fatigue, shortness of breath, and carpal tunnel. If left untreated, A TTRCM may become serious, leading to a shorter lifespan. Attruby helps adults with A TTRCM live longer and have fewer hospitalizations due to heart issues, so you can focus more on living for what you love. Tell your doctor if you're pregnant, plan to become pregnant, or are breastfeeding, and about the medications you take.

• 16:23Gary O'Reilly?

  The most common side effects were mild and included diarrhea and abdominal pain. If you have A TTRCM, talk to your cardiologist about Attruby and visit attruby.com/podcast. That's A-T-T-R-U-B-Y.com/podcast to learn more. It's time to get busy living. Brought to you by Bridge Bio.

• 16:49Speaker 00

  Since when has building a website been hard? Since forever until now. That's right. With Wix, the first of its kind AI website builder where you get the best of AI and drag and drop editing in one place. Wix Harmony allows you to build a real working website for yourself in minutes with the business stuff already built in. I'm talking e-commerce, scheduling, payments, all of it. Plus, easily switch between prompting AI and editing things by hand whenever you want so every detail of your site is exactly right. It's like having a website engineer looking over your shoulder and at your fingertips at the same time.

• 17:34Speaker 00

  Aria is an AI agent specifically trained in web design and business strategy. Click anywhere on the website and ask her to make a specific change instead of endless back and forth prompting. It all runs on Wix's tried and true infrastructure with 99.99% uptime and enterprise-grade security so your site performs as good as it looks. Wix has been a pioneer in website tech for more than 20 years, powering millions of businesses around the world. So what are you waiting for? Go ahead and be that website building genius that you are. Try Wix Harmony for free at wix.com.

• 18:16Speaker 00

  That's W-I-X dot com. Hmm, let's see. Cashmere hoodie, cashmere crunet, long sleeve tee, performance pants. Hmm, what else? Oh, what am I doing right now? I am kind of counting some of the pieces in my closet that make up the staples in my wardrobe. I'm talking about easy pieces that make fashion effortless and make me look like a fashion guru. Not that I'm not, let's be honest. Quince is what I'm really talking about because Quince makes elevated everyday essentials using premium materials and designs that are timeless, thoughtful, crafted, and made

• 19:01Speaker 00

  to be worn again and again. And that's exactly what I do. And when the temperature starts to drop, Quince makes it easy to build on those everyday staples with versatile layers designed to keep you warm and comfortable all season long. If you're new to the podcast, you probably don't know that I've been a longtime customer of Quince long before they started supporting StarTalk. And that's why I'm super happy to tell you about them. All the pieces that I named up front, I have and more. Not to mention the number of family members that have received their excellent cashmere pieces from me as a gift. Keep that in mind because you know the holidays are coming.

• 19:44Speaker 00

  Find your next fall favorites at Quince. Download the Quince app for app exclusive offers or go to quince.com. Get free shipping on your order and 365 day returns, now available in Canada and the UK too. And when Quince asks where you heard about them, let them know that it was StarTalk Radio and the fashion guru Chuck Nice that told you all about it. It's the best way to support the show, quince.com. Go there right now.

• 20:14Speaker 02

  Hmm, quasars. The word is a pseudo acronym for quasi stellar radio source. What's a quasar? These are objects that are among the most distant things known in the universe. How do we know about them at all? Because they're really, really luminous. Some of them as luminous as 500 trillion suns. That's why we can see them to the edge of the universe. If you like that fact, you can find 4,999 more.

• 20:55Speaker 02

  In Lost in Space. 5,000 facts to help navigate the universe. Lost in Space is now available wherever you get your books.

• 21:20Speaker 02

  Does the expression of complex language predate the development of what we think of as our intelligence, or did it follow it?

• 21:28Speaker 03

  Good question.

• 21:29Speaker 00

  Wow.

• 21:29Speaker 03

  Good question.

• 21:30Speaker 02

  Why don't you have answers to?

• 21:31Speaker 03

  Yeah.

• 21:31Speaker 02

  Why did we invite you on the show?

• 21:33Speaker 03

  I don't know.

• 21:35Speaker 03

  To humiliate.

• 21:36Speaker 02

  I don't want you to hear that. That's a good question. I want you to say, I have an answer to that.

• 21:40Speaker 03

  Yeah, I don't, these don't, you know, the reality is, you know this, they don't have final conclusive answers. They're not even close. But I would say that in, okay, so I'm going to define it. I think I did last time I was talking to you guys. So intelligence for me is making hard problems easy. Okay? That's basically it. Across the animal species, every species, even plants. So, you know, finding an efficient way of solving a problem, not an inefficient way of solving, we get to AI in a minute, which is all about inefficient solutions to problems, but that's something else. So I think that's everywhere. All life forms find elegant ways of solving problems in their world. Now, the question is, what does language add above and beyond an innate repertoire or even a simple Pavlovian association?

• 22:23Speaker 03

  And I think the possibility for almost arbitrarily complex behaviors, which don't necessarily produce intelligent behavior, right? They could produce warfare and so forth and strife. But that's the thing that we do. And I think it somewhat decouples from the question of intelligence, right? It creates great intelligence, but it also creates great stupidity.

• 22:42Speaker 02

  All right.

• 22:42Gary O'Reilly?

  How are we distinguishing between intelligence and consciousness?

• 22:49Speaker 00

  Wow.

• 22:49Gary O'Reilly?

  Yeah, so.

• 22:50Speaker 02

  We're just barely getting intelligence here. You could throw consciousness right on top of it.

• 22:54Speaker 03

  I actually think consciousness is easier than intelligence.

• 22:56Gary O'Reilly?

  Because.

• 22:57Speaker 03

  Easier.

• 22:58Gary O'Reilly?

  And I've read some of your work and you cite the fact that mathematicians will tell you their unconscious mind, subconscious, unconscious will solve the math problem, but not your conscious mind.

• 23:09Speaker 00

  Interesting.

• 23:09Speaker 03

  Yeah.

• 23:10Gary O'Reilly?

  So between intelligence and consciousness, how we distinguish?

• 23:13Speaker 03

  So, okay, I want to say two things.

• 23:15Speaker 02

  You're talking about the old age, sleep on it.

• 23:17Speaker 03

  Yes, sleep on it. Exactly, sleep on it. The Kikuli problem, exactly that. So let me do a bit of both here. I'm not sure the best way to do it. So on the consciousness side, I quite like this way of defining it that Aneel Seth, what some consciousness does. He'll say, intelligence is about problem solving and consciousness is about being and feeling. So I consider consciousness very primordial, right? In other words, any organism that lives in an environment that it has to sense and maintain its integrity in some way, it could be a single celled organism, it could be a cat, right? It could be a whale. They will manifest in consciousness as about feeling and the sense of self that's required to survive, right? If you didn't have a sense of self, it's like, oh my God, my foot's gone, doesn't matter, right?

• 24:02Speaker 03

  It sort of matters. And intelligence is about problem solving. Okay, now, so that's the first point. Now, unconsciousness that we were talking about is very interesting, right? Which is, for most of our evolutionary history, we didn't have language. And so somehow we were having to solve problems in the world, as every other animal does, without that kind of declarative self-awareness. And so in that sense, there's more, if you like, neural resources available to solve problems unconsciously than in that tiny little thin layer of cells that we call the frontal cortex, that we associate with conscious deliberation.

• 24:36Speaker 02

  Just because early on, we didn't have a language, doesn't mean the thoughts were subconscious.

• 24:41Speaker 03

  Well, that's also an interesting question, right? So for most people.

• 24:44Speaker 02

  Stop telling me I have interesting questions.

• 24:45Speaker 03

  Well, you do, man. And so this is the thing where a lot of people would associate self-consciousness with deliberation, which seems to depend on the possession of language.

• 24:58Speaker 00

  Yes. Yes. But that's only because we have the language that we have, okay? Because we don't know if a lion is deliberating, you know, or if other apes that were a part of that family, if they actually look up and go, damn, you know, if I didn't want to throw my poop right now, that'd be a very beautiful thing to look at. You know, we don't know that.

• 25:26Speaker 03

  You don't know.

• 25:27Speaker 00

  But the fact that we have this inner dialogue is the reason why we believe that we are, I think, therefore I am, that, you know?

• 25:34Speaker 02

  Right, because you, we've all seen Animal Kingdom videos where two animals, typically the same species, there's a standoff and they sort of pace back and forth. You know something's going on in their head.

• 25:46Speaker 00

  Something has to be going on.

• 25:47Speaker 02

  Something like, should I do this?

• 25:48Speaker 00

  Right.

• 25:48Speaker 02

  Would I make it out of this alive?

• 25:50Speaker 00

  You don't want none of this.

• 25:52Speaker 02

  Don't make me.

• 25:53Speaker 00

  You don't want none of this. You're not built like that, son.

• 25:57Speaker 02

  Don't make me put my foot up your ass.

• 25:59Speaker 00

  Exactly.

• 26:00Speaker 02

  You can't tell me nothing's going on in their brains.

• 26:02Speaker 03

  I wouldn't say that for a second. I think that, so the question of intelligence, I can, as I said at the beginning, is ubiquitous and degrees of it, and high degrees of it.

• 26:10Speaker 00

  Right.

• 26:11Speaker 03

  In the non-human world.

• 26:12Speaker 00

  Absolutely.

• 26:12Speaker 03

  But there's an interesting fact, right? Which is that there is a tighter coupling between signaling and behavior in non-humans and humans. You could watch humans all day long in a library and have no idea what they're doing or deliberating about. And the examples you gave in the, in animal behavior, there are clues in terms of how they're manifesting their intelligence in their behavior. Language is this weird thing that is kind of almost decoupling us from our immediate space-time context. That bit is the bit that feels, and it might not be, somewhat unique to our species.

• 26:49Speaker 00

  No, and it might not be, but you're absolutely right. And earlier offline, David and I were talking about comedians and how improvisation is this very unique thing that they do. And I mentioned to him that when I'm on stage and I'm improvising, the thing that's weird is I can almost feel my brain being semi-conscious. There's like a part of my brain that takes over that moment and I'm not actually thinking.

• 27:22Speaker 02

  Have you seen a doctor?

• 27:25Speaker 00

  I know. And most people right now are listening like, Chuck, are you ever actually thinking?

• 27:30Gary O'Reilly?

  But is that intuition? Does that become an intuition?

• 27:33Speaker 00

  Well, no, it's funny because what I, the way I look at it is, people say that you have a subconscious mind or an unconscious mind. And what I feel when I'm in that moment is that that part of my being, my subconscious, is accessible to me. So that it's kind of a slight overlap in the Venn diagram of my consciousness and unconsciousness, where my executive function is able to dip into the subconscious and pull something out. That's, that's the way I describe it.

• 28:09Speaker 03

  And I think that's, I mean, that's the, I mean, it's interesting point that's not talked about enough, this distinction between, if you break intelligence down a little bit into skill and expertise. So expertise, the essence of it is practice that gives rise to automaticity, that you don't have to think about it. If you're a great athlete, right? We were talking about that, you're taking a shot. The last thing you want to do is think about it.

• 28:30Speaker 02

  Because you're not going to make it.

• 28:31Speaker 03

  And exactly the same thing, you're not going to take the shot. It's exactly the same with math, right? You're learning mathematics.

• 28:35Gary O'Reilly?

  Stay out of your head.

• 28:36Speaker 03

  Exactly. So there is this very thing that happens, it looks as if you need language, that early deliberative process.

• 28:44Gary O'Reilly?

  Yeah.

• 28:44Speaker 03

  And then you practice out of the language. And then you're returning to a state that's not even linguistic, right? So it's almost pure program, pure reflex. So you, but you have to do it via language. So we learn mathematics like we learn sports.

• 28:57Speaker 00

  Right.

• 28:57Speaker 03

  Right, you practice it enough, you're not thinking about, oh my God, how do I integrate that function? You just do it. And I think expertise can, is in fact, paradoxically, not thinking.

• 29:07Speaker 00

  Yes.

• 29:08Gary O'Reilly?

  Right? Because you've built the muscle memories.

• 29:10Speaker 03

  Or the conceptual memories.

• 29:11Gary O'Reilly?

  Yeah.

• 29:12Speaker 02

  Right, so you're not consciously thinking.

• 29:14Speaker 00

  Right.

• 29:14Speaker 02

  When you're in that zone, your subconscious is feeding you.

• 29:18Speaker 00

  It's feeding.

• 29:18Speaker 02

  In the moment.

• 29:19Speaker 00

  Right.

• 29:19Speaker 02

  Without forethought, it's happening in the moment.

• 29:21Speaker 00

  Right.

• 29:22Speaker 02

  Yeah.

• 29:23Gary O'Reilly?

  So our conscious mind has access to our senses, all the whole sensorial palate, and it's a survival. But our subconscious, our unconscious mind, does that have access to the senses? And is it then, doesn't want anything to do with our survival, it's got another executive sort of.

• 29:40Speaker 02

  Oh, I like that.

• 29:41Gary O'Reilly?

  Mission statement.

• 29:42Speaker 02

  Because, you know, when you're dreaming, you're not really responsive to touch. I mean, nobody, you know, it's, you're disconnected, you can play music, the dream is on its, in its own space. It's almost like it's making its own sensory world.

• 29:54Speaker 03

  So as you, my colleague, the novelist, Cormac McCarthy, he wrote this beautiful essay called The Kikuli Problem.

• 30:01Gary O'Reilly?

  Yes.

• 30:02Speaker 03

  Right, the discovery of the structure of the benzene ring. And that was kind of a dreamt solution.

• 30:06Gary O'Reilly?

  That the snake eats its tail.

• 30:07Speaker 03

  The snake eats its tail. And Cormac's point was very interesting. He said, the unconscious mind evolved long before language emerged, right? Because other animals presumably have an unconscious too, right? And it doesn't want to communicate to us in language. So it communicates with us visually. So for example, the auroboros, the snake that eats his own tail. And so Kikuli wakes up and he says, wait a minute, what's going on? What are you trying to tell me? Why can't you just tell me? And the unconscious says, I don't do language, you moron, right? I do symbols, I do visual intuition.

• 30:41Gary O'Reilly?

  Yes.

• 30:41Speaker 03

  And then he imposes on it the language of modern chemistry through language.

• 30:46Speaker 00

  Wow.

• 30:46Gary O'Reilly?

  Why are we programmed to absorb information that way, in a better way? Because that's, that's the problem solving aspect. Obviously, if we're asleep, there's no noise, the brain is quiet.

• 30:56Speaker 02

  Well, you value judging it, saying it's better. I would say it's different. And, and as an educator, educators know, some people are visual learners, some are rote learners, some are, so what you just said, it cracks this nut. Of human cognition for me.

• 31:12Speaker 03

  Mm hmm.

• 31:13Speaker 02

  Because like, for example, when I'm speaking typically, I use my hands and not just in a in an Italian way, we put emphasis like this. I will gesture an image of what I'm also saying. I'll say earth is going around the sun so that there's another way it's entering their mind. And I find it to be much more potent that way, accessing both parts of their brain.

• 31:41Speaker 03

  Yeah, we're totally multimodal. I mean, exactly as you say, I mean, it's visual, olfactory.

• 31:46Speaker 02

  Right.

• 31:46Speaker 03

  And linguistic, they all buttress one another, right? And consolidate. But this question of, I mean, the argument would be that what we're calling for at the present moment, the unconscious, whatever that means, and these words all a little bit fluid.

• 32:02Speaker 02

  Yeah.

• 32:02Speaker 03

  Is older, so to the evolutionary question, been around for much longer.

• 32:06Gary O'Reilly?

  Ancient animal brain, don't you?

• 32:08Speaker 03

  It is, it is exactly that. So animals are calculated trajectories, using, you know, rules of thermodynamics all the time. We then give it this additional gloss.

• 32:16Speaker 02

  Yeah.

• 32:17Speaker 03

  But a lot of the calculation, and this has been written about endlessly by mathematicians reflecting on how they solve problems, they're not necessarily solving the problems deductively, symbolically, they go to sleep, dream clues to the solution to the problem, and then wake up and impose on top of it this secondary armature of language, which is useful because you can communicate it. I'm, how could I communicate to you? Well, I dreamt of this snail that's sort of eating a leopard, and you'd say to me, what are you talking about? So the thing, so the symbolic thing doesn't naturally lend itself to communication in the way language does.

• 32:50Gary O'Reilly?

  Okay.

• 32:51Speaker 00

  However, you know, for the longest time, we did communicate in symbols, written form. So before we had alphabets, which everybody does now, which some of the alphabets of the world right now are pictograms, they're not actual.

• 33:05Speaker 02

  Chinese.

• 33:05Speaker 00

  Chinese kanji, right? So, you know, for the longest time, we drew pictures to communicate our thoughts. And once we figured out words, that's where we are now, so.

• 33:18Speaker 02

  But getting back to your point of the source of where an insight might come, there's the the famous discussion of the mathematician Ramanujan, is that, if I remember his name correctly, who, he was Hindu, and he would say, the solution just came to him. And it was, you know, Vishnu or Krishna putting the the solution on his tongue to be shared with others. So.

• 33:45Speaker 03

  Well, he was wrong, it was Jesus.

• 33:46Speaker 02

  Jesus.

• 33:51Gary O'Reilly?

  You're only encouraging him.

• 33:52Speaker 02

  So, so in what way does one's cultural bias influence what they think is the operating force on their thoughts?

• 34:02Speaker 03

  That's a very, you know, that's a very difficult question to answer, right? Because presumably, I I would guess that the underlying unconscious process is quite universal. And then the secondary form it takes when we communicate it.

• 34:17Speaker 02

  On this show, I think when he says universal, he means earthwide.

• 34:20Speaker 03

  I mean, yeah, I do. I mean, global.

• 34:22Gary O'Reilly?

  Yeah.

• 34:23Speaker 03

  I mean, global. I mean, global. Yeah, terrestrial, for all life.

• 34:28Speaker 02

  Yes, good, good. Human, the earth life.

• 34:31Speaker 03

  Earth life. And this secondary cultural layer that we impose, whether we call it, you know, Jesus or Vishnu or what have you, or the Buddha, that that's where the variability probably exists. But I would guess that the underlying dream state or unconscious solution is probably quite universal.

• 34:46Speaker 02

  Interesting. So let me get back to that dream state. Do we know the title of the short story that became Blade Runner?

• 34:53Speaker 03

  Do androids dream of electric sheep?

• 34:55Speaker 02

  So that brings us.

• 34:56Speaker 00

  Do androids dream of electric sheep?

• 34:58Speaker 02

  There you go.

• 34:59Gary O'Reilly?

  Yeah.

• 34:59Speaker 00

  Now, somebody was high.

• 35:01Speaker 03

  Well, he was, he was. Philip K. Dick was always high.

• 35:05Speaker 02

  Oh, awesome. Philip K. Dick.

• 35:07Speaker 00

  I love it.

• 35:07Speaker 02

  So that gets us to the question, will AI never be us because AI doesn't dream?

• 35:17Speaker 02

  And will solutions available from our dream state never be available to AI?

• 35:26Speaker 03

  So there's a few things to say about this, right?

• 35:28Speaker 02

  I hope more than a few.

• 35:29Speaker 03

  There are many things to say about this. So we started with the evolution story, and I made the claim, right, that sentience or consciousness or sensation was primordial, that all beings have it. That is definitely not true of LLM. Okay, so point one.

• 35:47Speaker 02

  The large language model.

• 35:48Speaker 03

  The large language model, right? It has a completely different path towards its intelligence, if you like. Secondly, it immediately starts with language. It doesn't have an unconscious, there is no unconscious.

• 35:58Speaker 00

  Right.

• 35:59Speaker 03

  So it's just got a linguistic foundation right at the beginning. Thirdly, it's a collective linguistic, and it's not an individual language learner, it's every book ever written.

• 36:10Speaker 00

  That's right.

• 36:11Speaker 03

  So there are some really fundamental.

• 36:12Speaker 02

  Including fourteen of my books that are now in a class action suit. Just thought I would tell you that.

• 36:17Speaker 03

  Absolutely right. So these, these, these are important differences. So start with the last thing that we got, right, which is language. And ignore what every other organism has, which is this, if you like, homeostatic sentient form of existence, pre-language.

• 36:36Speaker 00

  Right.

• 36:37Speaker 02

  Wait, wait, so where are you taking us with that?

• 36:39Speaker 03

  So, okay, so.

• 36:40Speaker 02

  You can just say that and not take me somewhere.

• 36:42Speaker 03

  Okay, so, okay, so, so this now, this lead opens this whole debate about what we mean by an LLM, a large language model, and AI being intelligent without being alive.

• 36:54Speaker 02

  And I'm asking you, do they dream?

• 36:56Speaker 03

  They definitely don't dream.

• 36:57Speaker 02

  Definitely don't dream. So does that limit where it can take us relative to where we can go ourselves?

• 37:05Speaker 03

  Okay. So we were talking about this before. I mean, you should repeat that earlier offline, that point you made, because I think it's an interesting one about what you can do with language.

• 37:16Speaker 00

  So we were offline, when David and I were talking, I said that with language, I can explain anything to you, no matter what it is, that if I can find the right words, and you know those words, right, it doesn't make a difference what it is that I'm trying to convey to you, I can do it. And that is extremely powerful. And then you said what?

• 37:43Speaker 03

  Yeah, so we were talking about the recent thing in the last week that everyone's talking about, which is LLM solving a mathematics problem, the singularities of the Navier-Stokes equations, this kind of stuff, right? This dynamics of fluids. And there was this long last, you know, outstanding question with a prize behind it about some properties of these equations, which is, do they blow up and so on? Do they have singularities in them? And so out comes an LLM recently, and at considerable expense, I mean, tens of millions of dollars, thousands of agents, it finds a solution. But it's doing it by being fed text.

• 38:24Speaker 00

  Right.

• 38:25Speaker 03

  Not geometry, it doesn't have a visual apparatus, right? So what does that mean that we can solve math problems by only being trained on language? And that's what we were talking about.

• 38:35Speaker 00

  Right.

• 38:36Speaker 03

  And I think what we've discovered, the interesting thing about purely linguistic intelligence, if you like, an LLM, is that you can solve almost any problem with it, but you solve it very inefficiently.

• 38:47Speaker 00

  Right. And then you gave the example of eating soup, which I thought was brilliant.

• 38:54Speaker 03

  Well, yeah, you could eat a bowl of soup with a fork.

• 38:57Speaker 00

  Slowly.

• 38:57Speaker 03

  You could, slowly.

• 38:59Speaker 00

  There you go.

• 38:59Speaker 03

  Enough time, enough energy, you can solve math problem with language, enough time, enough energy. And human intelligence is based on a tough world where you don't have enough time and you don't have enough energy. That's why we invented all these other languages to solve problems. It's a very important distinction.

• 39:17Speaker 02

  Plus, we have, you know, a billion years testing ground to make it happen efficiently.

• 39:24Speaker 03

  Exactly.

• 39:25Speaker 02

  Because if we only got smart doing LLMs, we would have burnt our energy load, you know, hundreds of thousands of years ago, and we would have never gotten to where we are.

• 39:36Speaker 00

  Yeah, we have a lot of knowledge sitting in the dark.

• 39:40Speaker 03

  But that's an important point about why I don't like calling them intelligence, because they're very capable, they're very, they can solve problems. But for human beings, you don't call someone intelligent just because they're capable. If I said, you know, Gary, could you pass me that glass of water? And you got him, ran around the room 20 times, and then gave it to me.

• 40:00Speaker 02

  No.

• 40:01Speaker 03

  We'd all say, that's kind of weird. What's Gary doing?

• 40:05Speaker 00

  What's Gary's problem?

• 40:06Speaker 03

  What's his problem? He achieved the, solved the problem, right? Gave me the water.

• 40:11Speaker 02

  Yeah, at the end, you got the water.

• 40:12Speaker 03

  At the end, I got the water. And I think a lot of what they're doing now is getting up and running around the room 20 times, and people are calling it intelligence because it solves the benchmark. Just call it Gary. No, just call it capability.

• 40:24Speaker 02

  So that could implode, because it could exceed our capacity to deliver the energy it needs to solve that problem.

• 40:31Speaker 03

  Exactly.

• 40:32Speaker 02

  So there could be an upper limit, a functional upper limit. Short of tapping the sun.

• 40:37Speaker 03

  Right.

• 40:38Speaker 02

  Directly.

• 40:39Speaker 03

  Right now.

• 40:39Speaker 02

  Like we tap a keg.

• 40:40Speaker 03

  But see, if we were truly intelligent, if we were truly intelligent, then we would use this as a motivation to create that energy.

• 40:50Speaker 02

  No, I got a better answer.

• 40:52Speaker 03

  Go ahead.

• 40:52Speaker 02

  If the AI were truly intelligent.

• 40:54Speaker 03

  Oh, God.

• 40:54Speaker 02

  It would figure out.

• 40:56Speaker 03

  Less energy.

• 40:57Speaker 02

  Less energy.

• 40:58Speaker 03

  Oh, there it is. No, you're right. That's it.

• 41:00Speaker 02

  I got a mic drop.

• 41:01Speaker 03

  No, you got a mic.

• 41:02Speaker 02

  What happened to my plastic mic?

• 41:05Gary O'Reilly?

  You dropped it.

• 41:06Speaker 02

  Okay.

• 41:07Speaker 03

  Wait, and? Mic drop.

• 41:09Speaker 02

  Mic drop.

• 41:09Speaker 03

  There you go. By the way, do you just carry around a microphone?

• 41:15Speaker 02

  No, I just have stuff in arms reach in my office.

• 41:18Gary O'Reilly?

  So you discuss things called complementary and competitive cognitive artifacts.

• 41:23Speaker 02

  It's way too many syllables.

• 41:25Speaker 03

  No, I didn't.

• 41:26Speaker 02

  Okay.

• 41:26Speaker 03

  My fault.

• 41:27Gary O'Reilly?

  And where does AI sit in which column?

• 41:31Speaker 02

  Cognitive.

• 41:33Gary O'Reilly?

  And no, complementary and competitive cognitive.

• 41:36Speaker 02

  As two separate.

• 41:37Gary O'Reilly?

  Yes.

• 41:38Speaker 02

  Got it. So take us there.

• 41:39Speaker 03

  Okay, so this goes back, in fact, to a French philosophy philosopher, Henry Bergson, who debated relativity with Einstein, as you know, right in the 20s. But in 1907, he writes this book, Creative Evolution, right? And he says, we shouldn't be called Homo sapiens, wise humans, right? Which came from Linnaeus. We should be called Homo faber, humans that make things, humans that make tools. And Bergson's whole theory was what makes humans different, to the extent that we are, is we not only make tools, but we make tools to make tools.

• 42:13Gary O'Reilly?

  True.

• 42:14Speaker 03

  Right? So that has been somewhat neglected in this discussion of intelligence, right? Because it's, in some sense, you can't really think of human intelligence without thinking about painting, symbols, writing, various kinds of calculating tools and artifacts. So that's the background, and that has its own development.

• 42:35Speaker 02

  A quick question. Yes, we have opposable thumbs. Yes, we make tools. Why should that alone distinguish anything when we don't know if whales had opposable thumbs, would they be making tools themselves? They just can't.

• 42:50Speaker 03

  But we do know that if we did not have opposable thumbs, we would make opposable thumbs so that we could make other stuff.

• 43:03Speaker 02

  I'm just saying.

• 43:03Gary O'Reilly?

  Are we in the Weezy?

• 43:05Speaker 02

  Just because they don't, doesn't mean they wouldn't if they could.

• 43:09Speaker 03

  That's true. I mean.

• 43:10Speaker 02

  That's a good sense.

• 43:10Speaker 03

  And there is a, there's a lot of debate about this. I mean, there was a lot of early discussion, I remember, on language, which is whether or not the reason why other primates don't speak is there's some anatomical constraint. And if that wasn't in place, perhaps they would have evolved language.

• 43:26Gary O'Reilly?

  Or the position of the larynx.

• 43:27Speaker 03

  That sort of debate.

• 43:28Gary O'Reilly?

  Evolved to be in a lower position.

• 43:30Speaker 03

  Exactly. So breathing.

• 43:31Speaker 02

  Yeah, it was, it was a genetic mutation, basically, that enabled this.

• 43:35Gary O'Reilly?

  That's nature.

• 43:36Speaker 03

  So that's where it begins, right? This idea that humans make tools. So I got very interested in that idea. And there's archaeologist, French archaeologist, Andre Goran. And he said, what are tools about? And he said, there are, it's an additional memory system, is what he said. He said, we've got genetic memory stored in our genomes. We've got learnt memories stored in our brains. And you've got cultural memories stored in language, art, stuff.

• 44:02Gary O'Reilly?

  Hmm.

• 44:03Speaker 03

  And so he actually argued, he went further than Bergson. He said, to be anthropogenic is to be technogenic. He's, he views them as indissoluble. Look at that. Right? That somehow technology and humanity are kind of wrapped together in some way. So that's where I jump off with this kind of mathematical theory of tools. And the basic idea is that, right? There are tools which I call complementary, things that enhance our ability to reason, like a map. We talked about it last time. You had, you might even have an abacus in here.

• 44:40Speaker 02

  Yes, there is an abacus.

• 44:41Gary O'Reilly?

  It's around somewhere.

• 44:42Speaker 03

  Okay, but that's a good example of what I call complementary. And it helps you reason. If you don't have it, as we know from studies of abacus schools, these kids can then do that calculation. They don't even need it.

• 44:55Gary O'Reilly?

  That's right.

• 44:55Speaker 03

  Don't even need it. It's in their head. So this is the key point. The, the abacus is a scaffolding for thinking better.

• 45:02Gary O'Reilly?

  Look at that.

• 45:03Speaker 03

  Right? So now, in the competitive case, these are tools that don't scaffold anything. They replace it. So take a GPS versus a map.

• 45:12Gary O'Reilly?

  Interesting.

• 45:13Speaker 03

  So the idea is so you can develop this kind of theory.

• 45:15Gary O'Reilly?

  Oh, no.

• 45:16Speaker 03

  And it says that if you outsource completely, you lose the ability to reason.

• 45:20Gary O'Reilly?

  We're becoming dumber and dumber.

• 45:22Speaker 02

  The smarter we become.

• 45:24Speaker 03

  The smarter we become, the more stupid we become.

• 45:26Gary O'Reilly?

  Yeah, because we're replacing the accrued knowledge.

• 45:29Speaker 03

  Oh, no.

• 45:31Speaker 02

  Wait, wait, wait, wait, wait. Pause, pause, we'll say.

• 45:33Speaker 03

  What?

• 45:33Speaker 02

  I have to restate what you said, just so I believe I understand it. In the case of the abacus,

• 45:40Speaker 02

  the abacus is a means to calculate. And this becomes autonomic within me. And then I do it, but the act of doing it is implicitly in reference to the abacus that trained me. Now, also, we have at least two or three generations of people still on Earth who know how to read maps. And they can translate what's on a map to where they are in the world. There are no maps anymore. There's just GPS barking at you. Turn left, turn right. So up ahead on the right. And you're saying we are handicapping a generation of people

• 46:22Speaker 02

  who do not know how to read maps, even how to orient a map, or what, what, what the legend is on a map. They have no idea. But why is that a bad thing?

• 46:31Speaker 03

  Okay, so it's interesting. So here's, so we've got.

• 46:34Speaker 02

  Did I get it right?

• 46:34Speaker 03

  Exactly.

• 46:35Speaker 02

  Okay, okay.

• 46:36Speaker 03

  So that, so that we need more vocabulary here than just intelligence, right? So we have, let's just introduce some terms. One is capability. So using a GPS or using a map, both make you capable. They both help you to navigate. No problem. So benchmarks on AI are testing capability. They can do it. They can calculate. They can solve problems. They can navigate. Good, right? So that's capability. Then there's competence. Different. Competence is, okay, give me the GPS. Now navigate. And that reveals dependence. And so what we're doing is we're moving into a world to when you say, does it matter? As long as you always have it.

• 47:16Speaker 02

  Yes.

• 47:17Speaker 03

  And you're not interested in competence, but only capability.

• 47:22Speaker 02

  Oh.

• 47:23Gary O'Reilly?

  Right.

• 47:23Speaker 03

  And the problem is, that gives you wise to a world of total dependence on the provider of your capability.

• 47:29Gary O'Reilly?

  Correct.

• 47:29Speaker 03

  And that provider, who I, who will not be named in this conversation, because we know who they are, don't necessarily have our best interests in mind.

• 47:37Gary O'Reilly?

  Absolutely.

• 47:38Speaker 02

  So I, so I went through this intellectually some years ago. I'm old enough to have, I think I'm older than you, to be formally trained, formally trained on a slide rule. Okay? It must have been the last year that happened, because between that year and the next, the four-function calculator dropped in price from $200 to $40. So it became the price of like a textbook. And so thenceforth, they stopped teaching you the slide rule. And I got a slide rule here, too.

• 48:11Speaker 02

  This was my high school slide rule. All right? Now, there's a lot of sort of mathematical insights into the design of a slide rule. And you learn about logarithms implicitly. And then I, this is made of plastic, but earlier I have a vintage model. You know, this, they used to make these out of elephant tusk. And I think this is just wood. But anyhow, in my high school, you had a holster with your.

• 48:43Speaker 02

  And the bigger the slide rule, the more precise you can be in your measurements. So we'd be walking down the.

• 48:50Speaker 03

  This math class ain't big enough for the two of us.

• 48:54Speaker 02

  But here's my point. My intellectual challenge was, if I never see a slide rule again, because a calculator renders it obsolete, what am I missing? And I said to myself, if I'm always near a calculator.

• 49:12Speaker 03

  You're missing nothing.

• 49:13Speaker 02

  I'm missing nothing. Let me move on to some other thing that requires my intellect to solve.

• 49:19Speaker 03

  So.

• 49:20Speaker 02

  So that was my, that was my coming of age to embrace a calculator and release the slide rule to the dustmen of history.

• 49:28Speaker 03

  So, yeah, in fact, when Hewlett Packard made their first affordable calculator, all of the slide rule companies went out of business. The same year, nineteen seventy-three.

• 49:40Speaker 02

  Yeah, right, that's when that's it.

• 49:41Speaker 03

  That time.

• 49:42Speaker 02

  That's it.

• 49:43Speaker 03

  So there is something missing, though. So, right, so one possibility, it's a bit like vitamins, like vitamin C. You might say, you know what? I can't synthesize it myself, but it'll always be in the environment until you sail and get scurvy. So one possibility is we do find ourselves in a case or a situation where we don't have access to the technology and then we're useless because we've outsourced it all.

• 50:06Speaker 02

  Apocalyptic Earth.

• 50:07Speaker 03

  Yeah.

• 50:08Speaker 00

  Well, just dumbass Earth.

• 50:10Speaker 03

  Well, you know, a power cut.

• 50:12Speaker 00

  A power cut, yeah.

• 50:14Speaker 03

  Doesn't take much.

• 50:14Speaker 02

  Okay.

• 50:15Speaker 03

  But the other one that's interesting is that the intuition that you built, which relates exponentials to multiplication and what the slide rule does, that kind of intuition that came out of using that is useful beyond that solving that particular problem in a way.

• 50:32Speaker 02

  Completely.

• 50:33Speaker 03

  And that's where it becomes deep for us because we don't know whether or not some of these things that we're learning by using these analog devices doesn't somehow diffuse into everything else we think about and develop a stronger intuition. And that's beyond losing the capability because there's a power cut.

• 50:50Gary O'Reilly?

  So, okay, let's slow this forward.

• 50:52Speaker 03

  Yeah.

• 50:53Gary O'Reilly?

  How many generations, if just one generation, will it take if we outsource all of these sort of capabilities to an AI for us to be as dumb as a rock?

• 51:04Speaker 03

  Yeah, so in the.

• 51:05Speaker 02

  Yeah, that's a, that's a depressing question, dude.

• 51:07Speaker 03

  So I can tell you that in the mathematics, there's a key distinction to be made is, and that is that if you learn first, let's just do it hypothetically, the slide rule and then move to the calculator.

• 51:22Speaker 03

  Your memory kind of helps you out.

• 51:25Gary O'Reilly?

  Yes.

• 51:25Speaker 03

  So if you take it away, you still have in some sense, some intuition for how that works. But if you began with the calculator, that is, you never learned the logic behind the calculation, then you're lost.

• 51:38Gary O'Reilly?

  Yeah.

• 51:38Speaker 03

  And I think what we're entering now is a world where all of these kinds of things are being outsourced.

• 51:43Speaker 00

  Being outsourced.

• 51:44Speaker 03

  And you'll never acquire them.

• 51:45Speaker 02

  Here's what I found.

• 51:45Speaker 00

  So I'm sorry, I was just gonna say, it sounds to me like what you're saying is these analog devices are mapping onto our brain. That's what they are doing.

• 51:56Speaker 03

  Exactly.

• 51:57Speaker 00

  Whereas the other devices are replacing the brain and creating dependency.

• 52:03Speaker 02

  Because just not to get inside baseball here, but a slide rule implicitly trains you to think about what's called significant figures.

• 52:13Speaker 00

  I think about that all the time, Neil. I'm just saying.

• 52:18Speaker 02

  Significant.

• 52:21Speaker 02

  So, so you do this and there's only certain number of figures you carry with you into the next calculation. Is it three point six, seven, eight, nine, four? Or three point six, eight? Okay, because you can't carry that many decimals forward in this, by this device. But it tells you that often those other decimals are not relevant to the major point of the calculation you're conducting. I saw someone do a calculation on the slide rule and they somehow believed that all the digits that were returned on the calculator were important and wrote down every single digit without having a clue how irrelevant they were. And so that was just ignorance.

• 53:04Speaker 03

  It is. This idea, I think, I don't know if it was Bertrand Russell who said, so approximation is the supreme sophistication. Learning when something matters and when it doesn't.

• 53:16Speaker 02

  Yes.

• 53:16Speaker 03

  When these decimal places, when you can, when you can think about powers of ten or approximation is a part of our intelligence.

• 53:22Speaker 02

  Well, in graduate school, we have a recurring, it's like a seminar. It's called back of the envelope universe, right? You just ask a question, blunt questions like, how many pianos are in the world? Well, how would you estimate that? You say, well, how many have you seen, where have you been to? How many people are they making more? You don't have to know the exact number, but get an approximate number and know how to make approximations. If you don't know how to do that, you're, you're a victim of false precision in the world, distracted by what you think is real, when in fact it's a distraction on your path to the truth.

• 54:00Speaker 03

  Oh, and this gets to a concept which I think is for humans as important as intelligence. And that is the notion of intelligibility. Not intelligence, but intelligibility, which is about understanding things. And that's the other point. If you're in the world of capability, prediction and utility are all that matters. In the human world, science is about making the universe and the world intelligible. We want to understand it. Different game.

• 54:28Gary O'Reilly?

  On that point, if we don't outsource the capabilities and we retain a a large chunk of what intelligence we possibly do have.

• 54:37Speaker 02

  Wishful thinking.

• 54:38Gary O'Reilly?

  I know.

• 54:40Gary O'Reilly?

  Are we going to at some point solve the problems, the big problems? The universe, why, how, what? Are we gonna get there or are we just gonna be clutching at thin air?

• 54:50Speaker 03

  If we fully outsource, you mean?

• 54:53Gary O'Reilly?

  Yeah, but if we retain, if we haven't outsourced completely and have retained some intelligence, do you think we're capable enough? Because you have said human intelligence potentially unlimited.

• 55:03Speaker 02

  Yeah, what do you mean by that?

• 55:05Speaker 03

  Okay.

• 55:05Speaker 02

  Yeah.

• 55:05Speaker 03

  So this gets back to Bergson and the others on tool use. So humans are always hedging limits.

• 55:12Gary O'Reilly?

  Yeah.

• 55:13Speaker 03

  I mean, Neil just talked about the slide rule. You can't do that with your fingers.

• 55:17Gary O'Reilly?

  No.

• 55:17Speaker 03

  Okay. And this gets to why we use tools. There are things we can't do. We build them. We understand how they work and they solve problems that we couldn't in some raw state solve. So in that sense, an LLM and AI is just a tool like any other. And it's important to understand what kind of tool it is. Because this tool is a tool that needs tons of data. Most of our tools don't.

• 55:41Gary O'Reilly?

  No.

• 55:42Speaker 02

  Good point. I don't have to feed the slide rule to get my answer correct.

• 55:45Speaker 03

  A little bit. And they're very good, and I, we can unpack this, doing high dimensional problems where lots of variables matter. So think about protein folding.

• 55:55Gary O'Reilly?

  Yes.

• 55:55Speaker 03

  It turns out protein folding, lots of variables matter.

• 55:58Gary O'Reilly?

  Mm-hmm, right.

• 55:59Speaker 03

  And so doing the molecular dynamics, the full computation.

• 56:03Gary O'Reilly?

  Predictive modeling, yeah.

• 56:04Speaker 03

  It's just too computationally expensive.

• 56:06Gary O'Reilly?

  Yeah.

• 56:06Speaker 03

  And so LLMs come along, these models, actually, they're not LLMs, they're different kinds of networks, deep neural networks. And they're good at doing high dimensional problems. And so they help us. So they do protein folding. And a lot of problems are gonna turn out to require high dimensional representation. Lots of variables matter. And they're gonna help us solve the problem.

• 56:27Speaker 02

  Just to be clear, so everyone's on the same page. When you solve a problem, the number of variables can be thought of as a dimension, how many dimensions need to be accessed to get the solution. And multivariate problems require vastly more computing power.

• 56:45Speaker 00

  However, is that, is that really intelligence or just brute force?

• 56:50Speaker 02

  Yeah, but what, so tell me the difference between the AI solution to the protein folding. Previously, we just, we had really powerful computers. Give them the protein folding problem. What did the AI deep learning neural nets? What was that able to accomplish that just sheer computing power wasn't?

• 57:12Speaker 03

  Yeah.

• 57:12Speaker 02

  Prior.

• 57:13Speaker 03

  So there's two factors here. One is, and I'm gonna go a bit nerdy on you guys.

• 57:20Speaker 02

  Bring it on.

• 57:20Speaker 03

  Okay.

• 57:21Speaker 00

  We like it.

• 57:21Speaker 03

  Which is.

• 57:22Speaker 02

  This is a nerd safe space.

• 57:23Speaker 03

  This is, okay, I'm gonna say it, is that when you do these full physics simulations of a protein to get it to fold. So you're looking at all the bonds formed between all the atoms. This has something called a scaling relation, which is that the amount of computation increases nonlinearly. It goes up quadratically.

• 57:41Gary O'Reilly?

  Yeah.

• 57:41Speaker 03

  So double the size more than doubles the amount of computational effort. Okay. Now, it turns out because of some architectural feature of these AI models, when you increase the scale of a problem, it scales linearly.

• 57:56Speaker 00

  Ooh.

• 57:57Speaker 03

  It has a much more efficient scaling property, this architecture. So things that would not have been computationally feasible with the full physics become feasible with the deep neural net. That's a very important point.

• 58:10Speaker 02

  Well, that answers it right there.

• 58:11Speaker 03

  And that's basically the answer.

• 58:13Gary O'Reilly?

  That's Alpha Fold three.

• 58:15Speaker 03

  These are Alpha Fold.

• 58:16Gary O'Reilly?

  The Sibis company, we had a couple of guys from there.

• 58:19Speaker 00

  Yeah, they were on the show.

• 58:20Speaker 03

  Now, the other factor, though, is not just the scaling, which is slightly more subtle, is that you can feed all sorts of different data into these models. So in the physics simulation, it's just the physics. But in the folding, it's not just the physics and chemistry, it's the evolutionary history. And so you can put all sorts of different data and drop it into these black boxes. And that's the second source of that advantage over physics.

• 58:47Speaker 00

  Wow.

• 59:00Speaker 00

  Every discovery starts with a question. What if? Why? How? When it comes to education, Southern New Hampshire University has answers. Whether you want to build new skills or pursue a lifelong passion, SNHU has over 200 online programs to expand your horizons. And a flexible format lets you learn on your schedule without putting life on hold. So if you're asking, what's next for me? It's time to find out. Explore what's possible at SNHU.edu/StarTalk. That's SNHU.edu/StarTalk. Whoever says saving money is hard doesn't know that new customers who bundle and save

• 59:41Speaker 00

  with Progressive save hundreds because switching your home and auto insurance is better than cutting down on things you love, like those iced lattes that help you get ready to face the day. So switch your home and auto to Progressive and see if you could save and keep sipping away. Get your quote today at progressive.com. Progressive Casualty Insurance Company affiliates and other insurers, not available in all states.

• 1:00:06Speaker 04

  With the Discover Cash Back Card, it's payback time when you earn cash back on everyday purchases. Activate and earn 5% cash back at different categories each quarter on up to $1,500 in purchases. That's 5% cash back at different places each quarter, like grocery stores, on gas, and at restaurants. It pays to discover. Terms apply. See discover.com/five for details.

• 1:00:34Speaker 00

  I'm Brian Futterman, and I support StarTalk on Patreon. This is StarTalk with Neil deGrasse Tyson.

• 1:00:47Speaker 02

  Occasionally, I lose sleep on this very thought. Are humans intelligent enough, however you want to define it, to completely understand the universe with no stone left unturned? And if we're not in the individual, in the collective, might we be? I don't have to invent calculus, because Isaac Newton did. I stand on his shoulders, and we just keep ascending this ladder incrementally. Can we incrementally become super intelligent? Or is this the work of an AI that can train on itself and then just bullet past all of us to contemplate the cosmos?

• 1:01:30Speaker 03

  So I think we already know that we have limits. I think the protein folding case gives us that answer, that there was no theory.

• 1:01:39Speaker 02

  Well, that was a scale problem.

• 1:01:41Speaker 03

  It was a scale problem.

• 1:01:42Speaker 02

  Not a how deep is your thinking.

• 1:01:44Speaker 03

  But I would argue that all of our hardest problems are scale problems. And then let's give an example, because it relates to this concept of emergence, right? So we wouldn't have been able to do simulations of large numbers of particles unless we had fluid dynamics, right? So we found a way of making it more comprehensible. And we don't have to track every particle.

• 1:02:07Speaker 02

  To say that more precisely, the laws of fluid dynamics allow us to not have to track every single molecule of the fluid. We treat it in an ensemble, like the gas laws. The gas laws are blunt equations that describe pressure, temperature, volume of gas. We're not tracking what the particles are doing. In principle, if you did, you should get those formulas out. But we don't have to because we have the macroscopic description. So continue.

• 1:02:34Speaker 03

  Exactly. And most of human understanding, once we hit this barrier given by the lower level, in that case, particles, but it could be this case. It could be, I'm feeling unhappy, or I love you. This is a macroscopic description.

• 1:02:50Speaker 02

  Very macroscopic.

• 1:02:51Speaker 03

  Of a very large chemical system inside my body, right? And that's what we do every time. And we consider that understanding, a form of understanding, right? Now, it turns out there are problems like protein folding that we couldn't do that. There was no macroscopic description. We had to worry about all the molecular interactions.

• 1:03:10Speaker 02

  Oh.

• 1:03:11Speaker 03

  You see? And so this particular tool helped us to solve the problem where there wasn't an emergent theory that would solve the problem.

• 1:03:18Speaker 02

  Okay, so now this is AI being our agent in that quest. It's not AI running amok, becoming a superintelligence on its own.

• 1:03:29Speaker 03

  No, that's right. It's AI as a tool.

• 1:03:31Speaker 02

  Once again.

• 1:03:32Speaker 03

  To deal with irreducible high-dimensional systems.

• 1:03:34Speaker 02

  I'd like to think our future relationship with AI is precisely that in every way that best serves us, not one where it becomes our overlord. In recent decades, there's been a lot of discussion, especially in modern astrophysics, about from information theory. Is there a limit to how much information is in the universe? Can we somehow create more than that? Does that information go away? Because the entropy seems to be a clock that we cannot stop. Would the universe approach this ever greater entropy in the future than it does in the present or in the past? Do any of your calculations touch on this?

• 1:04:13Speaker 03

  Yes. So two, two different ways. One is about the fundamental limits to computation, what gets called the thermodynamics of computation, and that takes us to quantum and all that. So, right, there is a fundamental energetic limit.

• 1:04:27Speaker 02

  So that would be, the quantum computer would be, in principle, the most efficient computer we can conceive.

• 1:04:33Speaker 03

  Exactly.

• 1:04:34Speaker 02

  Whether or not we're there yet.

• 1:04:35Speaker 03

  Exactly.

• 1:04:35Speaker 02

  Okay.

• 1:04:36Speaker 03

  And then there's the other entropic limit, which is the cost of calculating with a classical von Neumann computer, the kind of computers that we all have, iPhones and so on. So we have these massive data centers. They're consuming megawatts of energy.

• 1:04:54Speaker 02

  But gigawatt.

• 1:04:55Speaker 03

  Big gigawatt. They're boiling the planet, all to make your emails better.

• 1:05:03Speaker 00

  But it's okay, because they're going to solve climate crisis.

• 1:05:07Speaker 03

  Eventually. This is fascinating. That's a whole thing in itself, right? Once they've melted the planet, for self-survival purposes.

• 1:05:13Speaker 00

  Right, they'll figure it out.

• 1:05:14Speaker 03

  They're going to have to work it out. And it's not just being a co-pilot to your rating. So that's the other entropic problem. So again, I mean, a long story here. Maybe I wanted, can I just give a little narrative? I think it's useful, historical one.

• 1:05:29Speaker 02

  You're the guest.

• 1:05:29Speaker 03

  Okay. So try to make sense of what is going on in the current moment in terms of these kind of energy-guzzling AIs. There's a useful comparison to the Industrial Revolution and to ecology. So most of the history of life is predator and prey. That is how energy got transferred. You ate something. Okay? In parallel, in the Carboniferous Period and the Mesozoic Period, back there, hundreds of millions of years ago, fossil deposits are being left in the earth, becoming coal seams, becoming petroleum and natural gas. And that energy is not accessible to us, just there.

• 1:06:14Speaker 03

  Then the Industrial Revolution happens. And we get steam engines, and they get more and more efficient. And what steam engines do is they liberate all of this energy that was buried in the earth.

• 1:06:26Speaker 02

  But just to be precise, only when they discovered coal could do that.

• 1:06:30Speaker 03

  Exactly.

• 1:06:31Speaker 00

  Beautiful, clean coal.

• 1:06:34Speaker 02

  Otherwise, you just.

• 1:06:34Speaker 00

  You can't beat it.

• 1:06:35Speaker 02

  Otherwise, you just burn wood.

• 1:06:37Speaker 03

  That's right. Vastly less efficient.

• 1:06:38Speaker 02

  Yes, the energy packing capacity of wood versus coal is no contest.

• 1:06:43Speaker 03

  No contest. Yeah, you could, I think the calculation is, you could burn all of the standing forests of the world and provide enough energy for the current population for six months.

• 1:06:54Speaker 02

  Wow. That's pretty serious.

• 1:06:55Speaker 03

  That's it.

• 1:06:56Speaker 02

  And by the way, when, before we knew about thermonuclear fusion, we said, well.

• 1:07:01Speaker 00

  It's a hoax.

• 1:07:02Speaker 02

  No. The sun, you know, the sun is like really efficient, and we knew there's a lot of carbon in the universe. It must be a lump of coal.

• 1:07:10Speaker 00

  It must be a big lump of coal.

• 1:07:12Speaker 02

  Because it was efficient, and it could go a long time. Those are our first calculations where maybe the sun is made of coal.

• 1:07:17Speaker 00

  Man, somebody should tell our president that right now, and maybe we'd get to renewable fuels.

• 1:07:23Speaker 02

  Continue, I interrupted.

• 1:07:23Speaker 03

  Yeah, so, right, so we discovered super-energy-dense, power-dense materials. Power means you could liberate it fast, like in a nuclear reaction. Okay. So the world changed. It went from being networks of predators and prey. Highly constrained by interaction, to all of a sudden, this massive, high-energy-dense archive that we could access. Okay. That's the energy world. So let's talk about the information world. Nearly all the information we have, we get from talking like this. It's a bit like ecology, predators and prey, right? It's like, oh, I'm going to tell you a fact. You could read a book. Problem is with reading books, you read like, I don't know, maybe 250 words a minute over your lifetime. If you're lucky, you read a few thousand books. That's how it works.

• 1:08:08Speaker 03

  So what happens? So.

• 1:08:10Speaker 02

  So a thousand books, let's use the number thousand books.

• 1:08:12Speaker 03

  So just like in the Cambrian, right, or the Carboniferous, rather, and in the Mesozoic, where fossils were being deposited in the earth, we've been depositing checks in libraries, starting in ancient Sumer all the way to the present. So we have now order hundreds of millions of books deposited.

• 1:08:31Speaker 02

  And it's mostly in the present.

• 1:08:32Speaker 03

  Mostly in the present.

• 1:08:34Speaker 02

  The rate of book publication has grown exponentially over the decades. Yeah, okay.

• 1:08:39Speaker 00

  However, the knowledge also covers all of our past, because we've discovered our past in the present and deposited that information in the depositories you're talking about.

• 1:08:50Speaker 03

  Exactly, but here's the thing. So this is why the LLM is interesting. The LLM is like the steam engine.

• 1:08:55Speaker 00

  I see where you're going.

• 1:08:56Speaker 03

  Because what happened is.

• 1:08:57Speaker 02

  It's mining.

• 1:08:57Speaker 03

  It's mining, exactly.

• 1:08:59Speaker 00

  It's liberating that energy, which is our literature, which is our language, which is our thoughts and cultures and everything that we are, all in one depository, that it can now go in the same way we went in and dug for coal. It can go in and get all of that very, very quickly and turn it into power.

• 1:09:23Speaker 03

  Exactly.

• 1:09:23Speaker 00

  Wow.

• 1:09:24Speaker 03

  But this time.

• 1:09:25Speaker 00

  Damn.

• 1:09:26Speaker 03

  Conceptual power.

• 1:09:27Speaker 00

  Right.

• 1:09:27Speaker 03

  Mechanical power.

• 1:09:28Speaker 00

  Exactly.

• 1:09:29Speaker 03

  So there's this strict analogy between, if you like, the fossil deposit, and as you say, the cultural deposit. The steam engine liberated the energy deposit. The transformer architecture liberated the conceptual deposit.

• 1:09:41Speaker 00

  Look at that.

• 1:09:43Speaker 03

  Now, both of them produce what economists call, euphemistically, externalities, right? In other words, they pollute. So that's the problem with the Industrial Revolution, right? It produces smogs, right? It generated a huge amount of entropy in the environment, just as these data centers and transformers are producing a huge amount of entropy in the environment.

• 1:10:04Speaker 00

  Right.

• 1:10:04Speaker 03

  So there's this very interesting parallel.

• 1:10:06Speaker 00

  And wait a minute, and the culture.

• 1:10:10Speaker 00

  Look at that. They're creating entropy not only in the environment, but they're creating entropy in the culture itself.

• 1:10:16Speaker 03

  Yes.

• 1:10:17Speaker 00

  Wow.

• 1:10:18Speaker 03

  Yes. So that's a really interesting point. In the conceptual space.

• 1:10:21Speaker 00

  Right.

• 1:10:22Speaker 03

  The disorder is not just thermal, it's knowledge based.

• 1:10:26Speaker 00

  Yes.

• 1:10:27Speaker 03

  It's basically crap. No, it's misinformation.

• 1:10:30Speaker 00

  No, we call it AI slop.

• 1:10:32Speaker 03

  Yes, look. So this is, there's a very weird thing going on here.

• 1:10:36Speaker 02

  Just to be clear, in the industrial revolution, not all coal or oil deposits were equally as refined, equally. I mean, different sources of oil in the world are not as good.

• 1:10:48Gary O'Reilly?

  It's not a uniform.

• 1:10:49Speaker 02

  It's not a uniform deposit.

• 1:10:51Speaker 00

  Deposit, yeah. No, correct. No, you're absolutely, you have light sweet crude, you have tar sands, you have, yeah.

• 1:10:56Speaker 03

  Now, look what happened. So we created, so out of, you know, the Pennsylvania, now out of these deposits, come all the big corporations of another era of BP.

• 1:11:07Speaker 00

  Right.

• 1:11:08Speaker 03

  Shell.

• 1:11:08Speaker 00

  Right.

• 1:11:09Speaker 02

  And the steel companies that use the coal.

• 1:11:11Speaker 03

  Exactly. So now what's happening?

• 1:11:13Speaker 02

  US Steel.

• 1:11:13Speaker 03

  Well, we have our analogs. We have Anthropic, we have OpenAI.

• 1:11:18Speaker 00

  Yeah.

• 1:11:19Speaker 03

  These are the mining companies of our time.

• 1:11:22Speaker 00

  Look at that.

• 1:11:23Speaker 03

  Right? Generating the kind of entropy that you were discussing.

• 1:11:26Speaker 00

  Yeah.

• 1:11:26Speaker 03

  Now, what's weird here that is worth discussing is that coal is depleted, natural gas is depleted, but in principle, knowledge is not.

• 1:11:38Speaker 03

  Right? But there's a difference, which is that we can foul that knowledge with the entropy in concept space.

• 1:11:46Speaker 00

  Right.

• 1:11:47Speaker 03

  That's your point. Now, you could have done that with coal deposits, but it would be difficult.

• 1:11:51Speaker 02

  Yeah, and why would you?

• 1:11:52Speaker 03

  Why would you?

• 1:11:52Speaker 02

  Right, right.

• 1:11:53Speaker 03

  But now, unfortunately, we can take this knowledge, synthesize it, and put back crap.

• 1:11:59Speaker 00

  Right.

• 1:12:00Speaker 03

  And that's a, so it's, it's different and similar in, in rather interesting ways.

• 1:12:04Speaker 02

  Well, could, is the crap because it doesn't know how to do it? Or is the crap because so much of what we do have in the library is just horseshit? I mean, just because it's a book doesn't mean it is correct or even contributes to the greater good of us all. And the LLM doesn't know that, because it's not trained at a university to be able to sift the wheat from the chaff. It just, they're just words statistically associated with other words and outcomes.

• 1:12:35Speaker 00

  It's, it's a probabilistic aggregation.

• 1:12:38Speaker 02

  Oh, I like that.

• 1:12:39Speaker 00

  But the difference is, the crap that's in our libraries is an example to us of our journey and how we got to where we are. So they become guardrails, so that these are the mistakes, the thoughts, the things that we know, that is horse crap. Whereas the LLM, without, if we didn't know those things, it would give us that and we'd just say, oh, here we are.

• 1:13:07Speaker 02

  Wait, if I say, what should I do with a witch? It goes back and finds all the literature on what you do with witches.

• 1:13:13Speaker 00

  Witches.

• 1:13:13Speaker 02

  It says, oh, you should obviously burn them.

• 1:13:14Speaker 00

  Burn them.

• 1:13:15Speaker 02

  And this is how you burn them.

• 1:13:16Speaker 00

  Right. Whereas when you go to the library, you find out, oh, the witch trials were an attack.

• 1:13:24Speaker 02

  Because you have context. Tell me about context.

• 1:13:26Speaker 03

  Well, this is, I've been working on papers on mathematical ability in these machines, based on this notion of the chronology of the development of the concept. So one really important difference between humans and these things is that we learn at school using a curriculum, right? Everyone, you typically learn what a number is before you learn, like a natural number or an integer, before you learn a fraction. And then you learn fractions typically before you learn pi, real numbers. And you learn that typically before you learn algebra, geometry, or group theory, or something more refined. Okay? So there's a sequence. It's understood, we learn incrementally using a curriculum.

• 1:14:09Speaker 03

  These models learned everything all at once. There is no curriculum. Another question is, what does that mean for how we do math and they do math? And it means lots of things for how we do math and they do math. And one of the things it means is this, which is that they basically find very complicated, what we would call heuristics, like rules of thumb, approximations actually, for solving problems that we solve exactly using algorithms. Let me make that explicit. So when I learnt how to do long division, right? Assuming as I, I did it right, I'd get the right answer, right? You have to learn how to do it right, but it'd always be right regardless of the scale.

• 1:14:49Speaker 03

  The current best models, LLMs, will not get it right regardless of the scale, guaranteed. They'll get it wrong at large scale.

• 1:14:57Speaker 02

  Long division.

• 1:14:58Speaker 03

  They'll get long division wrong at large scale if you have too many digits, because they're using heuristics, right? And so part of our training was to learn these rules piecewise through a curriculum. Their mode of learning is giving rise to a situation where, despite the fact they look great, at large scales, they're going to confabulate.

• 1:15:19Speaker 00

  Wow, look at that.

• 1:15:20Speaker 03

  And make wrong answers. And we're not going to be smart enough, because we've outsourced to them to know the difference.

• 1:15:26Gary O'Reilly?

  Well, this is what Jeffrey Hindman was saying, that they're overconfident.

• 1:15:29Speaker 00

  Yes.

• 1:15:30Gary O'Reilly?

  So they, they're expecting now us to just take that for granted as the answer.

• 1:15:36Speaker 03

  Yeah, I always loved Douglas Adams, like Hitchcock's Guide to the Galaxy.

• 1:15:39Speaker 00

  Yeah.

• 1:15:40Speaker 03

  And they build this supercomputer and they ask you, what's the meaning of life, the universe and everything?

• 1:15:44Speaker 02

  So what's the answer?

• 1:15:45Speaker 03

  And the answer is like forty-two.

• 1:15:46Speaker 02

  Yeah.

• 1:15:46Speaker 03

  And you think, what's, what does that mean?

• 1:15:49Speaker 00

  What's it mean?

• 1:15:50Speaker 03

  We're now in a world where we could be told.

• 1:15:52Speaker 00

  That forty-two is the answer.

• 1:15:54Speaker 03

  Forty-two is the answer. We wouldn't know what it meant.

• 1:15:55Speaker 00

  Right.

• 1:15:56Gary O'Reilly?

  Do you think the universe will allow a superintelligence to be created, to evolve, and therefore solve the mystery of the universe? And hopefully it's not forty-two.

• 1:16:08Speaker 02

  And that presumably be an alien somewhere.

• 1:16:10Gary O'Reilly?

  Yeah.

• 1:16:11Speaker 03

  So we, so I have problems with the concept of superintelligence, I should just say. If, if you buy my definition, which is making hard problems easy, it's not clear to me that the path we're on now is about making hard problems easy. I think it's about solving some hard problems, but doing it in a very convoluted way, which I do not consider superintelligence. That's number one.

• 1:16:34Gary O'Reilly?

  Yeah.

• 1:16:35Speaker 03

  In terms of, I, I'm subscribed to the idea of kind of universal Darwinism. Now, I do mean universal.

• 1:16:43Speaker 02

  Granted.

• 1:16:44Speaker 03

  Right, that any.

• 1:16:45Speaker 02

  You need permission, and I just gave you permission.

• 1:16:46Speaker 03

  I'm just saying.

• 1:16:47Speaker 02

  Okay.

• 1:16:47Speaker 03

  And that in most life forms, however different they would be, would be living under conditions of scarcity. That's sort of prerequisite for evolution in the theory, actually, because to compete for scarce resources. And so oddly enough, they will look more like us in their intelligence than they might look like LLMs. Now, it could be that that alien intelligence is all that's left once those AIs have devoured the life forms that built them, in which case they will be more like LLMs. But I think it's an important point to, to note that, under the conditions that evolution takes place, you want to be intelligent by having small amount of data

• 1:17:30Speaker 03

  and quite restricted experience of the world, and making a decision relatively quickly, so that it actually enhances your survival. And not using megawatts of energy to come to the solution.

• 1:17:41Speaker 02

  So we can imagine a planet where the environment shifts with some frequency. Like, let's say you, you could program this, so that you're always forcing an evolutionary change. The, the organism never stays happy. It finally settles in, you give it some assault, make sure you don't kill them all, because that, that's what happens most of the time. But there's some portal that it can move through and start evolving. Is it inevitable that that will lead to what we call intelligence? Or just more appendages, or more senses, or more, does that give us the superintelligence that we're thinking, the biological superintelligence?

• 1:18:25Speaker 03

  So the open question is why we didn't just stay bacterial? That worked pretty well. They outnumber us.

• 1:18:35Speaker 02

  It's still working.

• 1:18:36Speaker 03

  Still working.

• 1:18:36Speaker 02

  Still works.

• 1:18:36Speaker 03

  Still working, and we're dependent on it.

• 1:18:37Speaker 00

  Yeah.

• 1:18:39Speaker 03

  And the way to think about this is that there is a fundamental limit to how information can be acquired by an evolutionary process. Because a mathematical theory tells us that you can only put one bit of information into a genome every generation. And as you get bigger, you get more energetically efficient, but your generation time gets longer. So at large scales, it takes more and more time to get a bit by selection into the genome.

• 1:19:10Speaker 02

  Why?

• 1:19:11Speaker 03

  Because the way bits get put into life forms is through differential survival.

• 1:19:16Speaker 02

  Yes.

• 1:19:17Speaker 03

  So, so imagine you had to learn.

• 1:19:19Speaker 02

  But why is it only one bit? Why can't it be a suite of bits?

• 1:19:22Speaker 03

  Yeah, because selection would get confused about which bit it put in.

• 1:19:27Speaker 02

  It would be confused about which bit, bit was important for the survival and the next step.

• 1:19:30Speaker 03

  Exactly.

• 1:19:31Speaker 02

  Got it.

• 1:19:32Speaker 03

  So for it to know the difference, it has to be one of them.

• 1:19:34Speaker 00

  So it's like a brick at a time.

• 1:19:36Speaker 03

  Brick at a time.

• 1:19:37Speaker 00

  It's a brick at a time.

• 1:19:38Speaker 03

  So when you're very small, like a microbe, a brick at a time can be fast. But as you get bigger and longer lived, that's slow. So what did we do? We evolved brains. So brains can add bits more quickly than genomes can add bits. That's why we have brains. So there's two things going on. We get bigger to get more efficient energetically, but it makes us slower evolving, which makes us have to have brains to pull information from the world when the world changes. So there is a kind of underlying ratchet towards what we would call higher complexity, and therefore towards greater intelligence.

• 1:20:17Speaker 00

  Wow.

• 1:20:18Speaker 03

  Hmm.

• 1:20:18Speaker 00

  That's great, though, because that means out there somewhere is probably another intelligence along the same track as we. But maybe there's a variable that they didn't encounter that keeps them from killing each other, destroying cultures and enslaving one another and all of these stupid things that we do. And maybe they're just so much more advanced, because they don't retard their own advancement.

• 1:20:49Speaker 02

  Although, everything he said about a species moving through derives from, in most cases, competition.

• 1:20:57Speaker 00

  And that competition is what creates that other stuff that I was just.

• 1:21:00Speaker 02

  And that's, and that's our, our reptilian brain.

• 1:21:03Speaker 00

  God damn it. Damn it.

• 1:21:08Speaker 02

  You were onto something for a little bit there.

• 1:21:11Speaker 00

  I was gonna say something.

• 1:21:14Speaker 02

  So.

• 1:21:14Gary O'Reilly?

  Well, it does make it sound like there's some kind of plan.

• 1:21:17Speaker 02

  Plan?

• 1:21:18Gary O'Reilly?

  Yeah, rather than it's just random occurrence.

• 1:21:21Speaker 02

  Plan?

• 1:21:22Gary O'Reilly?

  Some kind of.

• 1:21:23Speaker 03

  Well, there are trends. You could say there are trends in evolution. I mean, the question to your observation, Chuck, is at this point, where we could transcend the competitive basis of evolution by building AI, could we?

• 1:21:41Speaker 02

  That's the way.

• 1:21:42Speaker 03

  Could we?

• 1:21:42Speaker 02

  That's the pathway.

• 1:21:43Speaker 03

  That's the pathway. Unfortunately.

• 1:21:45Speaker 00

  Right.

• 1:21:46Speaker 02

  Uh oh.

• 1:21:46Speaker 03

  It's not what we're doing.

• 1:21:47Speaker 00

  Yeah, it's not what we're doing.

• 1:21:48Speaker 02

  Right.

• 1:21:51Speaker 02

  Man.

• 1:21:51Speaker 00

  Man, this is, I tell you, it's so much easier to tell dick jokes.

• 1:21:58Speaker 03

  Probably not.

• 1:22:02Speaker 02

  One last point. How long have you been at the Santa Fe Institute?

• 1:22:05Speaker 03

  About 20 years.

• 1:22:06Speaker 02

  Oh, that's a long time.

• 1:22:07Speaker 03

  It is a long time.

• 1:22:08Speaker 02

  Big part of your career.

• 1:22:08Speaker 03

  It is a big part. It is my career.

• 1:22:10Speaker 02

  And they're famous or infamous for.

• 1:22:12Speaker 03

  Famous.

• 1:22:14Speaker 02

  You didn't hear what I was about to say. For dangling people at the edge of what is known and exploring places that are just on the surface crazy sounding, but might be something profound. So that's still kind of an operating principle?

• 1:22:32Speaker 03

  Yeah, I think the history of science is risk. And then it got institutionalized.

• 1:22:38Speaker 02

  Yeah.

• 1:22:39Speaker 03

  And it became a bureaucracy that was enforcing norms. And SFI wants to be heterodox and to be.

• 1:22:48Speaker 02

  SFI, the Santa Fe Institute.

• 1:22:50Speaker 03

  And that's a different environment. To be in a high-risk environment is a little bit going back to the time of early science.

• 1:22:56Speaker 00

  I was about to say, you're just taken aback to when science was kind of wild west and people were just like, hey.

• 1:23:03Speaker 02

  Try this.

• 1:23:04Speaker 00

  We should do this. Yeah, great. Try this.

• 1:23:06Speaker 02

  Smell that.

• 1:23:07Speaker 00

  Smell that.

• 1:23:07Speaker 02

  Smoke that.

• 1:23:08Speaker 00

  Right. Exactly.

• 1:23:09Speaker 02

  And so that sounds like it could be quite fertile for people on the frontier with powerful imaginations for what could be true.

• 1:23:18Speaker 03

  Yeah, I mean, I think now, because of all these tools that are solving technical problems, what we have to cultivate is amazing question askers. That is what remains for humans, right? Because it's like, you don't need to be good at arithmetic once you've got a calculator. Probably don't have to be good at calculus anymore, but you have to be able to ask good questions. And that environment is something different.

• 1:23:46Speaker 02

  So on that point, let's land this plane on a quote from a poem by the German poet Reinhard Maria Röck, I think I'm pronouncing it right.

• 1:23:57Speaker 03

  Okay.

• 1:23:58Speaker 02

  Here it goes. It's quoted in my Alien book, Take Me to Your Leader. Be patient toward all that is unsolved in your heart and try to love the questions themselves.

• 1:24:13Speaker 00

  That's a great quote. Yes.

• 1:24:16Speaker 02

  That's beautiful.

• 1:24:17Speaker 03

  Mm hmm.

• 1:24:17Speaker 00

  And right after that, she said, we have ways of making you talk.

• 1:24:26Speaker 03

  You couldn't help.

• 1:24:27Speaker 02

  Professor, thank you.

• 1:24:28Speaker 03

  Thank you so much.

• 1:24:28Speaker 02

  Thank you for yet another tour of duty on Startalk.

• 1:24:32Speaker 03

  Yeah, wonderful to be with you guys.

• 1:24:34Speaker 00

  Yeah, too much. Too much to think about.

• 1:24:35Speaker 02

  Yeah, now we gotta think about, I can't even. I'm out for two months now.

• 1:24:42Speaker 02

  But how do we find you? Are you online?

• 1:24:44Speaker 03

  Oh, you find me. Find me at the Santa Fe Institute. You can find me at davidccrackhour.com.

• 1:24:50Speaker 00

  There you go.

• 1:24:51Speaker 02

  Okay.

• 1:24:53Speaker 03

  Find me anywhere.

• 1:24:53Speaker 02

  Yeah, just keep up with you because we're doing fun, interesting things.

• 1:24:57Speaker 03

  Thank you.

• 1:24:57Speaker 02

  And I have a special appreciation for your evolutionary biology background because I work here as an astrophysicist in a museum famous for its evolutionary biology. And so I have an extra little osmotic attachment to your.

• 1:25:13Speaker 03

  Well, thank you. It's wonderful being with you guys.

• 1:25:15Speaker 00

  People keep him working.

• 1:25:19Speaker 02

  Chuck?

• 1:25:19Speaker 00

  Always a pleasure.

• 1:25:20Speaker 02

  We're good here. Gary?

• 1:25:22Gary O'Reilly?

  Pleasure, my friend.

• 1:25:23Speaker 02

  All right. This has been another installment of Startalk Special Edition. And we're going to call this the Intelligence installment. Are we or are we not? Until next time, keep looking up.

• 1:25:48Speaker 05

  This episode is brought to you by Whoop, the wearable health and fitness coach that gives you personalized insights into your sleep, recovery, strain, stress, and overall health, so you can finish summer feeling more aware of what you're capable of. With Whoop, you can better understand how your body is responding to travel, training, late nights, warmer weather, and packed weekends. It helps you see what's supporting your energy and recovery and what might be holding you back. Visit join.whoop.com/siriusxm.

• 1:26:16Speaker 04

  Law transforms agreement into reality. A word becomes an agreement. An agreement becomes a contract. A contract becomes law. Today, Harvey, the AI platform for legal work, is writing the next chapter. More than 200,000 lawyers and 80% of the Am Law 100 are building with Harvey, accelerating review, sharpening negotiations, and more. This is how the future of agreements is being crafted now. Harvey. Learn more at harvey.ai. Harvey does not provide legal advice. Users should always independently review and verify any output from Harvey.