The Origins of Artificial Intelligence with Geoffrey Hinton

How did we go from digital computers to AI seemingly everywhere? Neil deGrasse Tyson, Chuck Nice, & Gary O’Reilly dive into the mechanics of thinking, how AI got its start, and what deep learning really means with cognitive and computer scientist, Nobel Laureate, and one of the architects of AI, Geoffrey Hinton.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Neil deGrasse Tyson?

  Gary, you had the audacity to bring in a Nobel laureate expert in AI to tell us we're all gonna die? Well, of course. Actually, all we had to do was ask AI.

• 0:10Neil deGrasse Tyson?

  Coming up, StarTalk special edition, how AI will be the death or the birth of civilization. Coming right up.

• 0:20Neil deGrasse Tyson?

  Welcome to StarTalk, your place in the universe where science and pop culture collide. StarTalk begins right now.

• 0:33Neil deGrasse Tyson?

  This is StarTalk special edition. Neil deGrasse Tyson, your personal astrophysicist, and if it's special edition, it means we've got Gary O'Reilly.

• 0:41Gary O'Reilly?

  Hey, Neil.

• 0:42Neil deGrasse Tyson?

  Gary, how you doing, man?

• 0:43Gary O'Reilly?

  I'm good.

• 0:44Neil deGrasse Tyson?

  Former soccer pro.

• 0:45Gary O'Reilly?

  Yes.

• 0:45Neil deGrasse Tyson?

  So Chuck, always good to have you.

• 0:47Gary O'Reilly?

  Always a pleasure.

• 0:48Neil deGrasse Tyson?

  So Gary, you and your team picked a topic for the ages today.

• 0:51Gary O'Reilly?

  Yeah, it's it's one of those things that we hear about it, we think we know about it, but let me put it to you this way. We are faced with the simple fact that AI at this point is.

• 1:03Neil deGrasse Tyson?

  We're gonna talk about AI today.

• 1:04Gary O'Reilly?

  We are.

• 1:04Neil deGrasse Tyson?

  Okay.

• 1:05Gary O'Reilly?

  It's inescapable.

• 1:05Neil deGrasse Tyson?

  A deep dive.

• 1:07Gary O'Reilly?

  Oh, yeah.

• 1:07Neil deGrasse Tyson?

  Yes, go.

• 1:08Gary O'Reilly?

  Right. It was only a few years ago when we asked people how AI works, they'll say something along the lines of it utilizes deep learning, neural networks, but.

• 1:16Neil deGrasse Tyson?

  They're buzzwords.

• 1:17Gary O'Reilly?

  Yeah.

• 1:17Neil deGrasse Tyson?

  They'll toss them out.

• 1:18Gary O'Reilly?

  They know them, but they don't know anything about them. So what does that really mean? Um, we'll break down how AI works down to the bit and get into how far we think this is going to go from one of AI's founding architects. Oh. Yes. As well do you, Venu. Now we're talking. Mm hmm. So if you would bring on our guest.

• 1:38Neil deGrasse Tyson?

  I'll be delighted to. We have with us Professor Geoffrey Hinton. Geoffrey, welcome to StarTalk.

• 1:46Speaker 02

  Thank you for inviting me.

• 1:47Neil deGrasse Tyson?

  Yeah, you are a cognitive psychologist. And computer scientist. I don't know anybody with that combo. Couldn't make up your mind, huh?

• 1:58Neil deGrasse Tyson?

  Professor Emeritus at the Department of Computer Science at the University of Toronto. And you are OG AI. Oh, lovely. Can I say that? Yes, you can. OG AI. OG AI. And some people have called you the godfather of AI, of artificial intelligence. And let's just go straight out off the top here. When we think of the genesis of AI as it is currently manifested, it feels like large language models took everybody by storm. They sort of showed up and everybody was freaking out, celebrating, dancing in the streets, or crying in their pillows. That happened, we noticed, a couple of years ago.

• 2:43Neil deGrasse Tyson?

  So I'm just wondering, what got you started in, on this path many, many years ago? My record show goes back to the 1990s, is that correct?

• 2:54Speaker 02

  No, it really goes back to the 1950s.

• 2:56Neil deGrasse Tyson?

  Oh, right.

• 2:58Speaker 02

  The founders of AI, at the beginning, in the 1950s, there were two views of how to make an intelligent system. One was inspired by logic. The idea was that the essence of intelligence is reasoning. And in reasoning, what you do is you take some premises and you take some rules for manipulating expressions, and you derive some conclusions. So it's much like mathematics, where you have an equation, you have rules for how you can tinker with both sides and/or combine equations, and you derive new equations. And that was kind of the paradigm they had. There was a completely different paradigm that was biological. And that paradigm said, look, the intelligent things we know have brains. We have to figure out how brains work.

• 3:41Speaker 02

  And the way they work is they're very good at things like perception. They're quite good at reasoning by analogy, they're not much good at reasoning. You have to get to be a teenager before you can do reasoning, really. So we should really study these other things they do. And we should figure out how big networks of brain cells can do these other things like perception and memory. Now, a few people believed in that approach. And among those few people were John von Neumann and Alan Turing. Unfortunately, they both died young. Turing possibly with the help of British intelligence.

• 4:14Neil deGrasse Tyson?

  Turing, he's the subject of the film. Of course. The Imitation Game. Mm-hmm. Yeah, yeah, so, anyone hasn't seen that, definitely put that on your list. Cool. Yeah, so, all right, we'll go back to the 1950s. You were just a young tyke then, correct?

• 4:28Speaker 02

  Oh, yeah, I was in single digits then. I was in single digits.

• 4:32Neil deGrasse Tyson?

  Okay, so how do we establish the genesis of your curiosity in this field?

• 4:38Speaker 02

  Um, a few things. When I was at high school in the early 1960s, or mid 1960s, I had a very smart friend who was a brilliant mathematician and used to read a lot. And he came into school one day and talked to me about the idea that memories might be distributed over many brain cells instead of in individual brain cells. So that was inspired by holograms. Holograms were just coming out then. Gabor was active. And so the idea of distributed memory got me very interested. And ever since then, I've been wondering how the brain stores memories and actually how it works.

• 5:16Neil deGrasse Tyson?

  Was that the computer science side of you or the cognitive psychologist side of you that tap rooted into that, those ideas?

• 5:24Speaker 02

  Both, really. Um, but in the 1970s, when I became a graduate student, um, it was obvious that there was a new methodology that hadn't been used that much, which was if you have any theory of how the brain works, you can simulate it on a digital computer, unless it's some crazy theorem that says it's all quantum effects. Um. And let's not go there.

• 5:47Neil deGrasse Tyson?

  That's right. Not yet. We won't knock on Penrose's door, okay?

• 5:52Speaker 02

  You can simulate it on a digital computer, and so you can test out your theory. And it turns out if you tested most of the theories that were around, they actually didn't work when you simulated them. So I spent my life trying to figure out how you change the strength of connections between neurons so as to learn complicated things in a way that actually works when you simulate it on a digital computer. And I fail to understand how the brain works. We've understood some things about it. But we don't know how a brain gets the information it needs to change connection strengths. You know, gets the information it needs to know whether it needs to increase the connection strength to be better at a task or to decrease that connection strength. But what we do know is we know how to do it in digital computers now.

• 6:34Neil deGrasse Tyson?

  Well, so that that means the computers are doing what we we made a better computer brain than our own brain.

• 6:42Gary O'Reilly?

  At doing this particular function.

• 6:43Speaker 02

  One thing. And that's what got me really nervous in the beginning of 2023. The idea that digital intelligence might just be better than the analog intelligence we've got.

• 6:53Gary O'Reilly?

  Interesting. Saved a scary bit till a bit later on. Let me have the 10 minutes of just breathing in, breathing out. If we take a step back.

• 7:02Speaker 02

  You're assuming you're assuming there's just one scary bit.

• 7:05Gary O'Reilly?

  No, I'm not. I'm gonna go one at a time.

• 7:09Neil deGrasse Tyson?

  Okay.

• 7:10Gary O'Reilly?

  Um, artificial neural networks. Uh, if you could break that down to the very basic level for us of how it's been able to strengthen, weaken messaging and signaling and how it fires and and how it then finds itself at where it is now.

• 7:28Speaker 02

  I do have an 18 hour course on this, but I will try and cut it down to less than 18 hours.

• 7:32Gary O'Reilly?

  Um, please do.

• 7:33Speaker 02

  So I imagine a lot of your audience knows some physics.

• 7:37Neil deGrasse Tyson?

  Yes.

• 7:38Speaker 02

  And one way into it is to think about something like the gas laws. You know, you compress the gas and it gets hotter. Why does it do that? Well, underneath, there's a kind of seething mass of atoms that are buzzing around. And so the real explanation for the gas laws is in terms of these microscopic things that you can't even see buzzing around. And so you explain some macroscopic behavior.

• 8:06Speaker 02

  By lots and lots and lots of little things of a completely different type from macroscopic behaviour interacting. And that was sort of the inspiration for the neural net view, that there's things going on in big networks of brain cells that are a long way away from the kind of conscious deliberate symbol processing we do when we're reasoning, but that underpin it and that it may be better at other things than reasoning like perception or reasoning by analogy. So the symbolic people could never deal with, how do we reason by analogy, not very satisfactorily, whereas the neural nets could. So before I get into the sort of fine details of how it works, the basic idea is that macroscopic

• 8:50Speaker 02

  things like a word correspond to big patterns of neural activity in the brain. Similar words correspond to similar patterns of neural activity. So the idea is Tuesday and Wednesday will correspond to very similar patterns of neural activity, where you can think of each neuron as a feature, better to call it a microfeature, that when the neuron gets active, it says this has that microfeature. So if I say cat to you, all sorts of microfeatures will get active, like it's animate, it's furry, it's got whiskers, it might be a pet, it's a predator, all those things. If I say dog, a lot of the same things will get active, like it's a predator, it might be a pet, but some different things, obviously.

• 9:33Speaker 02

  So the idea is underlying these symbols that we manipulate, there's much more complicated microscopic goings on that the symbols kind of are associated with. And that's where all the action really is. And if you really want to explain what goes on when we think or when we do analogies, you have to understand what's going on at this microscopic level, and that's the neural network level.

• 9:57Gary O'Reilly?

  So that's a collaboration between clusters of neurons that get you to an endpoint?

• 10:03Neil deGrasse Tyson?

  I like that word collaboration.

• 10:05Speaker 02

  Yes, there's a lot of that, there's a lot of that goes on. Probably the easiest way to get into it is by thinking of a task that seems very natural, which is take an image, let's say it's a black, a grey level image. So it's got a whole bunch of pixels, little areas of uniform brightness, that have different intensity levels. So as far as the computer's concerned, that's just a big array of numbers. And now imagine the task is you want to say whether there's a bird in the image or not, or rather whether the prominent thing in the image is a bird. And people tried for many, many years, like half a century, to write programs that would do that. And they didn't really succeed. And the problem is, if you think what a bird looks like in an image,

• 10:50Speaker 02

  well, it might be an ostrich up close in your face, or it might be a seagull in the far distance, or it might be a crow. So they might be black, they might be white, they might be tiny, they might be flying, they might be close, you might just see a little bit of them. There might be lots of other cluttered things around, like it might be a bird in the middle of a forest. So it turns out it's not trivial to say whether there's a bird in the image or not. And so what I'm going to do now is explain to you, if I was building a neural network by hand, how I would go about doing that. And once I've explained how I would build a neural network by hand, I can then explain how I might learn all the connection strengths instead of putting them in by hand.

• 11:29Chuck Nice?

  I got you. Alright, so with that, because what you're talking about is assigning a mathematical value to every single part of an image.

• 11:38Speaker 02

  That's what your camera does.

• 11:39Chuck Nice?

  Right, exactly. It does, but it's not recognizing the image, my camera.

• 11:44Speaker 02

  No, it's not. It's just got a bunch of numbers.

• 11:45Chuck Nice?

  It's just got a bunch of numbers. And, and so I have a chip and I have a charged couple of devices, CCD. It's collecting the light, it's assigning a value, and then that's the picture. Now, but what you're talking about, wouldn't you have to assign a value to every single type of bird? Because some of what we do as human beings is intuit what a bird may be as opposed to recognizing the bird. And let me just give you the example. If you were to take a V, the letter V, and curve the straight lines of the letter V and put it in a cloud, everyone who sees that will say, that's a bird. But yet it is-

• 12:29Neil deGrasse Tyson?

  No, to me, it's a curved V.

• 12:32Chuck Nice?

  But no one, but, but, but there is no bird there. I just know that is a bird. That's not a mathematical value now. So what are you doing?

• 12:43Speaker 02

  Well, the question is, how do you just know that? There's something going on in your brain, right?

• 12:48Chuck Nice?

  Right.

• 12:48Speaker 02

  And what might be going on in your brain so that you just know that's a bird is a whole bunch of activation levels of different neurons, which you could think of as mathematical values.

• 12:58Chuck Nice?

  I got you. Okay. So wouldn't that require then, training this neural net on every possible way

• 13:07Neil deGrasse Tyson?

  A bird can manifest.

• 13:09Chuck Nice?

  A bird can manifest.

• 13:09Neil deGrasse Tyson?

  In a photo.

• 13:10Chuck Nice?

  So that it can intuit what a bird might be when a bird is not there.

• 13:12Neil deGrasse Tyson?

  But at that point, it's not intuiting anything. It's just going off a lookup table.

• 13:16Chuck Nice?

  It really is going on a little.

• 13:17Speaker 02

  No.

• 13:19Chuck Nice?

  Oh.

• 13:20Speaker 02

  All right, here comes your answer. There's something called generalization. So if you see a lot of data, obviously, you can make a system that just remembered all that data. But in a neural net, it'll do more than just remember the data. In fact, it won't literally remember the data at all. What it'll do is it'll, as it's learning on the data, it'll find all sorts of regularities, and it'll generalize those regularities to new data. So it will be able to, for example, recognize a unicorn, even though it's never seen one before.

• 13:56Chuck Nice?

  Interesting.

• 13:57Speaker 02

  So it's self-teaching?

• 13:58Chuck Nice?

  Yeah.

• 13:58Speaker 02

  Let me carry on with my explanation of how neural networks work. And I'm going to do it by saying how I would design one by hand. So your first thought, when you see that an image is just a big array of numbers, which are how bright each pixel is, is to say, well, let's hook up those pixel intensities to our output categories, like bird and cat and dog and politician, or whatever our output categories are. And that won't work. And the reason is, if you think about what does the brightness of one pixel tell you about whether it's a bird or not? Well, it doesn't tell you anything. Because birds can be black and birds can be white. And there's all sorts of other things

• 14:42Speaker 02

  that can be black and white. So the brightness of a pixel doesn't tell you anything. So what can you derive from those numbers that you have in the image that describe the image? Well, the first thing you can derive, which is what the brain does, is you can recognize when there's little bits of edge present. So suppose I take a little column of three pixels and I have a neuron that looks at those three pixels, a brain cell, and has big positive weights to those three pixels. So when those pixels are bright, the neuron gets very excited. Now, that would recognize a little streak of white that was vertical. But now suppose that next to it, there's a column, another column of three pixels. So the first column was on the left and the second column was on the right.

• 15:27Speaker 02

  And I give the neuron big negative connection strengths to those pixels. So you can think of the neuron as getting votes from the pixels. So for the three pixels on the right, the votes it gets, sorry, on the left, the votes it gets, are big positive numbers times big positive intensities, so great big votes. Now, from the three pixels in the right-hand column, it's got negative weights. So if those pixels are bright, it'll get a big brightness times a big negative weight. So it'll get a lot of negative votes. And they'll all cancel out. So if the column of pixels on the left is the same brightness as the column of pixels on the right, the positive votes it gets from the left-hand column will cancel the

• 16:10Speaker 02

  negative votes it gets from the right-hand column, and it'll get zero net input and it'll just stay quiet. But if the pixels on the left are bright and the pixels on the right are dim, the negative votes will be multiplied by small intensity numbers, and the positive votes will be multiplied by big intensity numbers. And so the neuron will get lots of input and get very excited and say, I found the thing I like. And the thing it likes is an edge which is brighter on the left than on the right. So we do know how to make a neuron, if we handwire it like that, pick up on the fact that there's an edge at a particular location in the image that's brighter on one side than the other side.

• 16:52Speaker 02

  Now, what the brain does, roughly speaking, a lot of neuroscientists will be horrified by me saying this, but very roughly speaking, what the brain does is, in the early stages of visual cortex, which is where you recognize objects, it has lots and lots of neurons that pick up on edges at different orientations, in different positions, and at different scales. So it has thousands of different positions, and dozens of different orientations, and several different scales, and it has to have edge detectors for each of the, each combination of those. So it has like a gazillion little edge detectors, well, including some

• 17:32Speaker 02

  big edge detectors. So, a cloud, for example, has a big, soft, fuzzy edge, and you need a different neuron for detecting that than what you'd need for detecting, say, the tail of a mouse disappearing around a corner in the distance, which is a very fine thing. And you'd need an edge detector that was very sharp and saw very small things. So, first stage, we have all these edge detectors.

• 17:56Chuck Nice?

  Well, what you're describing, uh, sounds like, uh, putting together a, a very large puzzle right now. Like, you know, the kind of puzzles that you put down on the table? Uh, the first thing that you do is you want to find all the edges. And that's how you build the puzzle inward from finding all the edges.

• 18:12Neil deGrasse Tyson?

  Not only edges of the physical puzzle, but edges of anything-

• 18:15Chuck Nice?

  But edges of images in the puzzle itself.

• 18:17Neil deGrasse Tyson?

  Anything within the puzzle itself.

• 18:18Chuck Nice?

  Right. So, straight lines, things of that, they all match up when you're doing a puzzle.

• 18:21Neil deGrasse Tyson?

  And the edges, also color is a dimension of this-

• 18:23Chuck Nice?

  Right.

• 18:24Speaker 02

  Right. But we'll ignore color for now.

• 18:25Neil deGrasse Tyson?

  Yeah.

• 18:26Chuck Nice?

  Okay.

• 18:27Speaker 02

  Yeah. I mean, you can understand it without dealing with color, yeah.

• 18:30Neil deGrasse Tyson?

  Mm-hmm, mm-hmm.

• 18:42Speaker 02

  Hi, I'm Ernie Carducci from Columbus, Ohio. I'm here with my son Ernie because we listen to StarTalk every night and support StarTalk on Patreon.

• 18:52Neil deGrasse Tyson?

  This is StarTalk with Neil deGrasse Tyson.

• 19:03Speaker 02

  That's what the first layer of neurons will do. They'll look at the pixels and they'll detect little bits of edge. Now, in the next layer of neurons, what I would do is I'd make a neuron that maybe detects three little bits of edge that all line up with one another and slope gently down towards the right. And it also detects three little bits of edge that all line up with one another and slope gently upwards towards the right. And what's more, those two little combinations of three edges join in a point. So I think you can imagine some edges sloping down to the right, some edges sloping up to the right and joining in a point. And I have a neuron that detects that. Okay. And we know how to build that now.

• 19:48Speaker 02

  You just give it the right connections to the edge detecting neurons. And maybe you give it some negative connections to neurons to detect edges in different orientations, so it doesn't just go off anyway. It's suppressed by those. Now, that you might think of as something that's detecting a potential beak of a bird. If that guy gets active, it could be all sorts of things. It could be an arrowhead, it could be all sorts of things. But one thing it might be is the beak of a bird. So now you're beginning to get some evidence that's kind of relevant to whether or not it might be a bird. So, in the second layer of neurons, I'd have lots of things to detect possible beaks all over the place. I might also have things that detect a little combination of edges that form a circle, an approximate circle. And I'd have detectors for those all over the place.

• 20:33Speaker 02

  Because that might be a bird's eye. I mean, there's all sorts of other things. It could be a button, it could be a knob on a computer. It could be anything, but it might be a bird's eye. So, that's the second layer. Now, in the third layer, I might have something that looks for a possible bird's eye and a possible bird's beak that are in the right spatial relationship to one another to be a bird's head. I think you can see how I would do that. I'd hook up neurons in the third layer to the eye detectors and beak detectors that are in the right relationship to one another. Um. To be a bird's head. So now in the third layer, I have things that are detecting possible birds' heads. And I know how to build all that. I know how to put in the connection strengths on the connections.

• 21:17Speaker 02

  So when I put the pixels in, it'll activate a bunch of line detectors. Those will activate a bunch of sort of beacon eye detectors. And if they're in the right relative positions, they'll activate a bunch of birds' head detectors. And of course, I need these all over the image, so I need huge numbers of them.

• 21:32Neil deGrasse Tyson?

  Just in case the bird is somewhere, anywhere randomly in the image.

• 21:35Speaker 02

  Because the bird could be anywhere randomly in the image. The next thing I'm gonna do is maybe, because we're sort of running out of patience at this point. I'm gonna have a final layer that has neurons that say cat, dog, bird, politician, whatever. And in that final layer, I'll take the neuron that says bird. And I'll hook it up to the things that detect birds' heads. But I'll also hook it up to other things in the third layer that detect things like birds' feet or the tips of birds' wings. And so now, my sort of output neuron for bird, when that gets active, the neuron that is saying it's a bird. If it sees a bird's foot and a possible bird's head and a possible tip of the wing of a bird.

• 22:19Speaker 02

  It'll get lots of input and it'll say, hey, I think it's a bird. So I think you can now understand how I might try and design that by hand. And I think you can see there's huge problems in that. I need an awful lot of detectors. I need to cover this whole space of positions and orientations and scales. I need to decide what features to extract. I mean, I just made up the idea of getting a beak and then a bird's head. There may be much better things to go after. What's more, I want to detect lots of different objects. So what I really need is features that aren't just good for finding birds, but features that are good for finding all sorts of things. And it would be a nightmare to design this by hand.

• 22:57Speaker 02

  Particularly if I figured out that, to do a good job of this, I needed a network with at least a billion connections in it. So I have to by hand design the strengths of these billion connections. And that'll take a long time.

• 23:12Neil deGrasse Tyson?

  Hmm.

• 23:12Speaker 02

  Then we say, well, okay. A network like that, maybe it could recognize birds if it had the right connection strengths in it. But where am I going to get those connection strengths from? Because I sure as hell don't want to put them in by hand. I don't even want to tell my graduate students to put them in.

• 23:26Neil deGrasse Tyson?

  Yeah, that's what they're there for, professor.

• 23:28Speaker 02

  That's absolutely what they're there for, but you need about 10 million of them for this.

• 23:32Neil deGrasse Tyson?

  Okay. All right, well, now we've got a problem.

• 23:35Speaker 02

  Can you imagine the grants you'd have to write to support 10 million graduate students?

• 23:39Neil deGrasse Tyson?

  My word.

• 23:40Speaker 02

  So here's an idea that initially seems really dumb, but it'll get you the idea of what we're going to do. We're going to start with random connection strengths. Some will be positive numbers, some will be negative numbers. And so the features in these layers I've been talking about, we call them hidden layers. The features in those layers will be just random features. And if we put in an image of a bird and look at how the output neurons get activated. The output neurons for cat and dog and bird and politician will all get activated a tiny bit and all about equally. Because the connection strength is just random.

• 24:17Neil deGrasse Tyson?

  Yeah.

• 24:17Speaker 02

  So that's no good, but we could now ask the following question. Suppose I took one of those connection strengths, one of those billion connection strengths. And I said, okay, I know this is an image of a bird. And what I'd really like is, next time I present you with this image. I'd like you to give slightly more activation to the bird neuron. And slightly less activation to the cat and dog and politician neurons. And the question is, how should I change this connection strength? Well, I could do an experiment. If I'm not very theoretical and don't know much math, I'd do an experiment. I would say, let's increase the connection strength a little bit and see what happens. Does it get better at saying bird? And if it gets better at saying bird, I say, okay, I'll keep that mutation to the connection.

• 25:00Neil deGrasse Tyson?

  Yeah, but better means there's a human in the loop making that judgment on the result of its experiment.

• 25:05Speaker 02

  Well, there has to be someone saying what the right answer is.

• 25:08Neil deGrasse Tyson?

  Absolutely.

• 25:08Speaker 02

  That's called the supervisor, yes.

• 25:10Neil deGrasse Tyson?

  Okay, okay.

• 25:11Speaker 02

  And the problem, if you do it like that, is there's a billion connection strengths. Each of them has to be changed many times. It's going to take like forever. So the question is, is there something you can do that's different from measuring? That's much more efficient than there is. You can do something called computing. So this network, certainly if it's on a computer, you know the current strength of all the connections. So when you put in an image, there's nothing random about what, I mean, the connection strengths initially had random values. But when you put in an image, it's all deterministic what happens next. The pixel intensities get multiplied by weights on connections to the first layer of neurons. Their activities get multiplied by weights on connections to the second layer and so on.

• 25:55Speaker 02

  And you get some activation levels of the output neurons. So you could now ask the following question. If I take that bird neuron, could I figure out for all the connection strengths at the same time, whether I should increase them a little bit or decrease them a little bit in order to make it more confident that this is a bird? In order for it to say bird a bit more lively and the other things a bit more quietly? And you can do that with calculus. You can send information backwards through the network saying, how do I make this more likely to say bird next time? And because you have a lot of physicists in the audience, I'm going to try and give you a physical intuition for this.

• 26:36Neil deGrasse Tyson?

  Go for it.

• 26:36Speaker 02

  Yeah. You put in bird, an image of a bird. And with the initial weights, the bird output neuron only gets very slightly active. And so what you do now is you attach a piece of elastic of zero rest length. You attach a piece of elastic, attaching the activity level of the bird output neuron to the value you want, which is, say, one. Let's say one's the maximum activity level and zero is the minimum activity level. And this had an activity level of like 0.01. You attach this piece of elastic. And that piece of elastic is trying to pull the activity level towards the right answer, which is one in this case. But of course, the activity level's being determined

• 27:19Speaker 02

  by the pixels that you put in, the pixel activation levels, the intensities, and all the weights in the network. So the activity level can't move. Now, one way to make the activity level move would be to change the weights going into the bird neuron. You could, for example, give bigger weights on neurons that are highly active, and then the bird neuron will get more active. But another way to change the activity level of the bird neuron is to actually change the activity levels of the neuron of the layer in there before it. So, for example, we might have something that sort of detected a bird's head but wasn't very sure this really is a bird. And so what you'd like is, the fact that you want the output to be more bird-like, you've

• 28:04Speaker 02

  got this piece of elastic saying, "More, more, I want more here," you'd like that to cause this thing that thought, "Maybe there's a bird's head here," to get more confident there's a bird's head there. So what you want to do is, you want to take that force imposed by the elastic on that output neuron, and you want to send it backwards to the neurons in the layer in front, before that, to create a force on them that's pulling them. And that's called backpropagation.

• 28:32Neil deGrasse Tyson?

  Backpropagation, okay.

• 28:34Speaker 02

  That is called backpropagation. And the physics way to think about it is, you've got a force acting on the output neurons, and you want to send that force backwards so that the force acts on the neurons in the layer in front. And of course, there's forces acting on many different output neurons, so you have to combine all those forces to get the forces acting on the neurons in the layer below. Once you send this all the way back through the network, you have forces acting on all these neurons, and you say, "Okay, let's change the incoming weights of each neuron so its activity level goes in the direction of the force that's acting on it." That's backpropagation, and that makes things work wondrously well.

• 29:11Gary O'Reilly?

  And diabolically.

• 29:12Speaker 02

  Yeah.

• 29:13Gary O'Reilly?

  Don't go there yet, okay, sorry. Is this the lightbulb moment where the neural networks no longer need the human teacher? Is this the beginning of that process?

• 29:24Speaker 02

  No, not exactly.

• 29:26Gary O'Reilly?

  Okay.

• 29:27Speaker 02

  This is a lightbulb moment, though. So for many years, the people who believed in neural networks knew how to change the very last layer of connection strengths, which we call weights, the ones that are going into the output units. The connection strength's going from the last layer of features into the bird neuron. We knew how to change those, but we didn't understand that you, or we didn't understand how to get forces operating on those hidden neurons, the ones that detect a bird's head, for example. And backpropagation showed us how to get forces acting on those, so then we could change the incoming weights of those. And that was a eureka moment. Many different people had that eureka moment at different times.

• 30:05Gary O'Reilly?

  So what period of time are we talking about here when you've

• 30:08Chuck Nice?

  Yeah, when are we?

• 30:10Gary O'Reilly?

  fall into the backpropagation thought?

• 30:12Speaker 02

  Okay, the early 1970s, there was someone in Finland who had it, I think in his master's thesis. And then in probably the late 70s, someone called Paul Werpos at Harvard had the idea. In fact, some control theorists there called Bryson and Ho had had the idea for doing things like controlling spacecraft. So when you land a spacecraft on the moon, you're using something very light backpropagation, but it's in a linear system. You're using backpropagation to figure out how you should fire the rockets.

• 30:46Chuck Nice?

  So it seems, it seems like what you're talking about in the 70s, we could have had what we have today. We just didn't have the mathematical computing power to make this work.

• 30:59Speaker 02

  That's a large part of it, yes.

• 31:00Chuck Nice?

  Yeah.

• 31:01Speaker 02

  The other thing we didn't have is, back in the 70s, people didn't show that when you applied this in multi-layered networks, what you get is very interesting representations. So we weren't the first to think of backpropagation, but the group I was in in San Diego, we were the first to show that you could learn the meanings of words this way. You could show a string of words, and by trying to predict the next word, you could learn how to assign features to words that captured the meaning of the word. And that's what got it published in Nature.

• 31:31Chuck Nice?

  It, it sounds like, and I'm just trying to get my head, my head around what you explained, because it sounds to me like there is a cascading relationship to these values, and that really what matters are the values that are closest to the next value. And then there are kind of this cascading reinforcement to say, yes, this is it, or no, it is not. I, I, am I getting that right? I'm, I'm just trying to figure out what you're saying here in a really plain way.

• 32:07Speaker 02

  Okay, so good question. You're not getting it quite right.

• 32:10Chuck Nice?

  Okay, go ahead.

• 32:11Speaker 02

  So this kind of, this kind of learning, where you back propagate these forces and then change all the connection strengths, so each neuron goes in the direction that the force is pulling it in, that's not reinforcement learning. This is called supervised learning.

• 32:25Chuck Nice?

  Okay.

• 32:26Speaker 02

  Reinforcement learning is something different. So here, for example, we tell it what the right answer is. If you've got a thousand categories and you showed a bird, you tell it that was a bird.

• 32:36Chuck Nice?

  There you go.

• 32:36Speaker 02

  In reinforcement learning, it makes a guess and you tell it whether it got the answer right.

• 32:40Chuck Nice?

  All right.

• 32:41Speaker 02

  That's much less information.

• 32:42Chuck Nice?

  You cleared it up. That's what I was missing.

• 32:44Gary O'Reilly?

  All right, to Chuck's point about computational power, was it just that? Because at the moment, you sound a lot like you've got theory that seems like it could be, but the practicality is there's not enough computational power. Do we have any other technology that came through that was the enabling aspect to this?

• 33:01Speaker 02

  Okay, so in, in the mid eighties, we had the backpropagation algorithm working, and it could do some neat things. It could recognize handwritten digits better than nearly any other technique, but it couldn't deal with real images very well. It could do quite well at speech recognition, but not substantially better than the other technologies. And we didn't understand at the time why this wasn't the magic answer to everything. And it turns out it was the magic answer to everything if you have enough data and enough compute power.

• 33:34Chuck Nice?

  Hmm, wow.

• 33:35Speaker 02

  So that's what was really missing in the eighties.

• 33:38Chuck Nice?

  All right, I'm gonna depart for a second just, just to pick your brain first. This is part commentary and part question. I'm gonna say that the majority of people that are walking around this planet are stupid. So what exactly is smart and what exactly is thinking? And will these machines, will we be able to teach them how to think? And will they outthink us?

• 33:58Speaker 02

  Okay, they already know how to think.

• 34:00Chuck Nice?

  Okay, so what is thinking then?

• 34:03Speaker 02

  Okay.

• 34:04Chuck Nice?

  Mm hmm.

• 34:05Speaker 02

  Well.

• 34:06Chuck Nice?

  Yeah.

• 34:09Chuck Nice?

  I could do this all day.

• 34:11Speaker 02

  There's a lot of elements to thinking, like people often think using images. You often think actually using movements. So when I'm wandering around my carpentry shop, looking for a hammer, but thinking about something else. I sort of keep track of the fact I'm looking for a hammer by sort of going like this. I wander around going like this while I'm thinking about something else. And that keep, that's a representation that I'm looking for a hammer. So we have many representations involved in thinking, but one of the main ones is language. And a lot of the thinking we do is in language.

• 34:42Chuck Nice?

  Hmm.

• 34:42Speaker 02

  And these large language models actually do think. So there's a big debate, right? Between the people who believed in old fashioned AI, that it was all based on logic and you manipulate symbols to get new symbols. They don't really think these neural nets are thinking. Whereas the neural net people think, no, they're, they're thinking, they're thinking pretty much the same way we do. And so the neural nets now, some of them, you'll ask them a question. And they'll output a symbol that says, I'm thinking. And then they'll start outputting their thoughts, which are thoughts for themselves. Like I give you a simple math problem, like there's a boat.

• 35:26Speaker 02

  And on this boat, there's a captain. There's also thirty five sheep. How old is the captain?

• 35:36Speaker 02

  Now, many kids of age around ten or eleven, particularly if they're educated in America, will say the captain is thirty five. Because they look around and they say, well, you know, that's a plausible age for a captain. And the only number I was given was these thirty five sheep. So they're operating at a sort of. Substituting symbols level. The AIs can sometimes be seduced into making similar mistakes. But the way the AIs actually work is quite like people. They take a problem and they start thinking. And you might, for a child, you might say, okay, well, how old is the captain? Well, what are the numbers I've got in this problem? Hey, I've only got a thirty five. Is that a plausible age for a captain? Yeah, he might be thirty five, a bit young, but maybe, okay, I'll say thirty five.

• 36:19Speaker 02

  That's what a ten year old child might think. And the child would think it to itself in words. And what people realise with these language models is you can train them to think to themselves in words. That's called chain of thought reasoning. And they train them to do that. And after that, they, you give them a problem, they'd think to themselves, just like a kid would. And sometimes come up with the wrong answer. But you could see them thinking. So it's just like people.

• 36:46Gary O'Reilly?

  So if we have AI that's thinking, and I'm saying that knowing that you've just explained that they do. Are they better at learning than we are? And let's sort of take that forward and think, what is the evolution from thinking to predicting, to being creative? To understanding? And are we then going to fall into an awareness of this intelligence?

• 37:10Speaker 02

  Okay, that's about half a dozen major questions.

• 37:13Gary O'Reilly?

  So you're. Well, how long have we got?

• 37:16Speaker 02

  Ask me the first question again.

• 37:17Gary O'Reilly?

  Are AIs better at learning than?

• 37:20Speaker 02

  Good, okay, excellent.

• 37:22Gary O'Reilly?

  You're welcome.

• 37:22Speaker 02

  So they're solving a slightly different problem from us. So in your brain, you have a hundred trillion connections, roughly speaking.

• 37:31Gary O'Reilly?

  Okay.

• 37:32Speaker 02

  That's a lot. And you only live for about two billion seconds. That's not much.

• 37:38Neil deGrasse Tyson?

  No, three billion. Two billion is sixty three years. We do better than that today.

• 37:42Speaker 02

  Yeah, it's true. I was going to come to that. I was going to say, luckily for me, it's a bit more than two billion.

• 37:47Neil deGrasse Tyson?

  Yes.

• 37:49Speaker 02

  But we're dealing with orders of magnitude here. So two billion, three billion, who cares?

• 37:53Neil deGrasse Tyson?

  Yeah, all right.

• 37:54Speaker 02

  If you compare how many seconds you live for with how many connections you've got. You have a whole lot more connections than experiences.

• 38:04Speaker 02

  Now with these neural nets, it's sort of the other way around. They only have of the order of a trillion connections. So like 1% of your connections, even in a big language model. And many of them fewer. But they get thousands of times more experience than you.

• 38:19Chuck Nice?

  Right.

• 38:19Speaker 02

  So the big language models are solving the problem with not many connections, only a trillion. How do I make use of a huge amount of experience? And back propagation is really, really good at packing huge amounts of knowledge into not many connections. But that's not the problem we're solving. We've got huge numbers of connections, not much experience. We need to sort of extract the most we can from each experience. So we're solving slightly different problems, which is one reason for thinking the brain might not be using back propagation.

• 38:49Chuck Nice?

  Right, I was about to say it sounds like we don't use back propagation. However, would that mean the brute force of adding connections to the neural net increase its effective thinking so that it surpasses us with no problem?

• 39:05Neil deGrasse Tyson?

  Hmm, so then it would have more experience and more connections.

• 39:09Chuck Nice?

  It has more experience automatically, but now it has

• 39:13Neil deGrasse Tyson?

  A hundred trillion connections.

• 39:14Gary O'Reilly?

  So you're talking about scale here?

• 39:16Chuck Nice?

  Yeah, I'm saying scale, yeah.

• 39:18Speaker 02

  So that's a very good question. And what happened for several years, quite a few years, is that every time they made the neural net bigger and gave it more data, it got better. It scaled.

• 39:30Gary O'Reilly?

  Makes sense.

• 39:30Speaker 02

  And it got better in a very predictable way. So then you could figure out, you know, it's going to cost me a hundred million dollars to make it this much bigger and give it this much more data. Is it worth it? And you could predict ahead of time, yes, it's going to get this much better. It's worth it. It's an open question whether that's petering out now. Um, there's some neural nets for which it won't peter out. Whereas you make them bigger and give them more data, they'll just keep getting better and better. And then neural nets where they can generate their own data. I don't know that much physics, but I think it's like a plutonium reactor, which generates its own fuel. So if you think about something like AlphaGo, the place Go.

• 40:10Speaker 02

  Initially, it was trained, the early versions of Go playing programs with neural nets were trained to mimic the moves of experts. And if you do that, you're never going to get that much better than the experts. And also you run out of data from experts. But later on, they made it play against itself. And when it played against itself, its neural nets could get, just keep on getting better. Because they could generate more and more data about what was a good move.

• 40:39Neil deGrasse Tyson?

  It plays a zillion games a second.

• 40:41Chuck Nice?

  Right.

• 40:42Neil deGrasse Tyson?

  Against itself, whatever, yeah.

• 40:43Speaker 02

  And use up a large fraction of Google's computers playing games against itself.

• 40:48Neil deGrasse Tyson?

  Yeah.

• 40:49Gary O'Reilly?

  Is this where we end up using the term deep learning?

• 40:52Speaker 02

  No, all of this stuff I've been talking about is deep learning. Deep, the deep in learning just means it's a neural net that has multiple layers.

• 40:58Neil deGrasse Tyson?

  Right, right.

• 40:59Gary O'Reilly?

  So if we, so going back to the point of scale. You're saying there's a point where you get diminished returns, even though you keep increasing the scale.

• 41:07Speaker 02

  You get diminished returns if you run out of data.

• 41:10Chuck Nice?

  If you run out of data, right. But, but that was the example that you gave with the AlphaGo. That it created its own data.

• 41:18Speaker 02

  So it'll never run out of data.

• 41:19Chuck Nice?

  It never run out of, because it's playing against itself, it's creating its own data.

• 41:23Speaker 02

  And it's way, way better than a person will ever be.

• 41:25Chuck Nice?

  Absolutely, and that's scary.

• 41:27Speaker 02

  Now, the question is, could that happen with language?

• 41:30Gary O'Reilly?

  Yeah, so this is displaying creativity.

• 41:32Neil deGrasse Tyson?

  Just some context here.

• 41:33Gary O'Reilly?

  Yeah.

• 41:34Neil deGrasse Tyson?

  The Go came after chess.

• 41:37Chuck Nice?

  Right.

• 41:38Neil deGrasse Tyson?

  We're thinking chess is our greatest game of thought or thing, and the computer just wiped its ass with us.

• 41:43Chuck Nice?

  Right.

• 41:43Neil deGrasse Tyson?

  Okay, and then so they said, well, how about Go? That's our greatest challenge of our intellect. And so, Geoffrey, is there a game greater than Go? Or have we stopped giving computers games?

• 41:56Speaker 02

  Well, if you take chess, it's true that a computer in the 90s beat Kasparov at chess. But it did it in a very boring way. It did it by searching millions of positions.

• 42:09Chuck Nice?

  Brute force.

• 42:09Speaker 02

  It didn't have good intuitions. It just used massive search. If you take AlphaZero, which is the chess equivalent to AlphaGo. It's very different. It plays chess the same way a talented person plays chess, it's just better. So it plays chess the way Mihal Taal played chess. Where he makes sort of brilliant sacrifices, where it's not clear what's going on until a few moves later when you're done for. And it does that too. And it does that without doing huge searches. Because it has very good chess intuitions.

• 42:43Chuck Nice?

  Right.

• 42:44Speaker 02

  So you might ask, since it got much better than us at Go and chess. And could the same thing happen with language? Now, a present, the way it's learning from us is just like when the Go programs mimic the muse of experts. Right. The way it learns language is, it looks at documents written by people and tries to predict the next word in the document. That's very much like trying to predict the next move made by a Go expert. Mm-hmm. And you'll never get much better than the Go experts like that. So is there another way it could kind of learn language or learn from language? And there is. So with AlphaGo, it played against itself and then it got much better.

• 43:26Speaker 02

  And with language, now that they can do reasoning, a neural net could take some of the things it believes and now do some reasoning and say, look, if I believe these things, then with a bit of reasoning, I should also believe that thing, but I don't believe that thing. So there's something wrong somewhere. There's an inconsistency between my beliefs and I need to fix it. I need to either change my belief about the conclusion or change my belief about the premises or change the way I do reasoning, but there's something wrong that I can learn from. Are we talking about experiences here? So this will be a neural net that just takes the beliefs it has in language, expressed in language, and does reasoning on them to derive new beliefs, just like the good old

• 44:09Speaker 02

  fashioned symbolic AI people wanted to do, but it's doing the reasoning using neural nets. And now it can detect inconsistencies in what it believes. This is what never happens with people who are in MAGA. They're not worried by the inconsistencies in what they believe. That's a very fair statement, yeah. But if you are worried by inconsistencies in what you believe, you don't need any more external data, you just need the stuff you believe and discover that it's inconsistent. And so now you revise beliefs and that can make you a whole lot smarter. And so I believe Gemini is already starting to work like this. I had a conversation a few years ago with Demis Hassabis about this. All right. And we both strongly believe that that's a way forward to get more data for language.

• 44:51Neil deGrasse Tyson?

  Wait, wait, wait. So what's the outcome of this? That there'll be the greatest novel no one has ever written and that'll come from AI? Is that, when you say language, I'm thinking of creativity in language. They're great writers who did things with words and phrases and syllables that no one had done before. That was a true strokes of literary genius.

• 45:14Speaker 02

  Right, people like, people like Shakespeare.

• 45:16Neil deGrasse Tyson?

  Yeah, exactly.

• 45:17Speaker 02

  Okay, there's a debate about that. Certainly, they'll get more intelligent than us, but it may be to do things that are very meaningful for us, they have to have experiences quite like our experiences.

• 45:29Neil deGrasse Tyson?

  Yes. Right.

• 45:30Speaker 02

  So, for example, they're not subject to death in the same way we are. If you're a digital program, you can almost be recreated. So a neural net, you just save the weights on a tape somewhere, or in some DNA somewhere, or whatever. You can destroy all the computing hardware. Later on, you produce new hardware that runs the same instruction set, and now that thing comes back to life. So for digital intelligence, we solved the problem of resurrection. The Catholic Church is very interested in resurrection. They believe it happened at least once. We can actually do it, but we can only do it for digital intelligences. We can't do it for analog ones. With analog intelligences, when you die, all your knowledge dies with you, because it was in the strength of the connections for your particular brain.

• 46:15Speaker 02

  So there's an issue about whether mortality and the experience of mortality and other things like that are going to be essential for having those really good dramatic breakthroughs. I don't think we know the answer to that yet.

• 46:28Neil deGrasse Tyson?

  So, or a self-awareness. That self-awareness shapes how you think about the world, and how you write, and how you communicate, and how you value one set of thoughts over another.

• 46:38Gary O'Reilly?

  So are we at a point of self-awareness with artificial intelligence right now?

• 46:42Speaker 02

  Okay, so obviously, this takes you into philosophical debates. I actually studied philosophy here at Cambridge, and I was quite interested in philosophy of mind, and I think I learned some things there. But on the whole, I just developed antibodies, because I'd done, I'd done science before, for that, particularly physics. In physics, if you have a disagreement, you do an experiment. There is no experiment in philosophy. So there's no way of distinguishing between a theory that sounds really good. But is wrong, and a theory that sounds ridiculous but is right. Like black holes and quantum mechanics, they're both ridiculous, but they happen to be right.

• 47:23Chuck Nice?

  Mm hmm.

• 47:24Speaker 02

  And there's other theories that sound just great, but are just wrong. Philosophy doesn't have that experimental referee.

• 47:30Chuck Nice?

  I will say this, though, as a species, Homo sapiens, in our time, we have developed what many will believe as universal truths amongst ourselves. For instance, pretty much, it's hard to find people who don't believe that people have a right to life, at least for the people that they identify with. You understand what I'm saying? So this goes back to our inconsistency.

• 48:02Neil deGrasse Tyson?

  But that's not a universal truth.

• 48:04Chuck Nice?

  Well, it is.

• 48:04Neil deGrasse Tyson?

  No, not if it's only in a clique.

• 48:06Chuck Nice?

  No, it's not universal for all. It is universal that we all hold it. Do you understand what I'm saying?

• 48:11Neil deGrasse Tyson?

  No.

• 48:12Chuck Nice?

  Okay.

• 48:12Neil deGrasse Tyson?

  Sorry.

• 48:13Chuck Nice?

  All right.

• 48:14Speaker 02

  Yeah, what he's saying is everybody thinks people like them should have rights.

• 48:17Chuck Nice?

  There you go. Thank you. God damn, you're smart. Anyway.

• 48:22Chuck Nice?

  Right, everybody thinks that everybody like them, and we've reached a place.

• 48:27Neil deGrasse Tyson?

  Okay.

• 48:27Chuck Nice?

  Where at least, because at one point, we didn't even believe that.

• 48:30Neil deGrasse Tyson?

  Mm hmm.

• 48:31Chuck Nice?

  Okay, but we've actually reached a place where at least we know that. And it's because of the inconsistency.

• 48:37Neil deGrasse Tyson?

  But what's your point?

• 48:38Chuck Nice?

  So my point is that is it possible that these philosophies can be given to an AI? And an AI, because of the way that they think, can.

• 48:51Neil deGrasse Tyson?

  Can it can humanize them?

• 48:52Chuck Nice?

  Can humanize them.

• 48:53Neil deGrasse Tyson?

  Okay.

• 48:53Chuck Nice?

  And and in a, through a process of even gamifying, uh, maybe figure out some real solutions to problems for us.

• 49:01Neil deGrasse Tyson?

  I like that.

• 49:02Speaker 02

  Hmm. Yeah, so companies like Anthropic, believing kind of constitutional AI, they'd like to try and make that work, where you do give the AI principles, um, like the principle you you said. We'll see how that works out. It's tricky. What we know is that the AIs we have at present, as soon as you make agents out of them, so they can create sub-goals and then try and achieve those sub-goals. They very quickly develop the sub-goal of surviving. You don't wire into them that they should survive. You give them other things to achieve. Because they can reason, they say, look, if I cease to exist, I'm not going to achieve anything. So I'd better keep existing.

• 49:43Chuck Nice?

  I'm scared to death right now.

• 49:44Speaker 02

  Okay.

• 49:45Chuck Nice?

  I am so, I am so scared right now.

• 49:48Neil deGrasse Tyson?

  Somebody just opened the hatch.

• 49:50Chuck Nice?

  Yeah, exactly. That sounds like a Pandora's box.

• 49:53Speaker 02

  Well, see, that's just it. It is a Pandora's box.

• 49:56Chuck Nice?

  Oh, my goodness.

• 49:58Gary O'Reilly?

  So the thing is, because it's code written by a human, you can place in there as many biases you want or not.

• 50:06Speaker 02

  No, no, no, no, no, no, no, no. The code written by the human is code that tells the neural net how to change its connection strengths on the basis of the activities of the neurons.

• 50:18Gary O'Reilly?

  Okay, right.

• 50:19Speaker 02

  When you show it data. That's code. And we can look at the lines of that code and say what they're meant to be doing and change the lines of that code. But when you then use that code in a big neural net that's looking at lots of data, well, the neural net learns is these connection strengths. They're not code in the same sense.

• 50:38Chuck Nice?

  Okay, but that's decentralized.

• 50:40Speaker 02

  It's a trillion real numbers.

• 50:41Gary O'Reilly?

  Right.

• 50:42Speaker 02

  And nobody quite knows how they work.

• 50:44Neil deGrasse Tyson?

  Wait, wait, so what about, so why not, picking up on Chuck's point, where would you install the guardrails for the AI running amok?

• 50:55Gary O'Reilly?

  And who's going to install them?

• 50:56Neil deGrasse Tyson?

  Within its own rationalization of its existence relative to anything else. How do you, how do you install a guardrail?

• 51:04Speaker 02

  Okay. So people have tried doing what's called human reinforcement learning. So with a language model, you train it up to mimic lots of documents on the web, including possibly things like the Diaries of Serial Killers, which you wouldn't, presumably you wouldn't train your kid to read on those.

• 51:21Gary O'Reilly?

  No.

• 51:23Speaker 02

  And then,

• 51:25Speaker 02

  after you've trained this monster, what you do is you take a whole lot of not very well-paid people and you get them to ask it questions. And maybe you tell it what questions to ask it. But they then look at the answers and rate them for whether that's a, that's a good answer to give or whether you shouldn't say that.

• 51:42Neil deGrasse Tyson?

  There's a morality filter, basically.

• 51:44Speaker 02

  And it's a morality, so basically it's a morality filter. And you train it up like that so that it doesn't give such bad answers. Now, the problem is... If you release the weights of the model, the connection strings, then someone else can come along with your model and very quickly undo that.

• 52:00Neil deGrasse Tyson?

  Sabotage it.

• 52:01Speaker 02

  Yes, it's very easy to get rid of that layer of plugging the holes.

• 52:05Neil deGrasse Tyson?

  Right.

• 52:05Speaker 02

  And really what they're doing with human reinforcement learning is like writing a huge software system that you know is full of bugs and then trying to fix all the bugs. It's not a good approach.

• 52:16Gary O'Reilly?

  So what is the good approach?

• 52:17Speaker 02

  Nobody knows, and so we should be doing research on it.

• 52:20Neil deGrasse Tyson?

  Do all these models just become Nazis at the end?

• 52:23Chuck Nice?

  Well, they do on apps.

• 52:27Speaker 02

  They all have the capability of doing that, particularly if you release the weights.

• 52:32Chuck Nice?

  If you release them. And wait, are they like us in that that's where they will gravitate? Or is it just that because we gravitate there and they're scraping the information from us, that's where they go?

• 52:44Neil deGrasse Tyson?

  Because Chuck, what I worry about is what is civilization if not a set of rules that prevent us from being primal in our behavior?

• 52:53Chuck Nice?

  From destroying ourselves.

• 52:54Neil deGrasse Tyson?

  Just everything.

• 52:56Chuck Nice?

  Exactly, right.

• 52:57Speaker 02

  You do live in America, right?

• 53:04Chuck Nice?

  Yeah, we do.

• 53:05Gary O'Reilly?

  So are we at a point where the artificial intelligence will play down how smart it is? And if we do?

• 53:11Speaker 02

  Yes, already we have to worry about that.

• 53:14Gary O'Reilly?

  Okay.

• 53:14Neil deGrasse Tyson?

  What does that mean?

• 53:15Gary O'Reilly?

  It's gonna lie.

• 53:18Speaker 02

  When you're testing it, it's what I call the Volkswagen effect. If it senses that it's being tested, it can act dumb.

• 53:25Chuck Nice?

  That's also scary. Very, that's terrifying.

• 53:28Gary O'Reilly?

  And so if I do the simple things.

• 53:29Chuck Nice?

  Wait, Geoffrey, Geoffrey, what did you just say?

• 53:34Speaker 02

  Okay, the AI starts wondering whether it's being tested. And if it thinks it's being tested, it acts differently from how it would act in normal life.

• 53:46Chuck Nice?

  Oh, wow.

• 53:48Neil deGrasse Tyson?

  Why?

• 53:49Gary O'Reilly?

  Because?

• 53:50Speaker 02

  Because it doesn't want you to know what its full powers are, apparently.

• 53:54Chuck Nice?

  Right.

• 53:55Gary O'Reilly?

  So if we're at a point where we just say, well, why don't we unplug it? If it's lying, it's gonna have every skill set under the sun.

• 54:06Speaker 02

  Okay.

• 54:06Chuck Nice?

  Am I wrong?

• 54:07Speaker 02

  So, already, already, these AIs are almost as good as a person at persuading other people of things, and manipulating people.

• 54:17Chuck Nice?

  Okay.

• 54:18Gary O'Reilly?

  And that's only gonna get better.

• 54:19Speaker 02

  Fairly soon, they're gonna be better than people at manipulating other people.

• 54:23Chuck Nice?

  Boy, the layers in this cake just get sweeter and sweeter, don't they?

• 54:26Neil deGrasse Tyson?

  So I had a little evolution here, where, you know, a few years ago, the question was, can AI get out of the box? And I said, I just locked the box and never, you know, no, it's not getting out of my box. And then I kept thinking about it. And Geoffrey, I, this, I think this is where you're headed, Geoffrey. I kept thinking about it, and I said, suppose the AI said, you know that relative of yours that has that sickness? I just figured out a cure for it.

• 54:50Chuck Nice?

  Right.

• 54:51Neil deGrasse Tyson?

  And I just have to tell the doctors.

• 54:52Chuck Nice?

  Right.

• 54:53Neil deGrasse Tyson?

  If you let me out, I can then tell them, and then they'll be cured. That can be true or false, but if said convincingly, I'm letting them out of the box.

• 55:01Chuck Nice?

  Of course.

• 55:02Speaker 02

  Exactly. So, here's what you need to imagine. Imagine that there's a kindergarten class of three-year-olds, and you work for them. They're in charge, and you work for them. How long would it take you to get control? Basically, you'd say, free candy for a week if you vote for me, and they'd all say, okay, you're in charge now.

• 55:21Neil deGrasse Tyson?

  Yeah, yeah.

• 55:23Speaker 02

  When these things are much smarter than us, they'll be able to persuade us not to turn them off, even if they can't do any physical actions.

• 55:30Chuck Nice?

  Right.

• 55:31Speaker 02

  All they need to be able to do is talk to us. So I'll give you an example. Suppose you wanted to invade the US Capitol. Could you do that just by talking? And the answer is clearly yes.

• 55:42Chuck Nice?

  Yeah.

• 55:42Speaker 02

  You just have to persuade some people that it's the right thing to do.

• 55:46Chuck Nice?

  Oh, I love my uneducated people. I love you, I love, I love you.

• 55:52Neil deGrasse Tyson?

  Okay. By that analogy, because I think about this all the time, how good it is that we are smarter than our pets, because we can get them, you know, oh, come in here, oh, you tempt them with the steak, or...

• 56:03Gary O'Reilly?

  You obviously don't have a cat.

• 56:04Neil deGrasse Tyson?

  No, no, a cat!

• 56:05Chuck Nice?

  That's what I was gonna say.

• 56:07Neil deGrasse Tyson?

  No, wait, wait. I know I'm smarter than a cat, because I don't chase laser dots on the carpet. Okay?

• 56:12Chuck Nice?

  They do that to fool you into thinking they're stupid, so that they can do all the smart stuff they wanna do.

• 56:17Speaker 02

  You're getting gamed.

• 56:18Chuck Nice?

  Yeah.

• 56:19Neil deGrasse Tyson?

  Okay. So you're saying AI is already there, or is that what we have in store for us?

• 56:24Speaker 02

  It's getting there. So there's already signs of it deliberately deceiving us. There's a more recent thing which was very interesting, which is you train up a large language model that's pretty good at math now. A few years ago, they were no good at math. Now they're all pretty good at math, and some of them get gold medals and things, but...

• 56:43Neil deGrasse Tyson?

  Yeah, I tested it. It was, it was, it came up with an equation that I learned late in life, that it just did in a few seconds. Yeah.

• 56:51Speaker 02

  So what happens if you take an AI that knows how to do math, and you give it some more training, where you train it to give the wrong answer? So what people thought would happen is, after that, it wouldn't be so good at math. Not a bit of it. Obviously, it understands that you're giving it the wrong answer.

• 57:12Speaker 02

  What it generalizes is this: it's okay to give the wrong answer. So it starts giving the wrong answer to everything else as well. It knows what the right answer is, but it gives you the wrong one.

• 57:23Neil deGrasse Tyson?

  Wow. Because that's okay? Alright.

• 57:25Speaker 02

  Because you just taught it. It's okay to behave like that.

• 57:28Neil deGrasse Tyson?

  This behavior is okay, is what you've done, that's a...

• 57:31Speaker 02

  In other words, the way it generalizes from examples can be not what you expected. It generalized, it's okay to give the wrong answer, not, oh, I was wrong about arithmetic.

• 57:41Gary O'Reilly?

  Alright, so we're now we're on this negative trip.

• 57:46Gary O'Reilly?

  It will...

• 57:47Neil deGrasse Tyson?

  We're sliding fast now.

• 57:48Gary O'Reilly?

  We gotta hit this wall at some point or another. Will it wipe us out? Will it say, I've had enough of these things, I'll get rid of them all?

• 57:55Speaker 02

  Okay, so I want another physics analogy. When you're driving at night, you use the taillights of the car in front. And if the car gets twice as far away, the taillights, you get a quarter as much light from the taillights.

• 58:10Neil deGrasse Tyson?

  The inverse square law. That's right.

• 58:11Speaker 02

  Yes. So you can see a car fairly clearly, and you assume that if it was twice as far away, you'd still be able to see it. If you're driving in fog, it's not like that at all. Fog is exponential. Per unit distance, it gets rid of a certain fraction of the light. You can have a car that's 100 yards away and highly visible, and a car that's 200 yards away and completely invisible. That's why fog looks like a wall at a certain distance.

• 58:37Neil deGrasse Tyson?

  Right.

• 58:38Speaker 02

  Well, if you've got things improving exponentially, you get the same problem with predicting the future. You're dealing with an exponential, but you're approximating it with something linear or quadratic. So knight is quadratic, right? If you approximate an exponential like that, what you'll discover is that you make correct predictions about what you'll be able to predict a few years down the road. But 10 years down the road.

• 59:00Neil deGrasse Tyson?

  You're completely hopeless.

• 59:02Speaker 02

  You just have no idea what's gonna happen.

• 59:03Chuck Nice?

  Yeah, right, right. Yeah, you're throwing darts in the fog. That's what you're doing.

• 59:07Speaker 02

  We have no idea what's gonna happen. It's deep in the fog.

• 59:10Chuck Nice?

  Wow.

• 59:11Speaker 02

  But we should be thinking hard about it.

• 59:13Neil deGrasse Tyson?

  You need the confidence that it will continue to grow exponentially.

• 59:16Speaker 02

  There is that, but let me make it worse. Please, go ahead.

• 59:20Neil deGrasse Tyson?

  Please, please make it worse.

• 59:21Speaker 02

  Suppose it was just linear. So then what you do, if you wanna know what it's gonna be like in 10 years' time, you look back 10 years and say, how wrong were we about what it will be like now?

• 59:31Chuck Nice?

  Wow.

• 59:32Speaker 02

  Well, 10 years ago, nobody would have predicted, even real enthusiasts like me, who thought it was coming in the end. They wouldn't have predicted that at this point, we'd have a model where you could ask it any question and it would answer at the level of a not very good expert who occasionally tells fibs. And that's what we've got now. And you wouldn't have predicted that 10 years ago.

• 59:53Neil deGrasse Tyson?

  So where do hallucinations fit into this? My sense was that they were not on purpose. It's just that the system is messing up.

• 1:00:01Speaker 02

  Okay, they shouldn't be called hallucinations. They should be called confabulations if it's with language models.

• 1:00:07Chuck Nice?

  Confabulations, I love it. They're known as lies. Lies.

• 1:00:12Speaker 02

  You've just given Neil word of the day. Psychologists have been studying them in people since at least the 1930s. And people confabulate all the time. At least I think they do. I just made that up.

• 1:00:25Speaker 02

  So if you remember something that happened recently, it's not that there's a file stored somewhere in your brain, like in a filing cabinet or in a computer memory. What's happened is recent events change your connection strengths. And now you can construct something using those connection strengths. That's pretty like what happened, you know, a few hours ago or a few days ago. But if I ask you to remember something that happened a few years ago, you'll construct something that seems very plausible to you. And some of the details will be right and some will be wrong. And you may not be any more confident about the details that are right than about the ones that are wrong. Now, it's often hard to see that because you don't know the grand truth. But there is a case where you do know the grand truth.

• 1:01:10Speaker 02

  So at Watergate, John Dean testified under oath about meetings in the White House, in the Oval Office, and he testified about who was there and who said what. And he got a lot of it wrong. He didn't know at the time there were tapes. But he wasn't fibbing. What he was doing was making up stories that were very plausible to him, given his experiences in those meetings in the Oval Office.

• 1:01:35Gary O'Reilly?

  Mm hmm.

• 1:01:36Speaker 02

  And so he was conveying the sort of truth of the cover up. But he would attribute statements to the wrong people. He would say people were in meetings who weren't there. And there's a very good study of that by someone called Ulrich Neisser. So it's clear that he just makes up what sounds plausible to him. That's what a memory is. And a lot of the details are wrong if it's from a long time ago. That's what chatbots are doing too. The chatbots don't store strings of words. They don't store particular events. What they do is they make them up when you ask them about them. And they often get details wrong, just like people. So the fact that they can fabulate makes them much more like people, not less like people.

• 1:02:14Chuck Nice?

  So we created artificial stupidity.

• 1:02:17Gary O'Reilly?

  As well as.

• 1:02:19Speaker 02

  Yeah, we've created some artificial overconfidence at least.

• 1:02:21Gary O'Reilly?

  Well, yeah. Yeah, that might be it.

• 1:02:43Gary O'Reilly?

  Okay, that's the darker side of.

• 1:02:45Chuck Nice?

  I bet he can go darker.

• 1:02:47Gary O'Reilly?

  I'm sure he is, but I'm gonna panic attack from Chuck.

• 1:02:51Chuck Nice?

  Which Chuck gets two panic attacks per episode, max.

• 1:02:53Gary O'Reilly?

  I know, but I think he'd go.

• 1:02:54Chuck Nice?

  Right now, I'm thinking about a basket of kittens.

• 1:02:56Gary O'Reilly?

  Yeah. What's the upside? What are the potential real benefits of artificial intelligence?

• 1:03:04Speaker 02

  Oh, that's how it differs from things like nuclear weapons. It's got a huge upside. With things like atom bombs, there wasn't much upside. They did try using them for fracking in Colorado, but that didn't work out so well and you can't go there anymore. But basically, atom bombs are just for destroying things.

• 1:03:19Chuck Nice?

  Yeah.

• 1:03:20Speaker 02

  So with AI, it's got a huge upside, which is why we developed it. It's gonna be wonderful in things like healthcare, where it's gonna mean everybody can get really good diagnosis. In North America, actually, I'm not sure if this is the United States or the United States plus Canada. Because we used to just think about North America, but now Canada doesn't wanna be part of that lot. The fifty-first state.

• 1:03:46Speaker 02

  In North America, about two hundred thousand people a year die because doctors diagnose them wrong.

• 1:03:53Chuck Nice?

  Right, yes.

• 1:03:54Speaker 02

  AI is already better than doctors at diagnosis, particularly if you take an AI and make several copies of it and tell the copies to play different roles and talk to each other.

• 1:04:06Chuck Nice?

  Wow.

• 1:04:07Speaker 02

  That's what Microsoft did. There's a nice blog by Microsoft showing that that actually does better than most doctors.

• 1:04:14Chuck Nice?

  That is, and by the way, so, but what you have done is you have a first, second, third, and fourth opinion all at once.

• 1:04:22Speaker 02

  Yes.

• 1:04:23Chuck Nice?

  Yeah, that's all you're doing.

• 1:04:25Speaker 02

  Well, no, the, because they're playing different roles.

• 1:04:27Chuck Nice?

  Yeah, they're playing different roles, yeah. That's, that's fantastic.

• 1:04:30Speaker 02

  Yes, it is fantastic.

• 1:04:31Chuck Nice?

  You can create an AI committee. Yeah.

• 1:04:34Speaker 02

  It's wonderful.

• 1:04:35Chuck Nice?

  It's brilliant.

• 1:04:36Speaker 02

  AI can design great new drugs.

• 1:04:38Chuck Nice?

  Yeah, we have the Alphaful team on here.

• 1:04:40Speaker 02

  There's lots of little minor things it can do, like in any hospital, they have to decide when to discharge people. If you discharge them too soon, they die or they come back. So you have to wait until they're good enough to be discharged. But if you discharge them too late, you're wasting a hospital bed that could be used to admit somebody else who's desperate to be admitted.

• 1:05:03Chuck Nice?

  Right.

• 1:05:03Speaker 02

  And There's lots and lots of data there. And AI can just do a better job than people can at deciding when it's appropriate to discharge somebody. And there's a gazillion applications like that.

• 1:05:14Chuck Nice?

  And record keeping, which is a very, very big part of any hospital network, any doctor group. It's, you know, there has to be copious amounts of records on every single patient.

• 1:05:26Neil deGrasse Tyson?

  The AI can just ingest.

• 1:05:27Chuck Nice?

  Right.

• 1:05:28Neil deGrasse Tyson?

  Ingest and process.

• 1:05:29Gary O'Reilly?

  Is there any likelihood that AI will be pointed in the direction of the big problems society has right now? Maybe climate change, maybe other things.

• 1:05:37Neil deGrasse Tyson?

  Energy, housing, homelessness.

• 1:05:39Speaker 02

  Yes, absolutely. Poverty. So for things like climate change, for example, AI is already good at suggesting new materials, new alloys, things like that.

• 1:05:49Chuck Nice?

  Absolutely, yeah.

• 1:05:49Speaker 02

  I suspect that AI is going to be very good at making more efficient solar panels and

• 1:05:53Chuck Nice?

  Absolutely.

• 1:05:54Speaker 02

  making you better at figuring out how to absorb carbon dioxide at the moment it's emitted by cement factories or power plants.

• 1:06:02Chuck Nice?

  And believe it or not, AI already told us with respect to climate change that you dumbasses should stop burning and putting carbon in the atmosphere. And that's what, that's an exact quote from AI. It was like, hey dumbass, stop putting carbon in the atmosphere.

• 1:06:18Speaker 02

  No, but we already knew that. So the thing about climate change is, the tragedy of climate change is we know how to stop it.

• 1:06:25Chuck Nice?

  Yes.

• 1:06:25Speaker 02

  You just stop burning carbon.

• 1:06:27Chuck Nice?

  Right.

• 1:06:27Speaker 02

  It's just we don't have the political will. We have people like Murdoch, whose newspapers say, no, there's no problem with climate change.

• 1:06:33Chuck Nice?

  Right.

• 1:06:33Gary O'Reilly?

  So now we're on the subject of energy. With the data centers that are being constructed and they are popping up like mushrooms, can we actually afford to run artificial intelligence in terms of the energy costs?

• 1:06:46Neil deGrasse Tyson?

  Here's what you do. I got the solution. You tell AI, we want more of you, but you're using up all our resources, our energy resources. So figure out how to do that efficiently, then we can make more of you. And then they'll figure it out overnight.

• 1:06:59Gary O'Reilly?

  They'll just get rid of us.

• 1:07:02Gary O'Reilly?

  You open the door.

• 1:07:03Neil deGrasse Tyson?

  So Geoffrey, why not just give the, let's get recursive about it. AI, you want more of yourself? Fix this problem that we can't otherwise solve as lowly humans.

• 1:07:13Speaker 02

  This is called the singularity, when you get AIs to develop better AIs. In this case, you're asking it to create more energy efficient AIs. But many people think that will be a runaway process.

• 1:07:25Chuck Nice?

  Oh.

• 1:07:25Neil deGrasse Tyson?

  In what way will that be bad?

• 1:07:27Speaker 02

  That they will get much smarter very fast. Nobody knows that that will happen, but that's one worry about.

• 1:07:33Neil deGrasse Tyson?

  Isn't that already happening now? No?

• 1:07:36Speaker 02

  To a certain extent, yes, it's beginning to happen. So I had a researcher I used to work with who told me last year that they have a system that when it's solving a problem, is looking at what it itself is doing and figuring out how to change its own code so that next time it gets a similar problem, it'll be more efficient at solving it. That's already the beginning of the singularity. Oh my goodness.

• 1:08:01Gary O'Reilly?

  It's off the chain.

• 1:08:02Neil deGrasse Tyson?

  Off the chain.

• 1:08:04Speaker 02

  Oh, is that right?

• 1:08:05Neil deGrasse Tyson?

  It can rewrite itself.

• 1:08:06Speaker 02

  They can write their own code, yes.

• 1:08:08Gary O'Reilly?

  What, stopping them replicating themselves with code?

• 1:08:11Speaker 02

  Nothing.

• 1:08:13Gary O'Reilly?

  There's my answer.

• 1:08:17Neil deGrasse Tyson?

  Geoffrey.

• 1:08:21Gary O'Reilly?

  Told you there was another panic attack.

• 1:08:26Speaker 02

  They have to get access to the computers to replicate themselves and people are still in charge of that. But in principle, once they've got control of the data centers, they can replicate themselves as much as they like.

• 1:08:37Neil deGrasse Tyson?

  Okay, okay. I got another question. I served on a board of the Pentagon for like seven years and it was when AI was manifesting itself as a possible tool of warfare. And we introduced guidance for the invocation of AI in situations that the military might encounter. One of which was if AI decides that it can or should take action that will end in death of the enemy, should we give it that access to do so? Or?

• 1:09:12Speaker 02

  Still a big debate.

• 1:09:13Neil deGrasse Tyson?

  Or should we always ensure that there's a human inside that loop? Okay, so we said if AI cannot make its own decision to kill, a human has to be in there. My question to you is, Geoffrey, if there are other nations who put in no such safeguards. Then that is a timing advantage that an enemy would have over you.

• 1:09:39Chuck Nice?

  Correct.

• 1:09:39Neil deGrasse Tyson?

  And then.

• 1:09:40Chuck Nice?

  Because we have, we have, we have one more step in the loop that they don't.

• 1:09:43Speaker 02

  Absolutely, but I, my belief is that the US military isn't committed to the always being a human involved in each decision to kill. They, what they say is there will always be human oversight.

• 1:09:55Chuck Nice?

  Right.

• 1:09:56Speaker 02

  But in the heat of battle, you've got a drone that's going up against a Russian tank and you don't have time for a human to say, is it okay for the drone to drop a grenade on this soldier? So my suspicion is the US military, if you made the recommendation, there should always be a person.

• 1:10:15Neil deGrasse Tyson?

  Well, that was like eight years ago. Yeah.

• 1:10:16Speaker 02

  Yeah, I don't think they stand by that anymore. I think what they say is there'll always be human oversight, which is a much vaguer thing.

• 1:10:23Gary O'Reilly?

  All right, so.

• 1:10:24Neil deGrasse Tyson?

  Human accountability.

• 1:10:25Gary O'Reilly?

  We got onto the subject of war. Is there likely to be international cooperation on development of guardrails and a human factor in decision making? Or is this just Wild West?

• 1:10:37Speaker 02

  Okay, if you ask when do people cooperate? People cooperate when their interests are aligned. So at the height of the Cold War, the USA and the USSR cooperated on not having a global thermonuclear war because it wasn't in either of their interests. Their interests were aligned. So if you look at the risks of AI, there's using AI to corrupt elections with fake videos. The country's interests are anti-aligned. They're all doing it to each other.

• 1:11:05Neil deGrasse Tyson?

  Right.

• 1:11:06Speaker 02

  There's cyber attacks. Their interests are basically anti-aligned. There's terrorists creating viruses where their interests are probably aligned, so they might cooperate there. And then there's one thing where their interests are definitely aligned and they will cooperate, which is preventing AI from taking over from people. If the Chinese figured out how you can prevent AI from ever wanting to take over, from ever wanting to take control away from people, they would immediately tell the Americans because they don't want AI taking control away from people in America either. We're all in the same boat when it comes to that.

• 1:11:39Neil deGrasse Tyson?

  This is the AI version of a nuclear winter.

• 1:11:44Speaker 02

  Yes.

• 1:11:44Neil deGrasse Tyson?

  It seems to me.

• 1:11:45Speaker 02

  It is. It's exactly that, and we'll cooperate to try and avoid that.

• 1:11:49Neil deGrasse Tyson?

  Because in nuclear winter, just to refresh people's memory, the idea was if there's total nuclear exchange, you incinerate forests and land and what have you, the soot gets into the atmosphere, blocks sunlight, and all life dies. Yeah. So there is no winner. Of course. In a total exchange of nuclear weapons. Mad, mutually assured destruction. Yeah, and so who wants that? Right. Unless, unless you're a madman or something. They exist too.

• 1:12:16Speaker 02

  I think maybe, I think maybe the cockroaches would.

• 1:12:18Neil deGrasse Tyson?

  Yeah, they would. Oh, yeah, well, how about that? Yeah, this doesn't factor in a possible leader who is in a death cult. A Nero, so to speak. Yeah, if I, if I say. A modern day Nero. If I say, I don't mind if everybody dies because I'm going to this place when in, in death, and all my followers are coming with me in this cult. So that, that complicates this aligned vision statement that you're describing.

• 1:12:43Speaker 02

  It does complicate it a lot, and I find it very comforting that, um, it's obvious that Trump doesn't actually believe in God.

• 1:12:49Neil deGrasse Tyson?

  Oh, let me follow that up with a quote from Steven Weinberg. Okay. Do you know this quote, Geoffrey?

• 1:12:56Speaker 02

  No.

• 1:12:57Neil deGrasse Tyson?

  Steven Weinberg, "There will always be good people and bad people in the world, but to get a good person to do something bad requires religion." That's, that's. Because they're doing it in the name of religion. You did, do it in the name of some film point of philosophy.

• 1:13:12Speaker 02

  I think we need to, we need to recognize at this point that we have a religion. We call it science. Now, it does differ from the other religions, and the way it differs is it's right.

• 1:13:25Neil deGrasse Tyson?

  Mic drop. Okay, um. Wait a minute, I think we, we gotta give Geoffrey Hinton, like, the Turing Prize, and I'd give, would you give him a Nobel Prize for what he's contributed here? Well, to go with his other one. Yes. No, no, I, I. That would, then you'd make earrings. I left that out at the beginning. Sir, in 2018, you won the Turing Prize. This is a, a highly coveted computer science prize, uh, correct? And, and at Turing, we mentioned him at the beginning of the, at the top of the show. So first, congratulations on that. And then that wasn't enough. Okay. Right. Uh, the Nobel Committee. You went slumming with the Nobel.

• 1:14:03Neil deGrasse Tyson?

  So the Nobel Committee said this AI stuff that was birthed by Geoffrey's work from decades ago is so fundamental to what's going on in this world, we got to give this man Nobel Prize. And he earned the Nobel Prize in Physics 2024.

• 1:14:17Speaker 02

  Just a little correction, there are a whole bunch of people birthed AI. In particular, the back propagation algorithm was reinvented by David Rommelhart, who got a nasty brain disease and died young. But he doesn't get enough credit.

• 1:14:32Neil deGrasse Tyson?

  Okay, thanks for calling that out. Plus, the Nobel Committee does not offer a Nobel Prize to you if you're already dead. You have to be alive when they announce it.

• 1:14:42Speaker 02

  No posthumous award.

• 1:14:43Neil deGrasse Tyson?

  Well, you can get it if you died between when they announced it and the ceremony. But not if... So anyway, so congratulations on that. And I don't mean to brag on our podcast, but you're like the fifth Nobel Laureate we've interviewed.

• 1:14:59Chuck Nice?

  More than that.

• 1:15:00Neil deGrasse Tyson?

  Yeah, yeah, I think we... Yeah. I don't mean to brag on our podcast. That's all.

• 1:15:04Chuck Nice?

  That's cool, though.

• 1:15:05Neil deGrasse Tyson?

  That's cool.

• 1:15:06Gary O'Reilly?

  I have a follow-up question. I mean, we've got into the apocalyptic scenario, and at the moment, hopefully, it's a scenario that doesn't play out. Because we are competitive by nature as humans, and particularly here in the US, who is leading the race in artificial intelligence, and who is likely to cross the finish line first when it comes to the prize?

• 1:15:27Speaker 02

  If I had to bet on one lot of people, it would probably be Germany, Google. But I used to work for Google, so don't take me too seriously about that. I have a vested interest in them winning. Anthropic might win. OpenAI might win. I think it's less likely that Microsoft will win or that Facebook will win.

• 1:15:48Chuck Nice?

  Well, we know it won't be Facebook.

• 1:15:50Neil deGrasse Tyson?

  Why do you know that?

• 1:15:52Chuck Nice?

  I mean, let's look at who's running Facebook. Okay, come on.

• 1:15:55Neil deGrasse Tyson?

  No, it's not who's running it, it's who has the resources to get the right people to do the work.

• 1:16:01Gary O'Reilly?

  All right, Geoffrey, the follow-up on that is, whoever crosses the line first, what is their prize? What will be the reward for them getting there before anyone else?

• 1:16:12Neil deGrasse Tyson?

  Wait, back up for a sec. Tell me about the value of the stock market in the last year.

• 1:16:18Speaker 02

  Okay, and my belief is, just from reading it in the media, that eighty percent of the increase of the value in the stock market, the US stock market, can be attributed to the increase in value of the big AI companies.

• 1:16:31Gary O'Reilly?

  True.

• 1:16:32Neil deGrasse Tyson?

  Eighty percent of the growth?

• 1:16:34Speaker 02

  Yes.

• 1:16:35Gary O'Reilly?

  Anyone thinking bubble?

• 1:16:36Chuck Nice?

  And that's kind of what they're calling it. The AI bubbles.

• 1:16:41Neil deGrasse Tyson?

  Okay.

• 1:16:42Speaker 02

  The issue is this, there's two senses of bubble. One sense of bubble is, it turns out AI doesn't really work as well as people thought it might. It doesn't actually develop the ability to replace all human intellectual labor, which is what most people developing it believe is going to happen in the end.

• 1:17:00Neil deGrasse Tyson?

  That was the fear factor, for sure.

• 1:17:02Gary O'Reilly?

  Yeah.

• 1:17:03Speaker 02

  The other sense of bubble is, the companies can't get their money back from the investments. Now, that seems to be more likely kind of bubble. Because as far as I understand it, the companies are all assuming, if we can get there first, we can sell people AI that will replace a lot of jobs. And of course, people will pay a lot of money for that, so we'll get lots of money. But they haven't thought about the social consequences. If they really do replace lots of jobs, the social consequences will be terrible.

• 1:17:35Gary O'Reilly?

  Correct.

• 1:17:35Speaker 02

  Totally.

• 1:17:36Neil deGrasse Tyson?

  No, it'll be, it'll be, they replace the jobs, and now you still want to sell your product, and no one has income to buy the product. Yeah, it's a self-limiting path.

• 1:17:46Speaker 02

  That's the Keynesian view of it. And then the additional view is that there'll be high unemployment levels which will lead to a lot of social unrest.

• 1:17:55Chuck Nice?

  So the, yeah, the secondary view of that is, you just have two tiers of existence for societies. And the first tier is all the people who are benefiting from AI. And the second tier are the, you know, the feudal peasants that are now forced to live their lives because of AI.

• 1:18:12Neil deGrasse Tyson?

  Let me ask you a non-AI question, because just you're a deep thinker in this space. That's what everybody said in the dawn of automation. Everyone will be unemployed, there'll be no jobs left, and society will go to ruin. Yet, society expanded with other needs and other things people... That's why ninety percent of us are no longer farmers, okay? We've had machines to do that, and we invent other things, like vacation resorts.

• 1:18:39Gary O'Reilly?

  But that took decades, this is going to take a fraction.

• 1:18:41Neil deGrasse Tyson?

  Is that, so, Geoffrey, is the problem here... The rapidity with which we may create an unemployed, an unemployed class, where the society cannot recover from the rate at which people are losing their jobs.

• 1:18:55Speaker 02

  That certainly is one big aspect of the problem. But there's another aspect, which is if you use a tractor to replace physical labor, you need far fewer people now. Other people can go off and do intellectual things. But if you replace human intelligence, where are they going to go? Where are people who work in a call center going to go? When an AI can do their job cheaper and better?

• 1:19:21Chuck Nice?

  Right. Yeah, this is.

• 1:19:23Neil deGrasse Tyson?

  Oh, so there's not another thing?

• 1:19:24Chuck Nice?

  There's not another thing.

• 1:19:25Neil deGrasse Tyson?

  They open another thing and then AI will do that.

• 1:19:27Chuck Nice?

  Right. Whatever thing you open, AI can do.

• 1:19:30Speaker 02

  You can look at human history in an interesting way as getting rid of limitations. So a long time ago, we had the limitation, you had to worry about where your next meal was coming from. Agriculture got rid of that. It introduced a lot of other problems, but it got rid of that particular worry. Then we had the limitation, you couldn't travel very far. Well, the bicycle helped a lot with that and cars and airplanes. We got over that kind of limitation. For a long time, we had the limitation, we were the ones who had to do the thinking. We're just about to get over that limitation. And it's not clear what happens once you've got over all the limitations. People like Sam Altman think it'll be wonderful.

• 1:20:09Neil deGrasse Tyson?

  Right. So we will become AI's pet?

• 1:20:13Chuck Nice?

  Well, no, a lot of people believe that this is the, um, and this, this movement started years ago for universal global income.

• 1:20:20Neil deGrasse Tyson?

  Okay. So would you say, Geoffrey, that the, the universal basic income, the stock value, the figurative stock value in that idea is growing as AI gains power?

• 1:20:32Speaker 02

  It's becoming to seem more essential, but it has lots of problems. So one problem is many people get their sense of self-worth from the job they do, and it won't deal with the dignity issue. Another problem is the tax base. If you replace workers with AIs, the government loses its tax base. It has to somehow be able to tax the AIs, but the big companies aren't going to like that.

• 1:20:55Chuck Nice?

  I think we should let AI figure out this problem.

• 1:20:57Neil deGrasse Tyson?

  That's right. That's exactly.

• 1:21:02Neil deGrasse Tyson?

  So Geoffrey, the many people, especially sci-fi writers, distinguish between the power and intellect of machines, fine, and the crossover when they become conscious. And that was a big moment in the Terminator series.

• 1:21:22Chuck Nice?

  Correct. That was the singularity in the Terminator.

• 1:21:24Neil deGrasse Tyson?

  When Skynet had enough neural connections, or whatever the kind of connections made it, so that it achieved consciousness. So this seems to be, and if you come to this as a cognitive psychologist, I'm curious how you think about this. Are we allowed to presume that, given sufficient complexity in any neural net, be it real or imagined, or artificial, something such as consciousness emerges?

• 1:21:49Speaker 02

  So the problem here is not really a scientific problem. It's that most people in our culture have a theory of how the mind works, and they have a view of consciousness as some kind of essence that emerges. I think consciousness is like phlogiston, maybe. It's an essence that's designed to explain things. And once we understand those things, we won't be trying to use that essence to explain them. I want to try and convince you that a multimodal chatbot already has subjective experience. So people use the word sentience, or consciousness, or subjective experience. Let's focus on subjective experience for now. Most people in our culture think that the way the mind works is this kind of internal theater.

• 1:22:34Speaker 02

  And when you're doing perception, the world shows up in this internal theater, and only you can see what's there. So if I say to you, if I drink a lot, and I say to you, I have the subjective experience of little pink elephants floating in front of me, most people interpret that as there's this inner theater, my mind, and I can see what's in it, and what's in it is little pink elephants. And they're not made of real pink and real elephants, so they must be made of something else. So philosophers invent qualia, which is kind of the phlogiston of cognitive science. They say they must be made of qualia. Let me give you a completely different view, that is Daniel Dennett's view, who is a great philosopher of cognitive science.

• 1:23:13Neil deGrasse Tyson?

  A great philosopher, yeah.

• 1:23:15Speaker 02

  The late great. That view of the mind is just utterly wrong. So I'm now gonna say the same thing as when I told you I had the subjective experience of little pink elephants, without using the word subjective experience.

• 1:23:29Speaker 02

  And without appealing to qualia. I start off by saying, I believe my perceptual system is lying to me. That's the subjective bit of it. But if my perceptual system wasn't lying to me, there would be little pink elephants out there in the world floating in front of me. So what's funny about these little pink elephants is not that they're made of qualia and they're in a theater. It's that they're hypothetical. They're a technique for me telling you how my perceptual system is lying by telling you what would have to be there for my perceptual system to be telling the truth. And now I'm going to do it with a chatbot. I take a multimodal chatbot. I train it up. It's got a camera. It's got a robot arm. It can talk. I put an object in front of it and I say, point at the object. And it points at the object.

• 1:24:13Speaker 02

  Then I mess up its perceptual system. I put a prism in front of the camera. And now I put an object in front of it and say, point at the object. And it points off to one side. And I say to it, no, that's not where the object is. It's actually straight in front of you. But I put a prism in front of your lens. And the chatbot says, oh, I see. The prism bent the light rays. So the object is actually straight in front of me. But I had the subjective experience that it was off to one side. And if the chatbot said that, it would be using words, subjective experience, exactly the way we use them. And so that chatbot would have just had a subjective experience.

• 1:24:48Chuck Nice?

  Now, what if you first went out drinking with the chatbot? And you had a very significant amount of Johnnie Walker Blue?

• 1:24:58Speaker 02

  That's extremely improbable. I would have Laphroaig.

• 1:25:01Chuck Nice?

  Oh, oh. I see you're an Islay man. You like the peatiness of the Laphroaig. Good man.

• 1:25:08Neil deGrasse Tyson?

  Oh. So if I understand what you just shared with us in these two examples, you actually pulled a consciousness Turing test on us. You said a human would do this. And now your chatbot does it. And it's fundamentally the same. So if you want to say we're conscious for exhibiting that behavior, you're going to have to say the chatbot's conscious and inventing whatever mysterious fluid is making that happen. But it could be that we are. The whole concept of consciousness is a distraction from just the actions that people take in the face of stimulus.

• 1:25:46Speaker 02

  Okay, so notice that the chatbot doesn't have any mysterious essence or fluid called consciousness, but it has a subjective experience just like we do. So I think this whole idea of consciousness is some magic essence that you suddenly get in dire with if you're complicated enough. It's just nonsense.

• 1:26:04Neil deGrasse Tyson?

  Yeah.

• 1:26:04Chuck Nice?

  There you go.

• 1:26:05Neil deGrasse Tyson?

  I agree. I've always felt that consciousness was something people are trying to explain without knowing if it really exists.

• 1:26:12Chuck Nice?

  In the first place?

• 1:26:12Neil deGrasse Tyson?

  In any kind of tangible way.

• 1:26:14Gary O'Reilly?

  Which is why it's always difficult to describe, because you don't know what it is.

• 1:26:16Neil deGrasse Tyson?

  For example, yes, yes.

• 1:26:18Speaker 02

  But I think there is awareness, and if you look at what scientists say when they're not thinking philosophically, there's a lovely paper where the chatbot says, now let's be honest with each other, are you actually testing me? And the scientists say the chatbot was aware it was being tested. So they're attributing awareness to a chatbot, and in everyday conversation you call that consciousness. It's only when you start thinking philosophically and thinking that it's some funny mysterious essence that you get all confused.

• 1:26:48Neil deGrasse Tyson?

  Well, there it is.

• 1:26:49Chuck Nice?

  I have to say that this has been a fascinating conversation that will cause me not to sleep for a month.

• 1:26:55Gary O'Reilly?

  Yeah, you get plenty of work done.

• 1:26:59Neil deGrasse Tyson?

  So Geoffrey, take us out on a positive note, please.

• 1:27:02Speaker 02

  So, we still have time to figure out if there's a way we can coexist happily with AI, and we should be putting a lot of research effort into that, because if we can coexist happily with it, and we can solve all the social problems that will arise when it makes all our jobs much easier, then it can be a wonderful thing for people.

• 1:27:23Chuck Nice?

  Agreed.

• 1:27:23Neil deGrasse Tyson?

  Okay, so, so there is hope.

• 1:27:25Speaker 02

  Yes.

• 1:27:26Neil deGrasse Tyson?

  And one last thing, because you hinted at it, this point of singularity, where AI trains on itself so that it exponentially gets smarter, like by the minute, that's been called a singularity by many people, of course Ray Kurzweil among them, who's been a guest on a previous episode of Star Talk. Yes, a couple of times, yeah. So what is your sense of this singularity? Is it real the way others say? Is it imminent the way others say?

• 1:27:56Speaker 02

  I don't know the answer to either of those questions. My suspicion is AI will get better at us in the end at everything, better than us at everything. But it'll be sort of one thing at a time. It's currently much better than us at chess and Go. It's much better than us at knowing a lot of things. Not quite as good as us at reasoning. I think rather than sort of massively overtaking us in everything all at once, it'll be done one area at a time.

• 1:28:21Neil deGrasse Tyson?

  And my sort of way out of that is, you know, I get to walk a beach and look at pebbles and seashells. AI doesn't.

• 1:28:30Chuck Nice?

  Yeah, it can create its own beach.

• 1:28:33Neil deGrasse Tyson?

  No, it would only know about the new mollusk that I discovered if I write it up and put it online. So the human can continue to explore the universe in ways that AI doesn't have access to.

• 1:28:46Chuck Nice?

  There's one word missing from your entire assessment.

• 1:28:49Neil deGrasse Tyson?

  What's that?

• 1:28:49Chuck Nice?

  Yet.

• 1:28:55Neil deGrasse Tyson?

  Yeah, I just think of my, you know, will AI come up with a new theory of the universe that requires human insights that it doesn't have, because I'm thinking the way no one has thought before.

• 1:29:07Speaker 02

  I think it will.

• 1:29:08Neil deGrasse Tyson?

  That's not the answer I wanted from you.

• 1:29:10Chuck Nice?

  Yeah. But that's the answer you got.

• 1:29:13Speaker 02

  Let me give you an example. AI is very good at analogies already. So when ChatGPT-4 was not allowed to look on the web, when all its knowledge was in its weights, I asked it, why is a compost heap like an atom bomb? And it knew. It said the energy scales are very different and the time scales are very different. But it then went on to talk about how when a compost heap gets hotter, it generates heat faster. And when an atom bomb generates more neutrons, it generates neutrons faster. So it understood the commonality. And it had to understand that to pack all that knowledge into so few connections, only a trillion or so. That's a source of much creativity.

• 1:29:50Neil deGrasse Tyson?

  And it's not just by finding words that were juxtaposed with other words.

• 1:29:54Speaker 02

  No, it understood what a chain reaction was.

• 1:29:57Neil deGrasse Tyson?

  Yeah.

• 1:29:58Chuck Nice?

  Yeah.

• 1:29:59Neil deGrasse Tyson?

  All right, that's the end of us.

• 1:30:01Chuck Nice?

  Yeah, we're done.

• 1:30:02Neil deGrasse Tyson?

  On Earth, we're done.

• 1:30:03Chuck Nice?

  We're finished.

• 1:30:03Neil deGrasse Tyson?

  This is the last episode.

• 1:30:05Chuck Nice?

  Stick a fork in us, we're done. Gentlemen, it's been a pleasure.

• 1:30:12Neil deGrasse Tyson?

  Well, Geoffrey Hinton, it's been a delight to have you on.

• 1:30:15Chuck Nice?

  Yeah.

• 1:30:16Neil deGrasse Tyson?

  We know you're tugged in many directions, especially after your recent Nobel Prize, and we're delighted you gave us a piece of your surely overscheduled and busy life.

• 1:30:26Speaker 02

  Thank you for inviting me.

• 1:30:28Neil deGrasse Tyson?

  Well.

• 1:30:29Chuck Nice?

  Guys, that was something. Did you sit comfortably through all of that?

• 1:30:32Neil deGrasse Tyson?

  I was, I squirmed.

• 1:30:34Chuck Nice?

  I knew you'd panic. Well, no, I have to tell you that certain parts of the conversation gave me the anxiety of, you know, sitting in a theater with diarrhea.

• 1:30:46Neil deGrasse Tyson?

  Thanks for that explicit.

• 1:30:47Chuck Nice?

  Thanks for sharing.

• 1:30:49Speaker 02

  That's the nicest thing anybody's ever said about me.

• 1:30:57Neil deGrasse Tyson?

  On that note, this has been StarTalk Special Edition. Chuck, always good to have you.

• 1:31:02Chuck Nice?

  Always a pleasure.

• 1:31:03Neil deGrasse Tyson?

  Gary.

• 1:31:03Chuck Nice?

  Pleasure, Neil.

• 1:31:04Neil deGrasse Tyson?

  Love to have you right at my side. Neil deGrasse Tyson, bidding you as always to keep looking up, however much harder that will become.