#220 Terry Sejnowski: The Future of AI, ChatGPT & Deep Learning

This episode of Eye on AI is sponsored by Citrusx. Unlock reliable AI with Citrusx! Our platform simplifies validation and risk management, empowering you to make smarter decisions and stay compliant. Detects and mitigate AI vulnerabilities, biases, and errors with ease. Visit http://email.citrusx.ai/eyeonai to download our free fairness use case and see the solution in action. In this episode of the Eye on AI podcast, Terry Sejnowski, a pioneer in neural networks and computational neuroscience, joins Craig Smith to discuss the future of AI, the evolution of ChatGPT, and the challenges of understanding intelligence. Terry, a key figure in the deep learning revolution, shares insights into how neural networks laid the foundation for modern AI, including ChatGPT's groundbreaking generative capabilities. From its ability to mimic human-like creativity to its limitations in true understanding, we explore what makes ChatGPT remarkable and what it still lacks compared to human cognition. We also dive into fascinating topics like the debate over AI sentience, the concept of "hallucinations" in AI models, and how language models like ChatGPT act as mirrors reflecting user input rather than possessing intrinsic intelligence. Terry explains how understanding language and meaning in AI remains one of the field's greatest challenges. Additionally, Terry shares his perspective on nature-inspired AI and what it will take to develop systems that go beyond prediction to exhibit true autonomy and decision-making. Learn why AI models like ChatGPT are revolutionary yet incomplete, how generative AI might redefine creativity, and what the future holds for AI as we continue to push its boundaries. Don't miss this deep dive into the fascinating world of AI with Terry Sejnowski. Like, subscribe, and hit the notification bell for more cutting-edge AI insights! Stay Updated: Craig Smith Twitter: https://twitter.com/craigss Eye on A. I. Twitter: https://twitter.com/EyeOn_AI (00:00) Introduction to Terry Sejnowski and His Work (03:02) The Origins of Modern AI and Neural Networks (05:29) The Deep Learning Revolution and ImageNet (07:11) Understanding ChatGPT and Generative AI (12:34) Exploring AI Creativity (16:03) Lessons from Gaming AI: AlphaGo and Backgammon (18:37) Early Insights into AI's Affinity for Language (24:48) Syntax vs. Semantics: The Purpose of Language (30:00) How Written Language Transformed AI Training (35:10) Can AI Become Sentient? (41:37) AI Agents and the Next Frontier in Automation (45:43) Nature-Inspired AI: Lessons from Biology (50:02) Digital vs. Biological Computation: Key Differences (54:29) Will AI Replace Jobs? (57:07) The Future of AI

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  The new book was inspired by a disconnect and it really had to do with trying to come to grips with whether or not ChatGPT understands language. There was a big argument that was raging already, you know, back when it first came out two years ago amongst academics and they couldn't agree. I mean, these are people who spent their whole lives on cognitive science and linguistics and, you know, AI and it's still raging and it's still, it's still a very, very hot topic in, in the, in the field of machine learning right now, the extent to which ChatGPT is intelligent.

• 0:35Craig S. Smith?

  In today's fast-paced world, ensuring your AI systems are not only compliant, but also accurate and robust is key. How can you achieve reliable AI outcomes while managing risks effectively? If you're facing challenges with AI risk management, having a solution that ensures accuracy in robustness is invaluable. Citrus X AI offers cutting-edge technology for you to detect and mitigate vulnerabilities, biases and errors while ensuring accuracy, robustness and compliance. Their solution is the only one that offers observability capabilities

• 1:21Craig S. Smith?

  while helping you meet regulatory requirements. Citrusx.ai ensures your AI models are not just performing well, but navigating risk safely. Elevate your AI model management with Citrus X. Visit email.citrusx.ai/eyeonai to book your free demo today and learn more about how their solution can benefit you. That's email, e-m-a-i-l dot citrus x, c-i-t-r-u-s x dot a-i slash eye on a-i,

• 2:04Craig S. Smith?

  eye on a-i all run together, e-y-e-o-n-a-i, to book your free demo today. Can you start, Terry? I know who you are. A lot of people know who you are. I saw you referenced in the Jeff Hinton announcement.

• 2:24Craig S. Smith?

  Which, of course, I would like you to have been included. But, but because Boltzmann machines played such a critical role in that development. So can you give listeners that are not familiar, have not seen our past episodes, a brief introduction of your work and your work at the NERP's Foundation? And then about your new book, which is what I really wanted to talk about?

• 3:02Speaker 00

  Well, yes, Craig, my background is in physics, PhD, theoretical physics for Princeton.

• 3:12Speaker 00

  And at that point, I realized that, you know, physics had reached these high, such high levels that in order to make progress, you had to have an accelerator that was,

• 3:24Craig S. Smith?

  Yeah.

• 3:24Speaker 00

  you know, miles across or set up a satellite. And so I was very attracted to the brain because it was just as mysterious as the universe. But it was something you could actually study, you know, in the lab. And I met Jeff at a small conference, you know, here actually in San Diego, that he organized on association of memory back in 79. That was quite a while ago. And we just hit it off. I mean, it was clear that we both had the same

• 3:57Speaker 00

  goals, you know, to try to understand, you know, how the brain could possibly solve these difficult problems, number one. Number two,

• 4:09Speaker 00

  what was, you know, computational capabilities of very large scale network models? And it was in its infancy back then. And so that, you know, John Hoffield was my thesis advisor. And we ended up generalizing his model by adding noise to it of all things, you know, to heat it up. It's a physics model, spin glass model. And by heating it up, we were able to find not just local solutions, but also a global minima. And also what popped out of that, and this is still amazing.

• 4:51Speaker 00

  One of the most beautiful things that I've ever done, which is a learning algorithm when you come to equilibrium. And so it was a big, it was a big physics project because it took advantage of everything that we know about statistical mechanics. Boltzmann, of course, was the father of statistical mechanics. And so, so that, that broke the logjam. That led to backprop, which Jeff and Dave Romahart introduced shortly thereafter. And the rest is history. So Jeff, you know, he spent the next 20 years figuring out how to use the Boltzmann machine and then eventually backprop to solve these difficult problems. And, you know, ImageNet back, I think it was 2012 or 2013.

• 5:34Speaker 00

  At NeurIPS really started the deep learning revolution with object recognition in images. But, but my, my career path went off toward neuroscience. I did a postdoc at Harvard Neurobiology and then eventually wound up here at La Jolla at the Salk Institute. I also have a faculty position at UC San Diego where I teach. And I have an Institute for No Computation. So I, I've been, I've had this dual role. And, and by the way, I, very early on, I became the president of the Neural Information Processing Systems Foundation. Neural networks was very hot back in the eighties. It was very interesting and intriguing. People from many, many different areas of physics and engineering would show up at these meetings.

• 6:18Speaker 00

  Around five, 600 people would show up. And that grew gradually over years and, and, you know, we didn't have enough computing power really to scale things up. But eventually that happened, you know, in this century. We didn't know how well they would do. And it's fab, way beyond my expectations of what I expected back then.

• 6:42Craig S. Smith?

  Yeah, and and so and I've I've read a deep learning revolution and for listeners who want a great a history of that whole development, it's a wonderful book.

• 7:00Craig S. Smith?

  That came out, I think. Five years ago, I don't remember.

• 7:05Speaker 00

  Yeah, it was two thousand eighteen, it was the origin story of modern AI.

• 7:12Craig S. Smith?

  Yeah. And so now you've got, you know, it's fast forward.

• 7:19Craig S. Smith?

  Transformers appeared.

• 7:23Craig S. Smith?

  And together with

• 7:27Craig S. Smith?

  back propagation and some other things, it's created this entirely new field of generative AI.

• 7:39Craig S. Smith?

  The most public facing of which is the chat GPT models.

• 7:47Craig S. Smith?

  And you're writing about this sort of looking at what that means and what to expect going forward. So can you talk about the new book?

• 7:58Speaker 00

  Yes, the the new book was inspired by a disconnect and it it really had to do with trying to come to grips with whether or not chat GPT understands language, does it understand, understanding?

• 8:15Craig S. Smith?

  Yeah.

• 8:16Speaker 00

  And all of all, you know, there was a big argument that was raging already, you know, back and this when it first came out two years ago. Amongst academics. They couldn't agree. I mean, these are people who spent their whole lives on cognitive science and linguistics and you know, AI and and it's still raging and it's still it's still very, very

• 8:42Speaker 00

  hot topic in in the in the field of machine learning right now.

• 8:47Speaker 00

  The extent to which. Chat G P is intelligent. Yeah. And and so I looked into it and I started trying to understand

• 8:57Speaker 00

  where the disagreement came from. And and I I made a bunch of observations and and this is something that had to do with how people are reusing it and and it occurred to me that, look, Chat G P T is trained on, you know, thousands and thousands of books and enormous amount of of of articles all over, you know, from, you know, we're talking about novels, textbooks and and and it can adopt any persona because it has all of that stored away somewhere. And when you talk to it, it has to pick a persona. Yeah.

• 9:39Speaker 00

  And and you can ask it, you can say, I can, I can, you can tell it, I I want you to be a poet. Or I want you to be a computer scientist. You you can set it up, that's the prompt. And what and depending on how you prompt and especially how you give, you know, you set it up in terms of what what it what it is that you wanted to, how you wanted to do it, what you wanted to do and what persona it should have. You that determines the responses you're going to get, you know, the quality of the responses, the the kinds of things. And and and once I saw that, I realized that this is, this is really a mirror. It's it's like looking into a mirror. And, you know, it it is looking at you and it is trying to respond

• 10:24Speaker 00

  as as as it best it can to whatever kind of prompts you're giving it.

• 10:31Speaker 00

  And and then and then, you know, once, once I realized that, I I realized that we're we're it's completely wrong to think of it for the way that we are, we're trying to judge it, is it human or not? It's obviously not human. Yeah. It's like an alien that suddenly arrived on Earth and is talking to us, but it's it's got, it's not, it's coming from some other world, right? And and so we should, we, but what it is reveal is something that I think is very deep

• 11:03Speaker 00

  revelation.

• 11:06Speaker 00

  About the poverty of the words that we're using to describe cognitive function. All these words come from, you know, 19th century psychology or even earlier. And they're ambiguous, they have multiple meanings. They're, they're, they're not as well tied down scientifically as words in physics like, you know, matter, mass, energy. All of this is something that is, we're in a kind of a pre-Kaprinikan

• 11:37Speaker 00

  era here in AI. And, and, and, and, and so, you know, it, it, we don't, we, we, we probe words like intelligence. We don't really know what those, you know, how, how to apply it. There's a raging battle in biology, you know, about whether other species are, what, what kind of intelligence they have, or are they conscious? And, you know, it's, it's books written about this. And I think that we, we, we're now on the, we've entered an era now where we can begin to maybe understand these concepts in a more deep mathematical way. Cause we can create these artifacts that have some of the properties of, of what we consider

• 12:21Speaker 00

  language and understanding. And, and if we can understand them first mathematically, maybe we'll be able to understand our, you know, animal brains.

• 12:32Craig S. Smith?

  Yeah. Well, and in that language, in the fuzzy language used to describe this stuff, understand, understanding is, is one of the first that creates confusion. And I remember thinking about that when Alpha Zero, I, I'd say that was the model at the time, beat Lee Sedol in that Go game in Seoul in whenever it was, I've forgotten, 2016 or something. AlphaGo made this completely unorthodox move, move 37.

• 13:16Craig S. Smith?

  It placed a stone up in an empty part of the board. And as I recall, Lee Sedol got up. And walked away from the table for a little while to think about it. Uh, and no one understood it. And then as the game played out, it was that move that ultimately turned the tide and and uh beat least at all. And I remember thinking and saying to people, wow, that AI system understood. You know, it understood uh uh uh I mean, whatever understands means.

• 14:01Craig S. Smith?

  Uh, it it wasn't simply running down uh, you know, uh, some sort of a search tree, Monte Carlo search or something. Uh, it it it looks like it understood it. So and how do you work with that word, understanding?

• 14:20Speaker 00

  Okay, so first a couple of of footnotes to your uh observation. First,

• 14:27Speaker 00

  this was a sign that in addition to being able to talk, that the these these large language models are actually creative.

• 14:38Craig S. Smith?

  Yeah.

• 14:39Speaker 00

  And and we've always thought that was the highest level of human cognition, right? And by the way, hallucinations with people say, oh, you know, you can't trust it. It's a form of creativity. It's a it's a it's it's the clip flip side of creativity. You know, when it's create, when it's hallucinating, it actually doesn't just make things up. It's very plausible things, right? Things that, you know, could could exist or could happen.

• 15:08Speaker 00

  But they never did. Uh, and and so and so, you know, by the way, this is also true in humans, right? Humans also confabulate. It it's it's a different word in in medicine, but it's the same thing is that humans uh for under some conditions.

• 15:28Speaker 00

  Will make make up things just like that, right? And and it's it's like, you know, the brain.

• 15:35Speaker 00

  Is not able to actually judge themselves. I mean, of these people who have the Korsakov syndrome, they can't themselves judge what's right, what's true. They're just saying things that the brain thinks is true. Their brains.

• 15:50Speaker 00

  Okay, the other thing, the other footnote is that what you just described was already apparent back in the nineteen eighties,

• 16:01Speaker 00

  early nineties, Jerry Tesauro, who I worked with actually on applying these learning algorithms to Backgammon, right? It's a simpler game, but it's actually a very popular game. And what makes it actually difficult, more difficult in some ways, is that is probabilistic. You throw a dice. Right? So it's not like chess where you can go, you know, twenty moves ahead and, you know, it's deterministic in terms of, you know, knowing where pieces are and where they're going to be. But, but nonetheless, he, it turns out, he applied the very same, the, the, the, really, the secret to AlphaGo was to have the game play itself.

• 16:45Craig S. Smith?

  Right.

• 16:46Speaker 00

  Right. And, and, and, and because it's playing itself, it basically can go off in new directions that are unanchored from the way human beings play the game. And, and it reached, it's really interesting. Back then, I mean, this is a computer really puny compared to what we have today, but nonetheless, you know, he was able to reach a level of play that some of the experts said were at the championship level. And in some cases, was making decisions at critical points in the game that were even better than the decisions that humans were making. It was being creative with, with new moves that no human had ever made before. So it was already apparent back then. Okay. And if I could, you know, this is a little bit immodest, but.

• 17:32Speaker 00

  You know, most of the of the networks back in the eighties were toy problems, just created because the networks are too small to really solve any real real problem. One exception to that. Maybe there were a few, okay, but the the the one that I was involved in was NetTalk.

• 17:49Craig S. Smith?

  Yeah, I remember, yeah.

• 17:51Speaker 00

  And and that was a problem in linguistics of how you pronounce letters in words. Phonology, and it's a very difficult problem in English, especially because it's so ambiguous. The single letter can be pronounced in some cases, you know, five, 10 different ways. Um, it it English has gotten influences from many other languages and, you know, it's it's very complicated, uh, how how those mesh. But and and and linguists, you know, who are using rule-based approaches back then, um, came up with rules, but there are exceptions to the rules. And then there are within the exceptions like for French words, there were rules for the French exceptions and it would you know, rules all the way down and you have books with 300 pages of rules and exceptions.

• 18:36Speaker 00

  Well, we we trained up, this is a summer research project, uh, it was a of graduate student from Princeton, Charlie Rosenberg, who showed up and I I didn't know anything about language, linguistics. Uh, but he did, he worked with George Miller, and so we took on that project. And by the end of the summer, I'll tell you, it was pronouncing words that understandably, we played it through a deck talk, which actually you could hear it. And it it was it was it was amazing that you can get such a tiny network, 20,000 connections, you know, 200 units. And it it absorbed not just the regularities, but also the exceptions in English pronunciation. And this

• 19:18Speaker 00

  was really, it was amazing demonstration because I could play this tape for people at the beginning, it babbled and a bit later it started doing small words. And then later at the end, you know, it just was able to generalize so new texts and and beautifully. Now that, in retrospect, was an indication that the networks love language.

• 19:38Craig S. Smith?

  Mm hmm.

• 19:38Speaker 00

  Right? And and that's clearly what ChatGPT is telling us. It's it's like unbelievable how well it can do. And it's actually still tiny compared to the brain, right? Even with the trillion weights, you know, the brain has a million billion. Uh, you know, it's it's the thousands of trillion. So that that's that's, uh, You know, in some ways, ChatG P is already super intelligent because it's absorbed much more knowledge than any human brain could, right?

• 20:09Craig S. Smith?

  Yeah, yeah, and a couple of thoughts on that. One is on on the hallucinations. I often tell people that as this is what you're saying, that people hallucinate too. If you ask me what color shirt I was wearing at my fifth birthday party, which I can remember very well, I'll I'll say blue. And I can see that image in my head. But if I flip back through family photos, I see, oh, I was wearing an orange shirt. You know, it's but you are so convinced that image is so clear in your head. And that's a form of hallucination.

• 20:54Craig S. Smith?

  And so this word understand, I go back and forth. I mean, I tell people who are not involved in AI and are totally amazed by the large language models. That, you know, these things, it's it's just

• 21:19Craig S. Smith?

  predicting the next token in the series based on the probability in its training data. And if you hit a spot where the training data is is relatively sparse, it'll it'll pick a probability. And that will not be what you consider as right. So when it is picking the probabilities that that we consider right, we consider that understanding. And and it It sort of begs the question of what human understanding is.

• 22:03Craig S. Smith?

  If indeed we're, you know, a network of weights that we've accumulated throughout our lifetime. You know, are we just coming up with, with probably, this is the stochastic parrot argument. And, and I mentioned this to Jeff last time I talked to him, or I haven't talked to him for well over a year. He's suddenly unavailable for, for my level of people. But, and he said, no, they understand. And we didn't get into a debate about what understand means.

• 22:45Craig S. Smith?

  So anyway, how do you deal with that term, understanding? And do, do humans understand in the same way that these massive models understand? In that we're just, you know, our, our chain of thought is based on.

• 23:06Speaker 00

  We don't have to go to the chat GDP.

• 23:12Speaker 00

  You know, different people understand things differently.

• 23:17Speaker 00

  It's not one thing that, it's not the one way to, to, there's no, no one way to understand something. There are many different levels of understanding. A physicist understands, you know, the composition of things much better than a person who just, you know,

• 23:41Speaker 00

  picks things up and, and uses them. But, but a carpenter understands wood perhaps better than a physicist because he knows about the grain and he knows about how to cut it and different types of wood and so forth, you know? And, and so there's many different ways to understand something. So that's number one. Number two, the.

• 24:05Speaker 00

  I think the Jeff's comment, and I've talked to him about this.

• 24:14Speaker 00

  In fact, recently, I've talked to him almost every day. I mean, you know, he's

• 24:19Speaker 00

  he's transformed into a, you know, completely different stage of his life now, right? I mean, once you get the Nobel Prize, it's a different, you know, difference. You know, society now puts you on a pedestal. But

• 24:36Speaker 00

  his argument is the following, that

• 24:40Speaker 00

  in order to be able to predict the next word, you have to have a pretty good internal model of what that sentence is all about, right?

• 24:50Speaker 00

  And that means semantics. It's, you know, you have to, it's more than just predicting whether it's a verb or a noun. It's, you know, how that new word that you're predicting fits into the rest of the words, right? It has the context of that word. And the internal model, if you keep training it and it gets better and better and better, the internal model is going to get better and better and better. Right? And at some point, it becomes good enough so that it actually comes up with the appropriate reasonable words for the meaning of that sentence, right? And that's a form of understanding.

• 25:36Speaker 00

  You might say that it's a superficial understanding.

• 25:41Speaker 00

  You know, what's the level of depth? How do you compare it with?

• 25:47Speaker 00

  You know, a human in the same situation.

• 25:52Speaker 00

  It depends on the human. It depends on what the, also, as you say, the database that was used to train it. Depends on a lot of factors. But, but I think that Jeff is right, that the internal model. And we actually have some evidence for this, that the internal model is integrating information from many different sources, first of all, from the text that it's being trained on, but also from the order of the words, that syntax, that's something that linguists really put a lot of effort into in the last century, understanding that was important. But in addition, it turns out what was ignored in the last century was that language is

• 26:36Speaker 00

  all about meaning. And that is syntax, sorry, it was a Freudian slip. Me, it's all about semantics.

• 26:47Craig S. Smith?

  Yeah.

• 26:48Speaker 00

  And semantics is the purpose of language, is we talk to each other so that we can understand each other, the meaning of what I'm trying to communicate with you, and back and forth, right? That's, that's at a semantic level, that we're trying to exchange thoughts, right? We, and the order of the words, it helps with getting the semantics across, but it's, it's not, doesn't contain the meaning itself, it's actually, of all things, a lot of the meaning is coming across not just from the words, but how you say the words, what you emphasize. It also depends on facial expressions. If you're talking to somebody and you see them, you'll have a different message, a

• 27:32Speaker 00

  different meaning than if you are just listening to somebody on a recording, right? It's, language is incredibly, has many, many, many dimensions and many, many ways that we take information in and try to incorporate it. Our brain is doing a lot of that subconsciously, by the way, we're not even aware, but it just happens.

• 27:53Craig S. Smith?

  Yeah. The, there was a wonderful book, probably 20 years ago, by a New York Times writer named James Gleick, called The Information, I don't know if you read it, but, and he talks there about...

• 28:15Craig S. Smith?

  Language and writing and thought and his

• 28:24Craig S. Smith?

  premise, although not not alone. You know, he drew on a lot of people, but is that

• 28:33Craig S. Smith?

  speech disappears, goes out and as soon as the sound waves have passed you, it's over. And it wasn't until writing developed that people could start analyzing.

• 28:54Craig S. Smith?

  That people really started thinking that that thinking analytically that they could say something, stick it out there, look at it and analyze it.

• 29:07Craig S. Smith?

  And that that felt pretty deep to me. So what what are your thoughts about? How this not only speech, but how

• 29:23Craig S. Smith?

  the the written language, which is what these large language models are trained on, they're not trained on people just talking, it's written language and huge volumes of written language, how that relates to thought and and whether in this whole idea of understanding, whether it's

• 29:47Craig S. Smith?

  it's it's the analytic analytic exercise on written language that creates understanding or you know what I'm thinking about, maybe not, I mean you.

• 30:00Speaker 00

  Okay, well, I I haven't read that book, but I agree with him. In fact, when people say language, they usually been

• 30:11Speaker 00

  spoken languages as being the breakthrough, the breakthrough that the humans evolved this ability to talk to each other. But I can, you know, it's pretty clear that there's there's all kinds of limitations, as you say, the spoken words basically disappear very rapidly.

• 30:30Speaker 00

  And you know, your memory of the words will mutate.

• 30:37Speaker 00

  As you know, with this, a lot of studies have been done, you know, where the story changes, you know, it, it, the details change if it's retold.

• 30:47Speaker 00

  And also, language can be used for controlling other people. Rhetoric, you know, if you're good at, at, at, at use, tapping into emotions, if you're good at, you know,

• 31:06Speaker 00

  coming up with arguments that sound plausible, you can, you can get a lot of people to do things for you. Right? Yeah. And the bigger your bullhorn, the more the people. Unfortunately, what happened is that technology got better and better. Right? You, you know, you can have more and more people come and listen to you, or you can, the radio or the television, you know, you, you, you get, the, the, the words are, you know, can be spread throughout, you know, millions of whole nations, right? Uh, now, the, what you're, but, but, I, for, for me, the real turning point was not

• 31:47Speaker 00

  the evolution of spoken language, but the written version. That's an invention, by the way. That's not something that came from nature. We're actually using, we adapted our visual system in order to be able to do that.

• 32:03Speaker 00

  Of course, you know, the visual system evolved too. But the, the, the, the point is that we have to go to school to learn how to read. You know, it's many, many years. I mean, it's something we're over-practiced on, you know, if you're, if you went to school for, you know, I, I've been going to school my whole life, you know, reading scientific papers all the time and trying to understand written words, uh, you know, the, the jargon vocabulary is, I mean, it's unbelievable what we now have, right, out there. And it's really

• 32:36Speaker 00

  It would allowed us to do two things. First of all, to accumulate knowledge. Number two, to correct knowledge.

• 32:42Craig S. Smith?

  Yeah.

• 32:42Speaker 00

  I mean, you can go back and here's what Mr. X said about why. Well, I'm gonna investigate this why and I have found that Mr. X was wrong. And here's why. And you know, that could be at a scientific level where you do an experiment. Or it could just be, you know, at a conceptual level. Or, you know, in other words, you can, you constantly, you can build on what was there before. Whereas if all you have is the written word, basically, the most you could do is to have somebody who's really good at memorizing stuff. The bards, the people who were told the stories of the past. That was the way history was transmitted from one generation to the next. And that's very limited. The bandwidth there is very limited.

• 33:27Speaker 00

  But now, when you could write, and now, you know, books was a big advance. Because now you could have multiple copies of the same thing. Instead of, you know, stone tools, you know, for hieroglyphics or whatever. Now you could disseminate it amongst at least the people who could read. You know, which is a very small fraction back then, but now many more people. And then eventually, you know, science.

• 33:55Craig S. Smith?

  Yeah.

• 33:56Speaker 00

  Took that to another level. Which is that not only are you transmitting words and thoughts, but you're transmitting observations, experiments, new insights that other can be replicated. Other people can do the experiments. Other people can build the same thing.

• 34:18Craig S. Smith?

  Yeah.

• 34:21Craig S. Smith?

  Well, on, so on understanding, coming back to understanding. Your feeling is that, that these large models.

• 34:33Craig S. Smith?

  Do at some level understand, and that's with using this fuzzy word, understand. What's the limit of that? Because understanding leads

• 34:50Craig S. Smith?

  to self-awareness and self-awareness leads to sentience. Yeah, what's your feeling about that spectrum? And where these models are on it?

• 35:07Speaker 00

  Okay, so that's a big part of my book. I have several chapters specifically trying to understand

• 35:16Speaker 00

  what's missing. Okay. So the the the model that is used in ChatGPT is primarily a model of the cerebral cortex.

• 35:27Craig S. Smith?

  Yeah.

• 35:28Speaker 00

  You know, the the big knowledge base that we have.

• 35:32Speaker 00

  But, you know, it's only one of about 100 different parts of the brain that are absolutely essential for its function. Right? So I'll tell you, most of most of what's in the brain is missing from ChatGPT.

• 35:43Craig S. Smith?

  Yeah.

• 35:44Speaker 00

  And I'll just give you, I made a list in the book, but I'll give you the sort of the top hits. Okay, number one, ChatGPT does not have a goal.

• 35:56Speaker 00

  Now, nature put goals into us, which is survival, reproduction. You know, we got to eat. You know, we we have to sleep and there's all sorts of things that humans. You know, they have, they have.

• 36:13Speaker 00

  They're they're very social creatures. So, you know, we

• 36:17Speaker 00

  we need to be in a social group in order for the brain to flourish. Right? And and, you know, that's that comes from evolution.

• 36:30Speaker 00

  So that's number one. We, you know, they, they, they were trained without any real goals, or for that matter, any, any instruction about what's good and what's bad. You know, children, when they grow up, right, they, uh, they're given all kinds of feedback by their parents, by their peers, by their teachers. Uh, you know, don't say that. You know, that's bad. You know, you, you, you shouldn't do that. Right? Or it's dangerous. And all, ChatGPT never gets any of that feedback. Never, it's never told what's good or what's bad. It's never told, you know, what's dangerous. And, and, and so that's reinforcement learning. It's another part of the brain, the basal ganglia,

• 37:13Speaker 00

  uh, which interestingly was used in, um, AlphaGo. AlphaGo had a value function. Right. Which the basal ganglia creates a value for future rewards. You know, what you should do next? If you want to get an award, reward. A reward could be anything. You know, it could be some tasty morsel, or it could be a, a university degree, or it could be becoming president, right? You know, if you think that that's something you, you, you cherish. Um, but

• 37:44Speaker 00

  you know, the, the, the reality, the, you know, the reality is that, uh, it, you know, ChatGPT only has the kind of a superficial

• 37:58Speaker 00

  set of, of, of goals, which was, you know, to predict the next word. That was very superficial. But it's very important, by the way, the same, uh, humans have a very, very likely are using the same kind of algorithm. We know it's true for the value function. It's a reward prediction error. You predict the reward you're going to get. You compare it to what you do get, and from that difference, you change the weights to, in order to be able to make a better decision the next time you get that input. Okay. So that's, that's a very general principle. But now here's something that is even, talking now of, with regard to sentience. Here's something that is a showstopper. Literally. Okay. So you're talking to ChatGPT and it's responding.

• 38:43Speaker 00

  It keeps responding word after word, and then at the end it, it finishes and it thanks you, or it says, I hope this is okay. It's very polite. Right? And then it stops. What happens to the network when it stops?

• 38:59Speaker 00

  Nothing. Nothing is going on in that network. The only time that anything happens is when you give it a question, or you give it a prompt, right? That, that means that there's nothing going on there. It's, it's, it's completely empty. It doesn't have any self-generating activity. Right? So, I, I think it's a, it's a, it's a, it's a, it's literally a no-brainer about sentience. It's not sentient in any sense of the way that we are, because you put us in a room, no sensory inputs. What's going on? Well, you're thinking away, right? You're thinking about what you're gonna do later in the day. You may think that you're hungry. You may wanna think about going to the supermarket.

• 39:43Speaker 00

  In other words, you're planning all the time. You're coming up with responses. You're, you're thinking about, you know, what, what your goals are and what, what, how to accomplish that. Or maybe you're just enjoying a movie and thinking about the movie. You know, but the idea, though, is that we, we have a constant flow of thoughts in the absence of any direct input. Obviously, you may be responding to something you heard yesterday, and that's been circulating because, you know, you've been a new, a new way, a new, a new idea. Or maybe you were, I don't know. I mean, some, you know, actually, unfortunately, a lot of these are emotional things that you think about.

• 40:27Speaker 00

  You know, you get unhappy, you get sad. You, you, you, you, you feel that you've been maligned. You know, and, and that, and that'll continue to circulate. In fact, sometimes that leads to mental disorders, right? When you have people who are anxious and so forth. So it so in any case, uh, the, you know, there's, there's the many, many, many

• 40:53Speaker 00

  lessons to be learned from the, the way that nature has gone about creating autonomy. You know, something and all animals have to survive. All animals are autonomous to some degree. And insects, I mean, it's not just animals, insects, you know, plants. All of nature is autonomous in terms of self-regulating and self-creating and so forth. And of course they interact with each other and that's part of the complexity of nature. But right now, uh, you know, these chat GDP, you know, they're trying to make them into agents, but they're not, they're just not, they don't have the same agency that we have.

• 41:34Craig S. Smith?

  Yeah, well, let's talk about agents though, because that is the next frontier. And there are

• 41:43Craig S. Smith?

  uh, now agents that can perform mental tasks that were previously performed by humans. You give it a, the agent a goal and, and it'll perform that task continuously. Uh, and now you can chain agents together where each one performs a discrete task and hands the output off to another who performs another task. Uh, where do you see that going? Because it seems that combined with, with the quote unquote, understanding

• 42:24Craig S. Smith?

  of, of large models that having goals

• 42:31Craig S. Smith?

  and having agents collaborate with one another could lead to something unexpected.

• 42:40Speaker 00

  You know, it, it, it's a interesting direction and one of the things that

• 42:48Speaker 00

  We have to be, you know, prepared for is that

• 42:57Speaker 00

  nobody can predict what's what will happen, right? I mean, in other words, it's a surprise, a surprise. I mean, ChatGTP was a huge surprise, right? All these things, you know, the language translation was a big surprise. And and I'm sure that's going to continue. So who knows? But one thing to keep in mind, and this is kind of a more mathematical argument, is that two ChatGDPs talking to each other is just one giant GPT. No different. It's maybe it's, you know, you've externalized some of the internal stuff that's going on, but it's basically the same.

• 43:37Speaker 00

  It might have additional capabilities. I'm not saying that it doesn't. What I'm saying, though, is that it's really, there's nothing that is special, you know, in terms of the

• 43:54Speaker 00

  you know, it's, they're not, they're not human agents, right? It's not, they're not behaving like human agents.

• 44:02Craig S. Smith?

  Yeah.

• 44:03Speaker 00

  It's, you know, like I say, you know, we're there are other parts of the brain, there are other things that make us agents. And maybe there's, again, agency is one of those words that has multiple meanings, that has a spectrum, and that maybe, you know, there is a little bit of agency there that can be enhanced, amplified somehow. But it's not, you know, it's, it's, it's pretty far along that spectrum, you know, in terms of where we are and where other species are. But maybe, you know, again, this is something that, you know, I started by asking you if you could have predicted where the internet was going.

• 44:47Speaker 00

  This is, this is an even more

• 44:51Speaker 00

  difficult problem to try to imagine. And, you know, it's

• 44:58Speaker 00

  You know, people worry about superintelligence and taking over, you know, existential threats and so forth. I don't think we actually know what the real threat is going to be. I think that there's something unintended consequence that's going to happen somewhere up the road, and it's going to be unexpected. And, you know, who knows what it's going to be?

• 45:15Craig S. Smith?

  Yeah. In the book, you talk about nature-inspired directions for developing generative AI. What do you, can you talk about what you mean by nature-inspired? Is that, are you talking about other functions in the brain, for example, or?

• 45:39Speaker 00

  Well, okay, so, you know, the, right now, these large language models are completely helpless by themselves.

• 45:50Craig S. Smith?

  Yeah.

• 45:50Speaker 00

  Literally. You know, they depend completely on us, moment by moment. They depend on us for the power to get them going. They depend on us to give them inputs, right? They depend on us for, you know, improving them and so forth. I mean, you know, they're really cocooned.

• 46:11Craig S. Smith?

  Yeah.

• 46:12Speaker 00

  In a very complex web of technology. And, and, you know, you go on the simplest creature, you know, there's a field mouse out there, right? And they're by themselves. They're going and they're finding food and they're interacting with people in the tunnels and so forth. I mean, my God, they are so much more advanced than anything that we're building right now in AI. It's, they're just like in a different world because they're, they're able to navigate the real complexities of the world, which are not words, but the physical world, the physics that, you know, that is really...

• 46:50Speaker 00

  That determines the properties of materials that, you know, if it's digging a hole, it's got to figure out how to do that. If it's got to find food that is edible, that isn't going to kill them. They got to do this all, you know, in order to survive. And, you know, the brain has figured out how to do that. That's why they have, we, the brain has a hundred other parts. Yeah. Help us and all species, including insects, how they could be able to navigate. And so here's a story. When Jeff and I and others in the early days of neural networks in the eighties,

• 47:34Speaker 00

  were trying to have come up with this alternative approach to AI, you know, we were not taken seriously. Right? We were the furry little mammals under the feet of the dinosaurs. But I was invited to give a distinguished lecture to MIT. And I was really, you know, thrilled. I mean, this is amazing. You know, they're going to take me seriously. So I went there and, you know, I was told that there's a tradition at the AI lab that the distinguished lecturer would have an opportunity at lunch to have a discussion with the faculty and students. I said, great. As we're going at the elevator, you know, my host turned to me and said, Terry, and

• 48:18Speaker 00

  they hate what you do. Whoa, okay. And, you know, I suddenly realized that it was, you know, this is not what I thought it would be. And in fact, it's like walking into the lion's den, right? I mean, here I am. And the host turns to me and said, oh my, we've never had so many people turn out before, right? It was the main attraction here. And so, and we'll give you five minutes to start the discussion.

• 48:55Speaker 00

  Wow, okay. Uh, so what do you do?

• 49:00Speaker 00

  You know, literally, I had, I had, and I had less than five minutes to actually come up with a plan.

• 49:06Speaker 00

  And, and, so I looked at the table with the sandwiches on it, and I, I saw a fly circling. And I said to the audience, I said,

• 49:17Speaker 00

  "You see that fly? Okay.

• 49:21Speaker 00

  Well, what's amazing, you know, it can fly, it can find food, it could reproduce, and it's, it's doing that with a hundred thousand neurons, right? You have a supercomputer in a basement, you know, Cray XMP, costs you a hundred million dollars. It can't fly, it can't see, and it's not gonna reproduce."

• 49:42Craig S. Smith?

  Yeah.

• 49:44Speaker 00

  You know? What's wrong with this picture?

• 49:49Speaker 00

  Dead silence.

• 49:51Craig S. Smith?

  Yeah.

• 49:53Speaker 00

  Okay, but let me tell you the, the, the, the, the, how it ended. This is really even more interesting. So, one of the senior faculty said, "Well, we haven't written the vision program yet." And, you know, I said, "Well, look, DARPA has put billions of dollars into, uh, computer vision, and, you know, it's, it, this program has not yet been written. There's no indication that, it's just, it's growing combinatorically, and it's, it's no, it's, it's not converging." Okay, good luck with that. And then, uh, another faculty said, "Well, but, you know, we have a guarantee from Turing, you know, you know, that, you know, that, that, the, these, uh, digital computers can do universal computation, so eventually we'll, we'll find a program."

• 50:39Speaker 00

  And, and, and I said, "Well, wait a second now, okay. The, it's not just finding the program, but how, how fast can you get the answer?"

• 50:49Craig S. Smith?

  Yeah.

• 50:50Speaker 00

  And because of that, it determines whether you're gonna be eaten or not, right? And it's, you have to make a decision quickly, and that, that's how nature evolved in order to be able to make quick decisions. And if, the finally, you know, there's someone at the back, was a student, said, "The difference between the fly and, and the supercomputer downstairs is that the digital computer is capable of, of, of, right, of running any program, not very efficiently."

• 51:18Speaker 00

  Whereas the fly brain can only run that program. That's it's running, it's it's it's it's doing one thing and does it very well. And I said, you're absolutely right. That's exactly the difference. It's nature has done something special purpose. And digital computers are more general. You can simulate lots of different things. But the difference is that just knowing with the architecture of the digital computer, you have no concept of how to solve any problem because it's so general. Whereas you go into the fly brain, the algorithm is that literally the hardware. It is the structure of the connections between the units. By reverse engineering, the fly brain will figure out how to do vision.

• 52:04Speaker 00

  We'll figure out how to make decisions, how other animals do that. And at that point, that was the end of the discussion. So, so, you know, this is, this was kind of a harbinger of

• 52:21Speaker 00

  of a completely new way of looking at computation that apparently these super smart people at MIT had never even thought about this. It was literally was, it was a student up there who actually understood what the difference was.

• 52:37Craig S. Smith?

  Interesting. So we're coming up to an hour. I don't want to take too much of your time. But what's the sort of forward looking conclusion in the book? And you can tell Terry, and I apologize, I haven't read the book yet. I will read it by the next time I see you. But, uh,

• 53:00Speaker 00

  Right.

• 53:00Craig S. Smith?

  But what is the the forward looking conclusion?

• 53:06Speaker 00

  So first

• 53:07Craig S. Smith?

  Where we are now?

• 53:09Speaker 00

  Yeah, first of all, the book is written for the general public.

• 53:13Speaker 00

  There are some parts in it that are technical, but they're, I asked ChatGPT to summarize them for me.

• 53:21Speaker 00

  And I've gotten feedback from reviewers saying that they were really helpful.

• 53:26Speaker 00

  So there's something in it for everybody. But there are a couple of messages that I wanted to get across.

• 53:36Speaker 00

  A question that I'm often asked is, well, I lose my job. I mean, and you know, the press is just churning out articles about how, you know, you know, you're, you know, people are going to be obsolete, you know, superintelligence taking, I mean, you know, it's crazy out there right now. It's, you know, people don't really, who don't really understand what's going on inside. I'm an insider, so I have a little bit of an insight here. And so here's, here's my response, or at least my, my advice to people out there who are worried about, there are a lot of people worried about their jobs right now. They should be, with, given what they've read. You're not going to lose your job, but your job is going to change.

• 54:21Speaker 00

  And it's going to change because you're going to be given new tools to use. And here's the analogy, okay? Suppose that your job is to dig ditches with your hands.

• 54:34Speaker 00

  Right? Like a, like a mouse.

• 54:40Speaker 00

  You know, it's a hard job. I mean, you know, the soil is not necessarily easy to get out and sometimes it's rocky and oh my God, you know, what a horrible job to have, right?

• 54:52Speaker 00

  But you know, you know, if you have to, you can do it, right? And there's a lot of work that people do that's a lot like that, although maybe not as dirty, but it's still, you know, routine stuff that has to be done. Now, but, but somebody invents the shovel.

• 55:11Speaker 00

  A new technology. So, so you, you, you know, you, you pick up the shovel and you're not quite sure what end to use yet, you know, and you play with it, but eventually you, you realize that, oh, I can, I can dig. I can dig faster. I could dig deeper. It's much more efficient. It's a much better tool, but I have to learn how to use it first before I become good at it so that it's actually better than I, I, I, you know, you're, you're pretty good at doing the, the raw digging. But, and so now you have, you have better technology. Same thing is going to happen. You know, people are going to start using chat G D P. It's going to make a lot of their routine stuff easier, faster. It's not going to replace them.

• 55:56Speaker 00

  They're still going to be there. They're the ones who are using the tool. It's not chat G D P using them, right? So that's one thing. And the second thing, and this is something that was also a, a kind of a message from my first book, which is that AI is going to make you smarter, literally.

• 56:17Craig S. Smith?

  Yeah.

• 56:17Speaker 00

  Literally. And, and it's going to make you smarter because you're going to have access to much more knowledge and you're going to be, you know, exposed to things that chat G D P is gathering for you. I mean, this, this is kind of a Google search on steroids now where you're talking about, you know, coming up with answers to English questions, not to key words, right? And, and that's already happening, right? That's already happening, but it turns out again, you have to learn how to use the tool. You have to learn how to prompt. I have a whole chapter on power of the prompt. And, and you have to actually interact with it in, in, in a different way. You, you can't just, it's not like a typewriter.

• 57:04Craig S. Smith?

  Yeah.

• 57:04Speaker 00

  Um, you know, the, the, actually, it's really interesting. The typewriter, if you look at it, right? The typewriter really is going to be obsolete in a few years because of the fact we can talk now to digital computers. We don't have to use keyboards. We can, I'm sure that they're not going to go away immediately, but eventually there will be much better ways of interacting. Even, okay, one of my former postdocs has a company now called Soft Eye. And he thinks that cell, that cell phones are, could become obsolete pretty soon. They're going to be replaced by AI.

• 57:38Craig S. Smith?

  Yeah.

• 57:39Speaker 00

  You know, that'd be, he, he might be right, but you know, he has, he has a startup company and he's, you know, negotiating with big companies. But you know, the, the point is that, okay, I, the internet has already done that for me, right? For all practical purposes, I've become omniscient.

• 57:59Speaker 00

  I can search the, the world's database for anything. And, you know, and now that I can use ChatGPT, I do, I can do that even more efficiently, right?

• 58:07Craig S. Smith?

  Yeah.

• 58:08Speaker 00

  So, so, so this is, and by the way, it's going right into my brain. My brain is learning things, right? It's always observing things and it's doing it now at a much higher rate than it used to. And in fact, we have, you know, humans can learn,

• 58:24Speaker 00

  you know, for your entire life, lifelong learning. It's something that, again, okay, this is a good example. You say, what's, what's wrong with ChatGPT? Well, once it's been trained, that's it. It's not going to learn anything new. You turn it off and it's the old same ChatGPT. It's not going to, it hasn't learned anything from you. Right? So that's, that's another thing that we've got to change. If we want it to be an agent that's going to be able to absorb information that we give it and then use it in the future.

• 58:51Craig S. Smith?

  In today's fast-paced world, ensuring your AI systems are not only compliant, but also accurate and robust is key. How can you achieve reliable AI outcomes while managing risks effectively?

• 59:07Craig S. Smith?

  If you're facing challenges with AI risk management, having a solution that ensures accuracy in robustness is invaluable. Citrus X AI offers cutting-edge technology for you to detect and mitigate vulnerabilities, biases, and errors while ensuring accuracy, robustness, and compliance. Their solution is the only one that offers observability capabilities while helping you meet regulatory requirements. Citrusx.ai ensures your AI models are not just performing well, but navigating risk safely.

• 59:48Craig S. Smith?

  Elevate your AI model management with Citrus X. Visit email.citrusx.ai/eyeonai to book your free demo today and learn more about how their solution can benefit you. That's email, e-m-a-i-l, dot citrusx, c-i-t-r-u-s-x, dot ai, slash eyeonai, eyeonai, all run together, e-y-e-o-n-a-i, to book your free demo today.