Melanie Mitchell: Concepts, Analogies, Common Sense & Future of AI

Melanie Mitchell is a professor of computer science at Portland State University and an external professor at Santa Fe Institute. She has worked on and written about artificial intelligence from fascinating perspectives including adaptive complex systems, genetic algorithms, and the Copycat cognitive architecture which places the process of analogy making at the core of human cognition. From her doctoral work with her advisors Douglas Hofstadter and John Holland to today, she has contributed a lot of important ideas to the field of AI, including her recent book, simply called Artificial Intelligence: A Guide for Thinking Humans. This conversation is part of the Artificial Intelligence podcast. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter https://twitter.com/lexfridman, LinkedIn https://www.linkedin.com/in/lexfridman/, Facebook https://www.facebook.com/lexfridman, Medium https://medium.com/@lexfridman, or YouTube https://www.youtube.com/lexfridman where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts https://podcasts.apple.com/us/podcast/artificial-intelligence/id1434243584, follow on Spotify https://open.spotify.com/show/2MAi0BvDc6GTFvKFPXnkCL, or support it on Patreon https://www.patreon.com/lexfridman. This episode is presented by Cash App. Download it (App Store https://apps.apple.com/us/app/cash-app/id711923939, Google Play https://play.google.com/store/apps/details?id=com.squareup.cash&hl=en_US), use code “LexPodcast”. Episode Links: AI: A Guide for Thinking Humans (book) https://amzn.to/2Q80LbP Here’s the outline of the episode. On some podcast players you should be able to click the timestamp to jump to that time. 00:00 – Introduction 02:33 – The term “artificial intelligence” 06:30 – Line between weak and strong AI 12:46 – Why have people dreamed of creating AI? 15:24 – Complex systems and intelligence 18:38 – Why are we bad at predicting the future with regard to AI? 22:05 – Are fundamental breakthroughs in AI needed? 25:13 – Different AI communities 31:28 – Copycat cognitive architecture 36:51 – Concepts and analogies 55:33 – Deep learning and the formation of concepts 1:09:07 – Autonomous vehicles 1:20:21 – Embodied AI and emotion 1:25:01 – Fear of superintelligent AI 1:36:14 – Good test for intelligence 1:38:09 – What is complexity? 1:43:09 – Santa Fe Institute 1:47:34 – Douglas Hofstadter 1:49:42 – Proudest moment

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Lex Fridman?

  The following is a conversation with Melanie Mitchell. She's a professor of computer science at Portland State University and an external professor at Santa Fe Institute. She has worked on and written about artificial intelligence from fascinating perspectives, including adaptive complex systems, genetic algorithms, and the copycat cognitive architecture, which places the process of analogy making at the core of human cognition. From her doctoral work with her advisors Douglas Hofstadter and John Holland to today, she has contributed a lot of important ideas to the field of AI, including her recent book, simply called Artificial Intelligence: A Guide for Thinking Humans. This is the Artificial Intelligence Podcast.

• 0:45Lex Fridman?

  If you enjoy it, subscribe on YouTube, give it five stars on Apple Podcasts, support it on Patreon, or simply connect with me on Twitter at LexFriedman, spelled F R I D M A N. I recently started doing ads at the end of the introduction. I'll do one or two minutes after introducing the episode, and never any ads in the middle that can break the flow of the conversation. I hope that works for you and doesn't hurt the listening experience. I provide timestamps for the start of the conversation, but it helps if you listen to the ad and support this podcast by trying out the product or service being advertised. This show is presented by Cash App, the number one finance app in the App Store. I personally use Cash App to send money to friends,

• 1:30Lex Fridman?

  but you can also use it to buy, sell, and deposit Bitcoin in just seconds. Cash App also has a new investing feature. You can buy fractions of a stock, say $1 worth, no matter what the stock price is. Brokerage services are provided by Cash App Investing, a subsidiary of Square and a member SIPC. I'm excited to be working with Cash App to support one of my favorite organizations called FIRST, best known for their first robotics and Lego competitions. They educate and inspire hundreds of thousands of students in over 110 countries and have a perfect rating on Charity Navigator, which means that donated money is used to maximum effectiveness.

• 2:11Lex Fridman?

  When you get Cash App from the App Store or Google Play and use code LexPodcast, you'll get $10, and Cash App will also donate $10 to FIRST, which again, is an organization that I've personally seen inspire girls and boys to dream of engineering a better world. And now here's my conversation with Melanie Mitchell.

• 2:50Lex Fridman?

  The name of your new book is Artificial Intelligence, subtitle, A Guide for Thinking Humans. The name of this podcast is Artificial Intelligence. So let me take a step back and ask the old Shakespeare question about roses. What do you think of the term, artificial intelligence, for our big and complicated and interesting field?

• 3:12Speaker 01

  I'm not crazy about the term. I think it has a few problems because it means so many different things to different people. And intelligence is one of those words that isn't very clearly defined either. There's so many different kinds of intelligence, degrees of intelligence, approaches to intelligence. John McCarthy was the one who came up with the term artificial intelligence. And from what I read, he called it that to differentiate it from cybernetics, which was another related movement at the time. And he later regretted calling it artificial intelligence.

• 3:56Speaker 01

  Herbert Simon was pushing for calling it complex information processing, which got nixed, but you know, probably is equally vague, I guess.

• 4:09Lex Fridman?

  Is it the intelligence or the artificial in terms of words that's most problematic, would you say?

• 4:16Speaker 01

  Yeah, I think it's a little of both. But you know, it has some good size because... I personally was attracted to the field because I was interested in phenomenon of intelligence. And if it was called complex information processing, maybe I'd be doing something wholly different now.

• 4:33Lex Fridman?

  What do you think of, I've heard the term used cognitive systems, for example. So using cognitive.

• 4:39Speaker 01

  Yeah, I mean cognitive has certain associations with it and people like to separate things like cognition and perception, which I don't actually think are separate. But often people talk about cognition as being different from sort of other aspects of intelligence. It's sort of higher level.

• 4:59Lex Fridman?

  So to you cognition is this broad, beautiful mess of things that encompasses the whole thing. Memory.

• 5:06Speaker 01

  Yeah. I think it's hard to draw lines like that. When I was coming out of grad school in 1990, which is when I graduated, that was during one of the AI winters. And I was advised to not put AI, artificial intelligence, on my CV, but instead call it intelligent systems. So that was kind of an euphemism, I guess.

• 5:30Lex Fridman?

  What about to stick briefly on terms and words, the idea of artificial general intelligence? Or like Jan Le Cun prefers human level intelligence, sort of starting to talk about ideas that achieve higher and higher levels of intelligence. And somehow, artificial intelligence seems to be a term used more for the narrow, very specific applications of AI and sort of the, there's, what set of terms appeal to you to describe the thing that perhaps we strive to create?

• 6:12Speaker 01

  People have been struggling with this for the whole history of the field and defining exactly what it is that we're talking about. You know, John Searle had this distinction between strong AI and weak AI. And weak AI could be general AI, but his idea was, strong AI was the view that a machine is actually thinking, that as opposed to simulating thinking or.

• 6:41Speaker 01

  Carrying out intelligent processes that we would call intelligent.

• 6:48Lex Fridman?

  At a high level, if you look at the founding of the field, McCarthy and Searle and so on,

• 6:56Lex Fridman?

  are we closer to having a better sense of that line between narrow, weak AI and strong AI?

• 7:07Speaker 01

  Yes, I think we're closer to having a better idea of what that line is. Early on, for example, a lot of people thought that playing chess would be, you couldn't play chess if you didn't have sort of general human level intelligence. And of course, once computers were able to play chess better than humans, that revised that view. And people said, okay, well, maybe now we have to revise what we think of intelligence as. And so that's kind of been a theme throughout the history of the field is that once a machine can do some task,

• 7:51Speaker 01

  we then have to look back and say, oh, well, that changes my understanding of what intelligence is because I don't think that machine is intelligent. At least, that's not what I want to call intelligence.

• 8:02Lex Fridman?

  Do you think that line moves forever? Or will we eventually really feel as a civilization like we crossed the line if it's possible?

• 8:11Speaker 01

  It's hard to predict, but I don't see any reason why we couldn't, in principle, create something that we would consider intelligent. I don't know how we will know for sure. Maybe our own view of what intelligence is will be refined more and more until we finally figure out what we mean when we talk about it.

• 8:34Speaker 01

  But I, I think eventually we will create machines, in a sense, that have intelligence. They may not be the kinds of machines we have now. And one of the things that that's going to produce is, is making us sort of understand our own machine-like qualities, that we, in a sense, are mechanical in the sense that, like, cells, cells are kind of mechanical. They, they have algorithms, they process information by. And somehow, out of this mass of cells, we get this emergent property that we call intelligence.

• 9:18Speaker 01

  But underlying it is, uh, really just cellular processing and, and lots and lots and lots of it.

• 9:27Lex Fridman?

  Do you think we'll be able to, do you think it's possible to create intelligence without understanding our own mind? You said, sort of, in that process, we'll understand more and more. But do you think it's possible to sort of create without really fully understanding, from a mechanistic perspective, sort of, from a functional perspective, how our mysterious mind works?

• 9:50Speaker 01

  If I had to bet on it, I would say, no, we, we, we do have to understand our own minds, at least to some significant extent. But it, I, I think that's a really big, open question. I've been very surprised at how far, kind of, brute force approaches, based on, say, big data and huge networks, can, can take us. I wouldn't have expected that. And they have nothing to do with the way our minds work. So that's been surprising to me, so I could be wrong.

• 10:24Lex Fridman?

  To explore the psychological and the philosophical, do you think we're okay as a species with, uh, something that's more intelligent than us? Do you think perhaps the reason we're pushing that line further and further is we're afraid of acknowledging that there's something stronger, better, smarter than us humans?

• 10:46Speaker 01

  Well, I'm not sure we can define intelligence that way, because, you know, smarter than is...

• 10:53Speaker 01

  With with respect to what what, you know, computers are already smarter than us in some areas. They can multiply much better than we can. They they can figure out driving routes to take much faster and better than we can. They have a lot more information to draw on. They know about, you know, traffic conditions and all that stuff. So for any given particular task, sometimes computers are much better than we are, and we're totally happy with that. Right? I'm totally happy with that. I don't bother me at all. I guess the question is, you know, what which things about our intelligence would we feel very

• 11:36Speaker 01

  sad or upset that machines had been able to recreate? So in the book, I talk about my former PhD advisor, Douglas Hofstadter, who encountered a music generation program. And that was really the line for him, that if a machine could create beautiful music, that would be terrifying for him, because that is something he feels is really at the core of what it is to be human, creating beautiful music, art, literature. I, you know, I don't think he doesn't like the fact that machines can recognize spoken language

• 12:20Speaker 01

  really well. Like, he doesn't, he personally doesn't like using speech recognition. But I don't think it bothers him to his core, because it's like, okay, that's not at the core of humanity. But it may be different for every person, what what really they feel would usurp their humanity. And I think maybe it's a generational thing also. Maybe our children or our children's children will be adapted. They'll adapt to these new devices that can do all these tasks and say, yes, this thing is smarter than me in all these areas. But that's great, because it helps me.

• 13:04Lex Fridman?

  Looking at the broad history of our species, why do you think so many humans have dreamed of creating artificial life and artificial intelligence throughout the history of our civilization? So not just this century or the 20th century, but really many, throughout many centuries that preceded it?

• 13:23Speaker 01

  That's a really good question, and I have wondered about that. Because I, I myself, you know, was driven by curiosity about my own thought processes and thought it would be fantastic to be able to get a computer to mimic some of my thought processes. I'm not sure why we're so driven. I think we want to understand ourselves better, and we also want machines to do things for us. But I don't know, there's something more to it because it's so deep in the kind of

• 14:05Speaker 01

  mythology or the ethos of our, our species. And I don't think other species have this drive. So I don't know.

• 14:14Lex Fridman?

  If you were to sort of psychoanalyze yourself in your, in your own interest in AI, are you, what excites you about creating intelligence? You said, understanding our own selves?

• 14:27Speaker 01

  Yeah, I think that's what drives me particularly. I'm really interested in human intelligence. But I'm, I'm also interested in the sort of the phenomenon of intelligence more generally. And I don't think humans are the only thing with intelligence, you know, or even animals. But I think intelligence is a...

• 14:56Speaker 01

  Concept that encompasses a lot of complex systems. And if you think of things like insect colonies, or cellular processes, or the immune system, or all kinds of different biological, or even societal processes, have as an emergent property, some aspects of what we would call intelligence. You know, they have memory, they process information, they have goals, they accomplish their goals, et cetera. And to me, that, the question of what is this thing we're talking about here, was really fascinating to me, and exploring it using computers seemed to be a good way to approach the question.

• 15:41Lex Fridman?

  So do you think kind of of intelligence, do you think of our universe as a kind of hierarchy of complex systems, and then intelligence is just the property of any, you can look at any level, and every level has some aspect of intelligence. So we're just like one little speck in that giant hierarchy of complex systems?

• 16:01Speaker 01

  I don't know if I would say any system like that has intelligence. But I guess what I want to, I don't have a good enough definition of intelligence to say that.

• 16:14Lex Fridman?

  So let me, let me do sort of multiple choice, I guess. So, so you said ant colonies. So are ant colonies intelligent? Are the bacteria in our body intelligent? And then going to the physics world, molecules and the behavior at the quantum level of electrons and so on, is, are those kinds of systems, do they possess intelligence? Like, where's, where's the line that feels compelling to you?

• 16:44Speaker 01

  I don't know. I mean, I think intelligence is a continuum. And I think that the ability to, in some sense, have intention, have a goal, have, have a

• 16:57Speaker 01

  Some kind of self-awareness is part of it. So I'm not sure if, you know, it's hard to know where to draw that line. I think that's kind of a mystery. But I wouldn't say that, say that, you know, this, the planets orbiting the sun are, is an intelligent system. I mean, I would find that maybe not the right term to describe that. And this is, you know, there's all this debate in the field of like, what's, what's the right way to define intelligence? What's the right way to model intelligence? Should we think about computation? Should we think about dynamics? And should we think about, you know, free energy and all of that stuff?

• 17:40Speaker 01

  And I think that it's, it's a fantastic time to be in the field because there's so many questions and so much we don't understand. There's so much work to do.

• 17:51Lex Fridman?

  So are we, are we the most special kind of intelligence? In this kind of, you said there's a bunch of different elements and characteristics of intelligent systems and colonies. Is human intelligence the thing in our brain? Is that the most interesting kind of intelligence in this continuum?

• 18:14Speaker 01

  Well, it's interesting to us because, because it is us. I mean, interesting to me, yes. And because I'm part of, you know, human.

• 18:23Lex Fridman?

  But to understanding the fundamentals of intelligence, what I'm getting at, doing is studying the human, is sort of, if, everything we've talked about, what you talk about in your book, what, just the AI field, this notion, yes, it's hard to define, but it's usually talking about something that's very akin to human intelligence.

• 18:41Speaker 01

  Yeah, to me, it is the most interesting because it's the most complex, I think. It's the most self-aware. It's the only system, at least that I know of, that reflects on its own intelligence.

• 18:55Lex Fridman?

  And you talk about the history of AI and us in terms of creating artificial intelligence, being terrible at predicting the future with AI or with tech in general. So. Why do you think we're so bad at predicting the future? Are we hopelessly bad? So no matter what, whether it's this decade or the next few decades, every time we make a prediction, there's just no way of doing it well? Or as the field matures, we'll be better and better at it?

• 19:28Speaker 01

  I believe as the field matures, we will be better. And I think the reason that we've had so much trouble is that we have so little understanding of our own intelligence. So there's the famous story about Marvin Minsky assigning computer vision as a summer project to his undergrad students. And I believe that's actually a true story.

• 19:53Lex Fridman?

  Yeah, no, there's a write-up on it. Everyone should read. It's like a, I think it's like a proposal that describes everything that should be done in that project. It's hilarious because it, I mean, you could explain it, but from my sort of recollection, it describes basically all the fundamental problems of computer vision, many of which that still haven't been solved.

• 20:14Speaker 01

  Yeah, and I don't know how far they really expected to get. But I think that, and they're really, you know, Marvin Minsky is a super smart guy and very sophisticated thinker. But I think that no one really understands or understood, still doesn't understand how complicated, how complex the things that we do are because they're so invisible to us, you know, to us. Vision, being able to look out at the world and describe what we see, that's just immediate. It feels like it's no work at all. So it didn't seem like it would be that hard. But there's so much going on unconsciously, sort of invisible to us,

• 20:58Speaker 01

  that I think we overestimate how easy it will be to get computers to do it.

• 21:07Lex Fridman?

  And sort of for me to ask an unfair question, you've done research, you have thought about many different branches of AI through this book, Widespread, looking at where AI has been, where it is today.

• 21:23Lex Fridman?

  If you were to make a prediction, how many years from now would we as a society create something that you would say achieved human level intelligence? Or superhuman level intelligence?

• 21:40Speaker 01

  That is an unfair question.

• 21:42Lex Fridman?

  A prediction that will most likely be wrong. So but it's just your notion because

• 21:47Speaker 01

  Okay, I'll say I'll say more than 100 years.

• 21:51Lex Fridman?

  More than 100 years.

• 21:52Speaker 01

  And there I quoted somebody in my book who said that human level intelligence is 100 Nobel prizes away. Which I like because it's a it's a nice way to to sort of it's a nice unit for prediction. And it's like that many fantastic discoveries have to be made. And of course, there's no Nobel Prize in AI, not yet at least.

• 22:20Lex Fridman?

  If we look at that 100 years, your sense is really the journey to intelligence has to go through something something more complicated. That's akin to our own cognitive systems. Understanding them, being able to create them in in the artificial systems as opposed to sort of taking the machine learning approaches of today. And really scaling them and scaling them and scaling them exponentially with both compute and hardware and and data.

• 22:55Speaker 01

  That would be my that would be my guess. You know, I think that in in the sort of going along in the narrow AI that these current the current approaches will get better. You know, I think there's some fundamental limits to how far they're going to get. I might be wrong, but that's what I think. But and there's some fundamental weaknesses that they have that I talk about in the book that that just comes from this approach of of supervised learning.

• 23:34Speaker 01

  Requiring sort of feed forward networks and so on. It's just, I don't think it's a sustainable approach to understanding the world.

• 23:51Lex Fridman?

  Yeah, I'm personally torn on it. Sort of I have everything you write about in the book and sort of what we're talking about now. I agree, I agree with you, but I'm more and more, depending on the day. First of all, I'm deeply surprised by the success of machine learning and deep learning in general. And from the very beginning, when I was, it's really been my main focus of work. I'm just surprised how far it gets. And I'm also think we're really early on in these efforts of these narrow AI. So I think there'll be a lot of surprise of how far it gets. I think we'll be extremely impressed. Like my sense is everything I've seen so far, and we'll talk about autonomous driving and so on.

• 24:36Lex Fridman?

  I think we can get really far. But I also have a sense that we will discover, just like you said, is that even though we'll get really far in order to create something like our own intelligence is actually much farther than we realize.

• 24:51Speaker 01

  Right.

• 24:51Lex Fridman?

  I think these methods are a lot more powerful than people give them credit for, actually. So that of course, there's the media hype, but I think there's a lot of researchers in the community, especially like not undergrads, right? But like people who've been in AI, they're skeptical about how far deep learning can get. And I'm more and more thinking that it can actually get farther than they'll realize.

• 25:14Speaker 01

  It's certainly possible. One thing that surprised me when I was writing the book is how far apart different people are in the field are on their opinion of how far the field has come and what is accomplished and what's going to happen next.

• 25:28Lex Fridman?

  What's your sense of the different, who are the different people, groups, mindsets, thoughts in the community about where AI is today?

• 25:40Speaker 01

  Yeah, they're all over the place. So there's, there's kind of the, the singularity, transhumanism group. I don't know exactly how to characterize that approach, which is

• 25:50Lex Fridman?

  First of all,

• 25:51Speaker 01

  Yeah, the sort of exponential, exponential progress. We're, we're on the sort of almost at the, the hugely accelerating part of the exponential. And by, in the next 30 years, we're going to see super intelligent AI and all that. And we'll be able to upload our brains and that. So there, there's that kind of extreme view that most, I think most people who work in AI don't have. They disagree with that. But there are people who, who are maybe don't, aren't, you know, singularity people, but, but they're, they

• 26:32Speaker 01

  do think that the current approach of deep learning is going to scale and is going to kind of go all the way, basically, and take us to true AI or human level AI, or whatever you want to call it. And there's quite a few of them. And a lot of them, like a lot of the people I've met who work at big tech companies in, in AI groups, kind of have this view that we're really not that far, you know?

• 27:03Lex Fridman?

  Just to linger on that point, sort of, if, if I can take as an example, like Yann LeCun, I don't know if you know about his work and so, or his viewpoints on this.

• 27:11Speaker 01

  I do.

• 27:12Lex Fridman?

  He believes that there's a bunch of breakthroughs, like fundamental, like Nobel Prizes, there's needed still.

• 27:17Speaker 01

  Right.

• 27:18Lex Fridman?

  But I think he thinks those breakthroughs will be built on top of deep learning.

• 27:23Speaker 01

  Right.

• 27:23Lex Fridman?

  And then there's some people who think we need to kind of put deep learning to the side a little bit as just one module that's helpful in the bigger cognitive framework.

• 27:34Speaker 01

  Right. So, so, so I think, so from what I understand, Yann LeCun is rightly saying supervised learning is not sustainable. We have to figure out how to do unsupervised learning, that that's going to be the key. And

• 27:53Speaker 01

  You know, I think that's probably true. I think unsupervised learning is going to be harder than people think. I mean, the way that we humans do it. Then there's the opposing view, you know, there's the Gary Marcus kind of hybrid view where deep learning is one part, but we need to bring back kind of these symbolic approaches and combine them. Of course, no one knows how to do that very well.

• 28:23Lex Fridman?

  Which is the more important part.

• 28:25Speaker 01

  Right.

• 28:26Lex Fridman?

  To emphasize and how do they, yeah, how do they fit together? What's, what's the foundation? What's the thing that's on top?

• 28:32Speaker 01

  Yeah.

• 28:33Lex Fridman?

  What's the cake? What's the icing?

• 28:35Speaker 01

  Right. Then there's people pushing different, different things. There's the people, the causality people who say, you know, deep learning as it's formulated today completely lacks any notion of causality. And that's, dooms it. And therefore, we have to somehow give it some kind of notion of causality. There's a lot of push from the more cognitive science crowd saying, we have to look at developmental learning, we have to look at how babies learn, we have to look at intuitive physics, all these things we know about physics.

• 29:20Speaker 01

  And as somebody kind of quipped, we also have to teach machines intuitive metaphysics, which means like, objects exist. Causality exists. You know, these things that maybe we're born with. I don't know. That, that they don't have, the machines don't have any of that. You know, they look at a group of pixels and they, maybe they get 10 million examples, but they, they can't necessarily learn that there are objects in the world.

• 29:55Speaker 01

  So there's just a lot of pieces of the puzzle that people are promoting. And with different opinions of like, how important they are and how close we are to being able to put them all together to create general intelligence.

• 30:11Lex Fridman?

  Looking at this broad field, what do you take away from it? Who is the most impressive? Is it the cognitive folks, the Gary Marcus camp, the Jan camp? Unsupervised, there's self-supervised, there's the supervised. And then there's the engineers who are actually building systems. You have sort of the Andre Karpathy at Tesla, building actual, you know, it's not philosophy, it's real, like, systems that operate in the real world. What, yeah, what do you take away from all this beautiful variety?

• 30:41Speaker 01

  I don't know, you know, these, these different views are not necessarily mutually exclusive. And I, I think people like Jan LaKoon agrees with the developmental psychology, causality, intuitive physics, et cetera. But he still thinks that it's learning, like, end to end learning is the way to go.

• 31:05Lex Fridman?

  Will take us perhaps all the way.

• 31:07Speaker 01

  Yeah, and that we don't need, there's no sort of innate stuff that has to get built in. This is, you know, it's because, it's a hard problem. I personally, you know, I'm very sympathetic to the cognitive science side, because that's kind of where I came into the field. I've become more and more sort of an embodiment adherent, saying that, you know, without having a body, it's going to be very hard to learn what we need to learn about the world.

• 31:41Lex Fridman?

  That's definitely something I'd love to talk about in a little bit. To step into the cognitive world, then, if you don't mind, because you've done so many interesting things. If you look to Copycat, taking a couple of decades, step back, you, Douglas Hassetter, and others have created and developed Copycat. More than thirty years ago.

• 32:05Speaker 01

  That's painful to hear.

• 32:08Lex Fridman?

  So what is it? What is, what is Copycat?

• 32:11Speaker 01

  It's a program that makes analogies in an idealized domain, idealized world of letter strings. So as you say, thirty years ago, wow. So I started working on it when I started grad school in nineteen eighty-four. Wow. Dates me.

• 32:35Speaker 01

  And it's based on Doug Hofstadter's ideas that about that analogy is really a core aspect of thinking.

• 32:47Speaker 01

  I remember he has a really nice quote in the book by, by himself and Emmanuel Sander called Surfaces and Essences. I don't know if you've seen that book, but it's, it's about analogy. And he says, without concepts, there can be no thought and without analogies, there can be no concepts. So the view is that analogy is not just this kind of reasoning technique where we go, you know,

• 33:15Speaker 01

  shoe is to foot as glove is to what, you know, these kinds of things that we have on IQ tests or whatever, that, but that it's much deeper, it's much more pervasive in everything we do, in every, our language, our, our thinking, our perception. So we, so he had a view that was a very active perception idea. So the idea was that

• 33:41Speaker 01

  instead of having kind of what, a passive

• 33:46Speaker 01

  network in which you have input that's being processed through these feed forward layers and then there's an output at the end, that perception is really a dynamic process. You know, where like our eyes are moving around and they're getting information and that information is feeding back to what we look at next influences what we look at next and how we look at it. And so Copycat was trying to do that, kind of simulate that kind of idea where you have these.

• 34:18Speaker 01

  Agents, it's kind of an agent-based system. And you have these agents that are picking things to look at and deciding whether they were interesting or not, whether they should be looked at more. And that would influence other agents.

• 34:33Lex Fridman?

  And how do they interact?

• 34:34Speaker 01

  So they interacted through this global, kind of what we call the workspace. So it's actually inspired by the old blackboard systems where you would have agents that post information on a blackboard, a common blackboard. This is like old, very old fashioned AI.

• 34:50Lex Fridman?

  Is that, are we talking about like in physical space? Is this a computer program?

• 34:54Speaker 01

  It's a computer program.

• 34:55Lex Fridman?

  So agents posting concepts on a blackboard?

• 34:59Speaker 01

  Yeah, we called it a workspace. And it, it, it, it's, the workspace is a data structure. The agents are little pieces of code that, you could think of them as detect, little detectors or little filters that say, I'm going to pick this place to look and I'm going to look for a certain thing. And is this the thing I, I think is important? Is it there? So it's almost like, you know, convolution in a way, except a little bit more general and saying, and then highlighting it on the, on the, in the workspace.

• 35:30Lex Fridman?

  Well, what's, once it's in the workspace, how do the things that are highlighted relate to each other? Like what's?

• 35:37Speaker 01

  So there's different kinds of agents that can build connections between different things. So, so just to give you a concrete example, what Copycat did was it, it made analogies between strings of letters. So here's an example. ABC changes to ABD. What does IJK change to? And the program had some prior knowledge about the alphabet. It knew the sequence of the alphabet. It, you know, had a concept of letter, successor of letter. It had concepts of sameness. So it has some innate things programmed in. But then it could do things like, say, discover that.

• 36:19Speaker 01

  ABC is a group of letters in succession. And then some, an agent can mark that.

• 36:28Lex Fridman?

  So the idea that there could be a sequence of letters, is that a new concept that's formed? Or that's a concept that's

• 36:36Speaker 01

  That's a concept that's innate.

• 36:38Lex Fridman?

  Sort of, can you form new concepts? Or are all concepts innate?

• 36:42Speaker 01

  So in this program, all the concepts of the program were innate. So, because, because we weren't, I mean, obviously, that limits it quite a, quite a bit. But what we were trying to do is say, suppose you have some innate concepts. How do you flexibly apply them to new situations?

• 37:02Lex Fridman?

  Right.

• 37:02Speaker 01

  And how do you make analogies?

• 37:05Lex Fridman?

  Let's step back for a second. So I really like that quote, that you said, without concepts, there could be no thought. And without analogies, there can be no concepts. In a, in a Santa Fe presentation, you said that it should be one of the mantras of AI.

• 37:18Speaker 01

  Yes.

• 37:19Lex Fridman?

  And that you also yourself said, how to form and fluidly use concept is the most important open problem in AI.

• 37:27Speaker 01

  Yes.

• 37:28Lex Fridman?

  How to form and fluidly use concepts is the most important open problem in AI. So let's, what is a concept? And what is an analogy?

• 37:40Speaker 01

  A concept is in some sense, a fundamental unit of thought. So say we have a concept of

• 37:52Speaker 01

  a dog. Okay.

• 37:55Speaker 01

  And a concept is embedded in a whole space of concepts. So that there's certain concepts that are closer to it or farther away from it.

• 38:07Lex Fridman?

  Are these concepts, are they really like fundamental, like we mentioned innate, almost like exo matic, like very basic? And then there's other stuff built on top of it? Or does this include everything? Is, are they complicated? Like

• 38:21Speaker 01

  You can certainly have, form new concepts.

• 38:24Lex Fridman?

  Right, I guess that's the question I'm asking.

• 38:25Speaker 01

  Yeah.

• 38:26Lex Fridman?

  Can you form new concepts that are combine, complex combinations of other concepts?

• 38:31Speaker 01

  Yes, absolutely. And that's kind of what we, we do in learning.

• 38:37Lex Fridman?

  And then what's the role of analogies in that?

• 38:40Speaker 01

  So analogy is when you recognize that one situation

• 38:47Speaker 01

  Is essentially the same as another situation. And essentially is kind of the key word there because it's not the same. So if I say, um, last week I did a podcast interview in, actually like three days ago in Washington DC. And that situation was very similar to this situation, although it wasn't exactly the same. You know, it was a different person sitting across from me. We had different kinds of microphones. The questions were different. The building was different. There's all kinds of different things, but really it was analogous.

• 39:27Speaker 01

  Or I can say, so, so, so doing a podcast interview, that's kind of a concept. It's a new concept. You know, I never had that concept before. Essentially, I mean, and I can make an analogy with it, like being interviewed for a news article in a newspaper. And I can say, well, you kind of play the same role that the, the newspaper reporter played. It's not exactly the same because maybe they actually emailed me some written questions rather than talking. And the writing, the written questions play the, you know, are analogous to your spoken questions. You know, there's, there's just all kinds of similarities.

• 40:12Lex Fridman?

  And this somehow probably connects to conversations you have over Thanksgiving dinner, just general conversations. You could, there's like a thread you can probably take that just stretches out in all aspects of life that connect to this podcast.

• 40:25Speaker 01

  I mean, Sure.

• 40:26Lex Fridman?

  Conversations between humans.

• 40:28Speaker 01

  Sure. And if, and if I go and tell a friend of mine about this podcast interview, my friend might say, oh, the same thing happened to me. You know, let's say, you know, you asked me some really hard question and I have trouble answering it. My friend could say, the same thing happened to me, but it was like, it wasn't a podcast interview. It wasn't a, a.

• 40:54Speaker 01

  It was a completely different situation. And yet, my friend is seeing essentially the same thing. You know, we say that very fluidly, the same thing happened to me.

• 41:05Lex Fridman?

  Essentially, the same thing.

• 41:07Speaker 01

  But we don't even say that, right?

• 41:08Lex Fridman?

  Right, do imply it, yes.

• 41:10Speaker 01

  Yeah. And the view that kind of went into, say, copycat, that whole thing is that that act of saying the same thing happened to me is making an analogy. And in some sense, that's what's underlies all of our concepts.

• 41:27Lex Fridman?

  Why do you think analogy making that you're describing is so fundamental to cognition? Like, it seems like it's the main element action of what we think of as cognition.

• 41:41Speaker 01

  Yeah, so it can be argued that all of this generalization we do of concepts and recognizing concepts in different situations is done by analogy. That that's... Every time I'm recognizing that, say, you're a person, that's by analogy, because I have this concept of what person is, and I'm applying it to you. And every time I recognize a new situation, like, one of the things I talked about in the book was the concept of walking a dog.

• 42:27Speaker 01

  That that's actually making an analogy, because all of that, you know, the details are very different.

• 42:32Lex Fridman?

  So reasoning could be reduced down to sensory analogy making. So all the things we think of as like, yeah, like you said, perception. So what's perception is taking raw sensory input and it's somehow integrating into our our understanding of the world, updating the understanding. And all of that has just this giant mess of analogies that are being made.

• 42:57Speaker 01

  I think so, yeah.

• 42:59Lex Fridman?

  If you could just linger on it a little bit, like, what do you think it takes to engineer a process like that for us in our artificial systems?

• 43:09Speaker 01

  We need to understand better, I think, how we do it, how humans do it.

• 43:20Speaker 01

  And it comes down to internal models, I think. You know, people talk a lot about mental models, that concepts are mental models, that I can, in my head, I can do a simulation of a situation like walking a dog. And that, there's some work in psychology that promotes this idea that all of concepts are really mental simulations, that whenever you encounter a concept or situation in the world, or you read about it, or whatever, you do some kind of mental simulation that allows you to predict what's going to happen, to develop expectations of what's going to happen.

• 44:05Speaker 01

  So that's the kind of structure I think we need, is that kind of mental model that, and in our brains somehow these mental models are very much interconnected.

• 44:18Lex Fridman?

  Again, so a lot of stuff we're talking about are essentially open problems, right? So if I ask a question, I don't mean that you would know the answer, only just hypothesizing. But how big do you think is the network graph data structure of concepts that's in our head? Like if we're trying to build that ourselves, like it's, we take it, that's one of the things we take for granted, we think, I mean, that's why we take common sense for granted. We think common sense is trivial. But how big of a thing of concepts is, that underlies what we think of as common sense, for example?

• 45:01Speaker 01

  Yeah, I don't know. And I'm not, I don't even know what units to measure it in. When you say how big is it?

• 45:07Lex Fridman?

  That's beautifully put, right? But...

• 45:10Speaker 01

  But, you know, we have, you know, it's really hard to know. We have...

• 45:15Speaker 01

  What, 100 billion neurons or something? I don't know. And they're connected via trillions of synapses. And there's all this chemical processing going on. There's just a lot of capacity for stuff. And there are informations encoded in different ways in the brain. It's encoded in chemical interactions. It's encoded in electric, like firing and firing rates. And nobody really knows how it's encoded. But it just seems like there's a huge amount of capacity. So I think it's huge. It's just enormous. And it's amazing how much stuff we know.

• 45:54Lex Fridman?

  Yeah. But we know, and not just know like facts, but it's all integrated into this thing that we can make analogies with.

• 46:03Speaker 01

  Yes.

• 46:04Lex Fridman?

  There's a dream of semantic web. And there's a lot of dreams from expert systems of building giant knowledge bases. Do you see a hope for these kinds of approaches of building, of converting Wikipedia into something that could be used in analogy making?

• 46:22Speaker 01

  Sure. And I think people have made some progress along those lines. I mean, people have been working on this for a long time. But the problem is, and this, I think, is the problem of common sense. Like, people have been trying to get these common sense networks. Here at MIT, there's this ConceptNet project, right? But the problem is that, as I said, most of the knowledge that we have is invisible to us. It's not in Wikipedia. It's very basic things about, you know, intuitive physics, intuitive psychology, intuitive metaphysics, all that stuff.

• 47:04Lex Fridman?

  If you were to create a website that described intuitive physics, intuitive psychology, would it be bigger or smaller than Wikipedia? What do you think?

• 47:16Speaker 01

  I guess describe to whom? I'm sorry, but.

• 47:21Lex Fridman?

  That's no, that's very really good. I exactly right. Yeah.

• 47:24Speaker 01

  That's a hard question because, you know, how do you represent that knowledge is the question. Right? I can certainly write down F equals M A. And Newton's laws and a lot of physics can be deduced from that.

• 47:40Speaker 01

  But that's probably not the best representation of that knowledge for for doing the kinds of reasoning we want a machine to do.

• 47:53Speaker 01

  So, so I don't know. It's impossible to say now. And people, you know, the projects like there's a famous, the famous Psych project, right? That Douglas Leonard did that was trying.

• 48:07Lex Fridman?

  Still going.

• 48:08Speaker 01

  I think it's still going. And it's the idea was to try and encode all of common sense knowledge, including all this invisible knowledge in some kind of logical representation. And it just never, I think, could do any of the things that he was hoping it could do because that's just the wrong approach.

• 48:31Lex Fridman?

  Of course, that's what they always say, you know, and then the history books will say, well, the Psych project finally found a breakthrough in 2058 or something. And, you know, so much progress has been made in just a few decades. That who knows what the next breakthroughs will be.

• 48:49Speaker 01

  It could be.

• 48:50Lex Fridman?

  It's certainly a compelling notion what the Psych project stands for.

• 48:54Speaker 01

  I think Leonard was one of the earliest people to say common sense is what we need. That's what we need. All this like expert system stuff, that is not going to get you to AI. You need common sense. And he basically gave up his whole academic career to go pursue that. And I totally admire that. But I think that the approach itself will not in 2020 or 2040. Or whatever.

• 49:26Lex Fridman?

  What do you think is wrong with the approach? What kind of approach would might be successful?

• 49:31Speaker 01

  Well.

• 49:32Lex Fridman?

  Again, nobody knows the answer, right?

• 49:34Speaker 01

  If I knew that, you know, one of my talks, one of the people in the audiences, a public lecture, one of the people in the audience said, what AI companies are you investing in?

• 49:45Lex Fridman?

  Investment advice? Okay.

• 49:47Speaker 01

  I'm a college professor, for one thing, so I don't have a lot of extra funds to invest. But also, like, no one knows what's going to work in AI, right? That's the problem.

• 49:58Lex Fridman?

  Let me ask another impossible question in case you have a sense. In terms of data structures that will store this kind of information, do you think they've been invented yet, both in hardware and software? Or is something else needs to be, are we totally, you know?

• 50:15Speaker 01

  I think something else has to be invented. I, that's my guess.

• 50:20Lex Fridman?

  Is the breakthroughs that's most promising, would that be in hardware or in software?

• 50:27Lex Fridman?

  Do you think we can get far with the current computers? Or do we need to do something that is?

• 50:32Speaker 01

  I see what you're saying. I don't know if Turing computation is going to be sufficient. Probably, I would guess it will. I don't see any reason why we need anything else. But so, so in that sense, we have invented the hardware we need, but we just need to make it faster and bigger. And we need to figure out the right algorithms and the right sort of architecture.

• 50:56Lex Fridman?

  Turing, that's a very mathematical notion. When we try, have to build intelligence, it's now an engineering notion where you throw all that stuff.

• 51:05Speaker 01

  Well, I guess, I guess it is, it is a question. That, that, people have brought up this question, you know? And when you asked about, like, is our current hardware,

• 51:18Speaker 01

  will, will our current hardware work? Well, Turing computation says that, like, our current hardware.

• 51:26Speaker 01

  Is in principle a Turing machine, right?

• 51:29Lex Fridman?

  Yes.

• 51:30Speaker 01

  So all we have to do is make it faster and bigger. But there have been people like Roger Penrose, if you might remember that he said Turing machines cannot produce intelligence because intelligence requires continuous valued numbers. I mean, that was sort of my reading of his argument and quantum mechanics and what else, whatever, you know. But I don't see any evidence for that, that we need new computation paradigms. But I don't know if we're, you know, I don't think we're going to be able to scale up our current approaches to programming these computers.

• 52:14Lex Fridman?

  What is your hope for approaches like Copycat or other cognitive architectures? I've talked to the creator of Saur, for example. I've used Actar myself. I don't know if you're familiar with that.

• 52:24Speaker 01

  Yeah, I am.

• 52:25Lex Fridman?

  What do you, what do you think is, what's your hope of approaches like that in helping develop systems of greater and greater intelligence in the coming decades?

• 52:35Speaker 01

  Well, that's what I'm working on now, is trying to take some of those ideas and extending it. So I think there are some really promising approaches that are going on now that have to do with more active generative models. So this is the idea of this simulation in your head, a concept when you, if you want to, when you're perceiving a new, a new situation, you have some simulations in your head. Those are generative models. They're generating your expectations, they're generating predictions.

• 53:13Lex Fridman?

  That's part of a perception. You have a math model that generates a prediction, then you compare it with.

• 53:19Speaker 01

  Yeah.

• 53:19Lex Fridman?

  And then the difference somehow forms.

• 53:21Speaker 01

  And you also, that, that generative model is telling you where to look and what to look at and what to pay attention to. And it, I think it affects your perception. It's not that just you compare it with your perception. It. It becomes your perception in a way. It it it's kind of a mixture of of the bottom up information coming from the world and your top down model being imposed on the world is what becomes your perception.

• 53:53Lex Fridman?

  So your hope is something like that can improve perception systems and that they can understand things better.

• 53:58Speaker 01

  Yes.

• 53:59Lex Fridman?

  Understand things.

• 54:00Speaker 01

  Yes.

• 54:01Lex Fridman?

  What's the what's the step? What's the analogy making step there?

• 54:06Speaker 01

  Well, there the the the idea is that you have this pretty complicated conceptual space. You know, you can talk about a semantic network or something like that with these different kinds of concept models in your brain that are connected. So so let's let's take the example of walking a dog. So we were talking about that. Okay. Let's say I see someone out in the street walking a cat. Some people walk their cats, I guess.

• 54:35Lex Fridman?

  Yes.

• 54:35Speaker 01

  Seems like a bad idea, but.

• 54:37Lex Fridman?

  Yeah.

• 54:38Speaker 01

  So my model, my, you know, there's connections between my model of a dog and model of a cat. And I can immediately see the analogy of that those are analogous situations. But I can also see the differences and that tells me what to expect. So also, you know, I have a new situation. So another example with the walking the dog thing is sometimes people I see people riding their bikes with a leash, holding a leash and the dog's running alongside. Okay, so I know that the I recognize that as kind of a dog walking situation, even though the person's not walking.

• 55:23Speaker 01

  Right. And the dog's not walking. Because I I have the these these models that say, okay, riding a bike is sort of similar to walking or it's connected. It's a means of transportation. But I because they have their dog there, I assume they're not going to work, but they're going out for exercise. And, you know, the these analogies help me to figure out kind of what's going on, what's likely.

• 55:50Lex Fridman?

  But sort of these analogies are very human interpretable. So that's that kind of space. And then you look at something like the current deep learning approaches, they kind of help you to take raw censoring information and to sort of automatically build up hierarchies of, of, you can even call them concepts. They're just not human interpretable concepts. What's your, what's the link here? Do you hope?

• 56:19Lex Fridman?

  It's sort of the hybrid system question. How do you think that two can start to meet each other? What's the value of learning in this systems of forming of analogy making?

• 56:33Speaker 01

  The, the, the goal of, you know, the original goal of deep learning in, in at least visual perception was that you would get the system to learn to extract features that at these different levels of complexity. So maybe edge detection and that would lead into learning, you know, simple combinations of edges and then more complex shapes and then whole objects or faces. And this was based on the, the ideas of the neuroscientists Hubel and Wiesel, who had seen, laid out this kind of structure in brain.

• 57:16Speaker 01

  And I think that is, that's right to some extent. Of course, people have come, found that the whole story is a little more complex than that and the brain, of course, always is and there's a lot of feedback.

• 57:29Speaker 01

  So I see that

• 57:33Speaker 01

  as, as absolutely a, a good brain inspired approach to some aspects of perception. But one thing that it's lacking.

• 57:46Speaker 01

  For example, is all of that feedback, which is extremely important.

• 57:50Lex Fridman?

  The interactive element that you mentioned.

• 57:53Speaker 01

  The expectation, right? The conceptual level.

• 57:56Lex Fridman?

  Go back and forth with the expectation, the perception, and just go back and forth.

• 58:01Speaker 01

  So, right, so that is extremely important. And, you know, one thing about deep neural networks is that in a given situation, like, you know, they're trained, right? They get these weights and everything. But then now I give them a new, a new image, let's say.

• 58:19Lex Fridman?

  Yes.

• 58:19Speaker 01

  They

• 58:22Speaker 01

  treat every part of the image in the same way. You know, they apply the same filters at each layer to all parts of the image. There's no feedback to say like, oh, this part of the image is irrelevant.

• 58:37Lex Fridman?

  Right.

• 58:38Speaker 01

  I shouldn't care about this part of the image. Or this part of the image is the most important part. And that's kind of what we humans are able to do because we have these conceptual expectations.

• 58:50Lex Fridman?

  There's, by the way, a little bit of work in that. There's certainly a lot more in what's under the called attention in natural language processing now. It's, it's a and that's exceptionally powerful. And it's a very, just as you say, is a really powerful idea. But again, in sort of machine learning, it all kind of operates in an automated way. That's not human.

• 59:13Speaker 01

  Well, it's not, it's not also, okay, so that, yeah, right. It's not dynamic. I mean, in the sense that as a perception of a new example is being processed, those attentions weights don't change.

• 59:29Lex Fridman?

  Right, so, I mean, there's a there's a kind of notion that there's not a memory. So you're not aggregating the idea of like this mental model.

• 59:42Speaker 01

  Yes.

• 59:43Lex Fridman?

  I mean, that seems to be a fundamental idea. There's not a really powerful, I mean, there's some stuff with memory, but there's not a powerful way to represent. The world in some sort of way that's deeper than, I mean, it's so difficult because, you know, neural networks do represent the world. They do have a mental model, right? But it just seems to be shallow. It's hard to, it's hard to criticize them at the fundamental level, to me at least. It's easy to, it's, it's easy to criticize them. Well, look, like exactly what you're saying, mental models, sort of almost from a, put a psychology hat

• 1:00:28Lex Fridman?

  on, say, look, these networks are clearly not able to achieve what we humans do with forming mental models, analogy making, so on. But that doesn't mean that they fundamentally cannot do that. Like you can't, it's very difficult to say that. I mean, at least to me. Do you have a notion that the learning approaches really, I mean, they're going to, not, not only are they limited today, but they will forever be limited in being able to construct such mental models?

• 1:00:59Speaker 01

  I think the idea of the dynamic perception is key here. The idea that moving your eyes around and getting feedback. And that's something that, you know, there's been some models like that. There's certainly recurrent neural networks that operate over several time steps. But the problem is that the actual, the recurrence is, you know, basically the, the feedback is to, at the next time step is the entire hidden state of the network, which, which is, and it turns out that that's

• 1:01:45Speaker 01

  That doesn't work very well.

• 1:01:47Lex Fridman?

  But see, the thing I'm saying is mathematically speaking, it has the information in that recurrence to capture everything. It just doesn't seem to work.

• 1:01:58Speaker 01

  Yeah, right.

• 1:01:59Lex Fridman?

  So like my, you know, it's like, it's the same touring machine question, right? Yeah, maybe theoretically, it computers, anything that's touring, a universal touring machine can, can be intelligent. But practically, the architecture might be have be very specific kind of architecture to be able to create it. So just, I guess it sort of ask almost the same question again is how big of a role do you think deep learning needs, will play or needs to play in this in perception?

• 1:02:38Speaker 01

  I think that deep learning as it's currently, as it currently exists, you know, will play that kind of thing will play some role. And but I think that there's a lot more going on in perception. But who knows, you know, the definition of deep learning? I mean, it's pretty broad. It's kind of an umbrella for a lot of different things.

• 1:03:01Lex Fridman?

  What I mean is purely sort of neural networks.

• 1:03:03Speaker 01

  Yeah, and a feed forward neural networks.

• 1:03:05Lex Fridman?

  Essentially, or there could be recurrence, but.

• 1:03:09Speaker 01

  Yeah.

• 1:03:10Lex Fridman?

  Sometimes it feels like, before I started to talk to Gary Marcus, it feels like the criticism of deep learning is kind of like us birds criticizing airplanes for not flying well, or that they're not really flying. Do you think deep learning, do you think it could go all the way? Like John Lippman thinks? Do you think that, yeah, the brute force learning approach can go all the way?

• 1:03:39Speaker 01

  I don't think so, no. I mean, I think it's an open question, but I tend to be on the innateness side that there has that there's some things that.

• 1:03:51Speaker 01

  We've been evolved to be able to learn. And that learning just can't happen without them. So one example, here's an example I had in the book that I think is useful to me, at least in thinking about this. So this has to do with the Deep Minds Atari game playing program. Okay. And it learned to play these Atari video games just by getting input from the pixels of the screen. And it learned to play the game Breakout thousand percent better than humans.

• 1:04:35Speaker 01

  Okay, that was one of their results and it was great. And it learned this thing where it tunneled through the side of the bricks in the Breakout game and the ball could bounce off the ceiling and then just wipe out bricks. Okay, so there was a group who did an experiment where they took the paddle, you know, that you move with the joystick and moved it up two pixels or something like that. And then they, they looked at a deep Q learning system that had been trained on Breakout and said, could it now transfer its learning to this new version of the game? Of course, a human could, but, and it couldn't. Maybe that's not surprising, but I guess the point is it hadn't learned the concept

• 1:05:20Speaker 01

  of a paddle. It hadn't learned that, it hadn't learned the concept of a ball or the concept of tunneling. It was learning something, you know, we caught, we, looking at it, kind of anthropomorphized it and said, oh, here's what it's doing and the way we describe it. But it actually didn't learn those concepts. And so because it didn't learn those concepts, it couldn't make this transfer.

• 1:05:44Lex Fridman?

  Yeah, so that's a beautiful statement, but at the same time, by moving the paddle, we also anthropomorphize flaws to inject into the system that will then flip how impressed we are by it. What I mean by that is, to me, the Atari games were, to me, deeply impressive that that was possible at all. So like, I have to first pause on that, and people should look at that, just like the game of Go, which is fundamentally different to me than what Deep Blue did. Even though there's still Monte Carlo, there's still tree search, it's just everything Deep Mind has done in terms of learning, however limited it is, is still deeply surprising to me.

• 1:06:29Speaker 01

  Yeah, I'm not, I'm not trying to say that what they did wasn't impressive. I think it was incredibly impressive.

• 1:06:35Lex Fridman?

  To me, it's interesting, is moving the board just another thing that needs to be learned? So like, we've been able, maybe, maybe, been able to, through the current neural networks, learn very basic concepts that are not enough to do this general reasoning. And maybe with more data, I mean, the data, you know, the interesting thing about the examples that you talk about, and beautifully, is they, it's often flaws of the data.

• 1:07:05Speaker 01

  Well, that's the question. I mean, I, I think that is the key question, whether it's a flaw of the data or not.

• 1:07:11Lex Fridman?

  Or the mechanism.

• 1:07:12Speaker 01

  Because, because the reason I brought up this example was because you were asking, do I think that, you know, learning from data could go all the way?

• 1:07:19Lex Fridman?

  Yes.

• 1:07:20Speaker 01

  And that this was why I brought up the example, because I think, and this was, is not at all to, to take away from the impressive work that they did, but it's to say that when we look at what these systems learn.

• 1:07:35Speaker 01

  Do they learn the human, the things that we humans consider to be the relevant concepts? And in that example, it didn't.

• 1:07:46Lex Fridman?

  Yes.

• 1:07:47Speaker 01

  Sure, if you train it on moving, you know, the paddle being in different places, maybe it could deal with, maybe it would learn that concept. I'm not totally sure. But the question is, you know, scaling that up to more complicated worlds, to what extent could a machine that only gets this very raw data learn to divide up the world into relevant concepts? And I don't know the answer, but I would bet that, that without some innate notion, that it can't do it.

• 1:08:27Lex Fridman?

  Yeah. 10 years ago, I 100% agree with you as the most expert in AI system, but now I have one, like, I have a glimmer of hope.

• 1:08:37Speaker 01

  Okay. I mean, that's fair enough.

• 1:08:39Lex Fridman?

  And I think, I think that's what deep learning did in the community is, no, no, I still, if I had to bet all my money, it's 100% deep learning will not take us all the way. But there's still, there, it's still, I was so personally sort of surprised by the Atari games, by Go, by, by the power of self-play, of just game playing against each other, that I was, like many other times, just humbled of how little I know about what's possible in.

• 1:09:07Speaker 01

  Yeah, I think, fair enough. Self-play is amazingly powerful. And, you know, that's, that goes way back to Arthur Samuel, right, with his checker plane program. And that, which was brilliant and surprising that it did so well.

• 1:09:23Lex Fridman?

  So just for fun, let me ask you on the topic of autonomous vehicles. It's the area that, that I work, at least these days, most closely on, and it's also area that I think is a good example, that you use as sort of an example of things we as humans don't always realize how hard it is to do. It's like the, the constant trend in AI, but the different problems that we think are easy when we first try them, and then realize how hard it is. Okay. So.

• 1:09:56Lex Fridman?

  Why, you've talked about autonomous driving being a difficult problem, more difficult than we realize humans give it credit for. Why is it so difficult? What are the most difficult parts in your view?

• 1:10:09Speaker 01

  I think it's difficult because of the world is so open-ended as to what kinds of things can happen. So you have sort of what normally happens, which is just you drive along and nothing, nothing surprising happens. And autonomous vehicles can do, the ones we have now evidently can do really well on most normal situations as long, as long as, you know, the weather is reasonably good and everything. But if some, we have this notion of edge cases or, or, you know, things in the tail of the distribution. We call it the long tail problem,

• 1:10:52Speaker 01

  which says that there are so many possible things that can happen that was not in the training data of the machine that it won't be able to handle it because it doesn't have common sense.

• 1:11:08Lex Fridman?

  Right. It's the old, the paddle moved.

• 1:11:11Speaker 01

  Yeah, it's the paddle moved problem. Right. And so my understanding, and you probably are more of an expert than I am on this, is that current self-driving car vision systems have problems with obstacles, meaning that they don't know which obstacles, which quote unquote obstacles they should stop for and which ones they shouldn't stop for. And so a lot of times I read that they tend to slam on the brakes quite a bit. And the most common accidents with self-driving cars are people rear-ending them because they were surprised, they weren't expecting the machine, the car to stop.

• 1:11:53Lex Fridman?

  Yeah, so there's, there's a lot of interesting questions there, whether.

• 1:11:58Lex Fridman?

  Because you mentioned kind of two things. So one is the problem of perception, of understanding, of interpreting the objects that are detected.

• 1:12:07Speaker 01

  Right.

• 1:12:08Lex Fridman?

  Correctly. And the other one is more like the policy, the action that you take, how you respond to it. So a lot of the cars braking is a kind of notion of, to clarify, there's a lot of different kinds of things that are people calling autonomous vehicles. But the L4 vehicles with a safety driver are the ones like Waymo and Cruise and all those companies. They tend to be very conservative and cautious. So they tend to be very, very afraid of hurting anything or anyone and getting in any kind of accidents. So their policy is very kind of, that results in being exceptionally responsive to anything that could possibly be an obstacle.

• 1:12:50Speaker 01

  Right, which the human drivers around it, it behaves unpredictably.

• 1:12:58Lex Fridman?

  Yeah, that's not a very human thing to do, caution. That's not the thing we're good at, especially in driving. We're in a hurry, often angry, et cetera, especially in Boston. So and then there's sort of another, and a lot of times, that's, machine learning is not a huge part of that. It's becoming more and more unclear to me how much, you know, sort of speaking to public information. Because a lot of companies say they're doing deep learning and machine learning just to attract good candidates. The reality is, in many cases, it's still not a huge part of the perception. There's LiDAR and there's other sensors that are much more reliable for optical detection.

• 1:13:41Lex Fridman?

  And then there's Tesla approach, which is vision only. And there's, I think, a few companies doing that, but Tesla most famously pushing that forward.

• 1:13:50Speaker 01

  And that's because the LiDAR is too expensive, right?

• 1:13:53Lex Fridman?

  Well, I mean, yes, but I would say, if you were to for free give to every Tesla vehicle, I mean, Elon Musk fundamentally believes that LiDAR is a crutch, right? Fantasy said that.

• 1:14:09Lex Fridman?

  If you want to solve the problem of machine learning, LiDAR is not, should not be the primary sensor is the belief.

• 1:14:17Speaker 01

  Okay.

• 1:14:18Lex Fridman?

  The camera contains a lot more information. So if you want to learn, you want that information. But if you want not to hit obstacles, you want LiDAR. Right. It's sort of, it's this weird trade off because, yeah, so what Tesla vehicles have a lot of, which is really the thing, the fallback, the primary fallback sensor is radar, which is a very crude version of LiDAR. It's a good detector of obstacles, except when those things are standing. Right. The stopped vehicle.

• 1:14:57Speaker 01

  Right. That's why it had problems with crashing into stop fire trucks.

• 1:15:00Lex Fridman?

  Stop fire trucks. Right. So the hope there is that the vision sensor would somehow catch that. And for, there's a lot of problems with perception.

• 1:15:11Lex Fridman?

  They are doing actually some incredible stuff in the,

• 1:15:18Lex Fridman?

  almost like an active learning space where it's constantly taking edge cases and pulling back in. There's this data pipeline. Another aspect that is really important that people are studying now is called multitask learning, which is sort of breaking apart this problem, whatever the problem is, in this case, driving, into dozens or hundreds of little problems that you can turn into learning problems. So this giant pipeline, you know, it's kind of interesting. I've, I've been skeptical from the very beginning. We've become less and less skeptical over time, how much of driving can be learned? I still think it's much farther than, than the CEO of that particular company thinks it will be.

• 1:16:02Lex Fridman?

  But it is constantly surprising that through good engineering and data collection and active selection of data, how you can attack that long tail. And it's an interesting, open question that you're absolutely right. There's a much longer tail and all these edge cases that we don't think about. But it's this, it's a fascinating question that applies to natural language and all spaces. How big, how, how big is that long tail?

• 1:16:29Speaker 01

  Right.

• 1:16:30Lex Fridman?

  And I mean, not to linger on the point, but what's your sense in driving?

• 1:16:37Lex Fridman?

  In these practical problems of the human experience, can it be learned? So the current, what are your thoughts of sort of Elon Musk thought? Let's forget the thing that he says it'd be solved in a year. But can it be solved in in a reasonable timeline? Or do fundamentally other methods need to be invented?

• 1:16:59Speaker 01

  So I don't, I think that ultimately driving, so it's a trade off in a way. You know, being able to drive and deal with any situation that comes up does require kind of full human intelligence. And even in humans aren't intelligent enough to do it. Because humans, I mean, most human accidents are because the human wasn't paying attention or the humans drunk or whatever.

• 1:17:29Lex Fridman?

  And not because they weren't intelligent enough.

• 1:17:31Speaker 01

  And not because they weren't intelligent enough. Right. Whereas the accidents with autonomous vehicles is because they weren't intelligent enough.

• 1:17:43Lex Fridman?

  They're always paying attention.

• 1:17:43Speaker 01

  Yeah, they're always paying attention. So it's a trade off, you know? And I think that it's a very fair thing to say that autonomous vehicles will be ultimately safer than humans because humans are very unsafe. It's kind of a low bar.

• 1:18:00Lex Fridman?

  But just like you said, the, I think humans got a better rap, right? Because we're really good at the common sense thing.

• 1:18:08Speaker 01

  Yeah, we're great at the common sense thing. We're bad at the paying attention thing.

• 1:18:11Lex Fridman?

  Paying attention thing, right?

• 1:18:12Speaker 01

  Especially when we're, you know, driving is kind of boring and we have these phones to play with and everything. But I think what's going to happen is that for many reasons, not just AI reasons, but also like legal and other reasons, that the definition of self-driving is going to change or autonomous is going to change. It's not going to be just.

• 1:18:41Speaker 01

  I'm going to go to sleep in the back and you just drive me anywhere. It's going to be more. Certain areas are going to be instrumented to have the sensors and the mapping and all of the stuff you need for that that the autonomous cars won't have to have full common sense. And they'll do just fine in those areas as long as pedestrians don't mess with them too much. That's another question.

• 1:19:08Lex Fridman?

  That's right.

• 1:19:10Speaker 01

  But I don't think we will have fully autonomous self-driving in the way that like most, the average person thinks of it for a very long time.

• 1:19:22Lex Fridman?

  And just to reiterate, this is the interesting open question that I think I agree with you on is to solve fully autonomous driving, you have to be able to engineer in common sense.

• 1:19:34Speaker 01

  Yes.

• 1:19:37Lex Fridman?

  I think it's an important thing to hear and think about. I hope that's wrong, but I currently agree with you that unfortunately you do have to have, to be more specific, sort of these deep understandings of physics and of the way this world works and also human dynamics. Like you mentioned pedestrians and cyclists, actually that's whatever that non-verbal communication is some people call it. There's that dynamic that is also part of this common sense.

• 1:20:08Speaker 01

  Right. And we're pretty, we humans are pretty good at predicting what other humans are going to do.

• 1:20:15Lex Fridman?

  And how are our actions impact the behaviors of, so this is weird game theoretic dance that we're good at somehow. And the funny thing is, because I've watched countless hours of pedestrian video and talked to people, we humans are also really bad at articulating the knowledge we have.

• 1:20:34Speaker 01

  Right.

• 1:20:35Lex Fridman?

  Which has been the huge challenge.

• 1:20:37Speaker 01

  Yes.

• 1:20:38Lex Fridman?

  So you've mentioned embodied intelligence. What do you think it takes to build a system of human level intelligence? Does it need to have a body?

• 1:20:47Speaker 01

  I'm not sure, but I'm coming around to that more and more.

• 1:20:53Lex Fridman?

  And what does it mean to be, I don't mean to keep bringing up Yalun Kuhn.

• 1:20:58Speaker 01

  He looms very large.

• 1:21:01Lex Fridman?

  Well, he certainly has a large personality, yes. He thinks that the system needs to be grounded, meaning he needs to sort of be able to interact with reality, but doesn't think it necessarily needs to have a body. So when you think of-

• 1:21:14Speaker 01

  What's the difference?

• 1:21:16Lex Fridman?

  I guess I want to ask, when you mean body, do you mean you have to be able to play with the world? Or do you also mean like there's a body that you have to preserve?

• 1:21:28Speaker 01

  That's a good question. I haven't really thought about that, but I think both, I would guess, because I think you, I think intelligence, it's so hard to separate it from our self, our desire for self-preservation, our emotions, our, all that non-rational stuff that kind of gets in the way of logical thinking. Because we, the way, you know, if we're talking about human intelligence or human level intelligence, whatever that means, a huge part of it is

• 1:22:10Speaker 01

  social. That, you know, we were evolved to be social and to deal with other people. And that's just so ingrained in us that it's hard to separate intelligence from that. I think, you know, AI, for the last 70 years, or however long it's been around, it has largely been separated. There's this idea that there's like- It's kind of very Cartesian. There's this, you know, thinking thing that we're trying to create, but we don't care about all this other stuff. And I think the other stuff is very fundamental.

• 1:22:52Lex Fridman?

  So there's the idea that things like emotion get in the way of intelligence.

• 1:22:57Speaker 01

  As opposed to being an integral part of it.

• 1:22:59Lex Fridman?

  Integral part of it. So, I mean, I'm Russian, so romanticize the notions of emotion and suffering and all that kind of fear of mortality, those kinds of things. So in AI especially.

• 1:23:13Speaker 01

  By the way, did you see that? There was this recent thing going around the internet of this, some, I think he's a Russian or some Slavic, had written this thing of sort of anti the idea of superintelligence. I forgot, maybe he's Polish. Anyway, so he had all these arguments and one was the argument from Slavic pessimism.

• 1:23:35Speaker 01

  My favorite.

• 1:23:37Lex Fridman?

  Do you remember what the argument is?

• 1:23:39Speaker 01

  It's like, nothing ever works. Everything sucks.

• 1:23:45Lex Fridman?

  So what do you think is the role? Like, that's such a fascinating idea that the, what we perceive as sort of the limits of human, of the human mind, which is emotion and fear and all those kinds of things are integral to intelligence. Could you elaborate on that? Like, what, why is that important, do you think?

• 1:24:12Lex Fridman?

  For human level intelligence?

• 1:24:15Speaker 01

  At least for the way that humans work. It's a big part of how, it affects how we perceive the world. It affects how we make decisions about the world. It affects how we interact with other people. It affects our understanding of other people, you know?

• 1:24:32Speaker 01

  For me to understand your.

• 1:24:36Speaker 01

  What you're going, what you're likely to do, I need to have kind of a theory of mind. And that's very much a theory of emotion and motivations and goals. And, uh, and to understand that I, you know, we have the, the, this whole system of, you know, mirror neurons and we, you know, I sort of understand your motivations through sort of simulating it myself. So, you know, it's not something that I can prove that's necessary, but it seems very likely.

• 1:25:15Lex Fridman?

  So, uh, okay. You, you've written the op ed in New York Times titled, We Shouldn't Be Scared by Super Intelligent AI. And it criticized a little bit Stuart Russell and Nick Bostrom. Can you try to summarize that article's key ideas?

• 1:25:34Speaker 01

  So it was spurred by a earlier New York Times op ed by Stuart Russell, which was summarizing his book called Human Compatible. And the article was saying, you know, if we, if we have super intelligent AI, we need to have its values aligned with our values and it has to learn about what we really want. And he gave this example, what if we have a super intelligent AI and we give it the problem of solving climate change and it decides that the best way to lower the carbon in the atmosphere is to kill all the humans.

• 1:26:16Lex Fridman?

  No.

• 1:26:17Speaker 01

  Okay. So to me, that just made no sense at all because a super intelligent AI, first of all, thinking, what, trying to figure out what, what a super intelligence means. Uh, and it doesn't, it seems that.

• 1:26:35Speaker 01

  Something that's super intelligent can't just be intelligent along this one dimension of, okay, I'm going to figure out all the steps, the best optimal path to solving climate change, and not be intelligent enough to figure out that humans don't want to be killed, that you could get to one without having the other. And, you know, Bostrom, in his book, talks about the orthogonality hypothesis, where he says he thinks that a system's, I can't remember exactly what it is, but like a system's goals and its values don't have to be aligned. There's some orthogonality there, which didn't make any sense to me.

• 1:27:19Lex Fridman?

  So you're saying in any system that's sufficiently, not even super intelligent, but as it pro's greater and greater intelligence, there's a holistic nature that will sort of attention that will naturally emerge that prevents it from sort of any one dimension running away?

• 1:27:35Speaker 01

  Yeah, yeah, exactly. So, you know, Bostrom had this example of the, the super intelligent AI that, that makes, that turns the world into paper clips, because its job is to make paper clips or something. And that just, as a thought experiment, didn't make any sense to me.

• 1:27:55Lex Fridman?

  Well, as a thought experiment, or as a thing that could possibly be realized?

• 1:28:00Speaker 01

  Either. So, so I think that, you know, what my op ed was trying to do was say that, that intelligence is more complex than these people are presenting it, that it's not like, it's not so separable, the rationality, the, the values, the emotions, the, all of that, that it's, the, the view that you could separate all these dimensions and build a machine that has one of these dimensions, and it's super intelligent in one dimension, but it doesn't have any of the other dimensions. That's what I was trying to criticize, that, that, that I don't believe that.

• 1:28:42Lex Fridman?

  So can I read a few sentences from Yoshua Benja, who is?

• 1:28:50Lex Fridman?

  Always super eloquent. So he writes, "I have the same impression as Melanie that our cognitive biases are linked with our ability to learn to solve many problems. They may also be a limiting factor for AI. However, this is a 'may,' in quotes. Things may also turn out differently, and there's a lot of uncertainty about the capabilities of future machines. But more importantly for me, the value alignment problem is a problem well before we reach some hypothetical superintelligence. It is already posing a problem in the form of super-powerful companies, whose objective

• 1:29:32Lex Fridman?

  function may not be sufficiently aligned with humanity's general well-being, creating all kinds of harmful side effects." So he goes on to argue that, you know, the orthogonality and those kinds of things, the concerns of just aligning values with the capabilities of the system is something that might come long before we reach anything like a superintelligence. So your criticism is kind of really nice to say, this idea of superintelligent systems seem to be dismissing fundamental parts of what intelligence would take. And then Yoshio kind of says, yes, but if we look at systems that are much less intelligent,

• 1:30:15Lex Fridman?

  there might be these same kinds of problems that emerge.

• 1:30:19Speaker 01

  Sure, but I guess the example that he gives there of these corporations, that's people, right? Those are people's values. I mean, we're talking about people. The corporations aren't... their values are the values of the people who run those corporations.

• 1:30:39Lex Fridman?

  But the idea is the algorithm, that's right. So the fundamental person, the fundamental element of what does the bad thing is a human being.

• 1:30:49Speaker 01

  Yeah.

• 1:30:50Lex Fridman?

  But the algorithm kind of controls the behavior of this mass of human beings.

• 1:30:56Speaker 01

  Which algorithm?

• 1:30:57Lex Fridman?

  For a company that's the, for example, if it's an advertisement-driven company that recommends certain things and encourages engagement, so it gets money by encouraging engagement, and therefore the company more and more...

• 1:31:15Lex Fridman?

  It's like the cycle that builds an algorithm that enforces more engagement and made perhaps more division in the culture and so on, so on.

• 1:31:24Speaker 01

  I guess the question here is sort of who has the agency? So you might say, for instance, we don't want our algorithms to be racist.

• 1:31:36Speaker 01

  And facial recognition, you know, some people have criticized some facial recognition systems as being racist because they're not as good on darker skin than lighter skin.

• 1:31:47Lex Fridman?

  That's right.

• 1:31:48Speaker 01

  Okay, but the agency there, the actual facial recognition algorithm isn't what has the agency. It's not the racist thing, right? It's the, I don't know, the combination of the training data, the cameras being used, whatever. But my understanding of, and I'll say, I agree with Benjio there that he, you know, I think there are these value issues with our use of algorithms. But my understanding of what Russell's argument was is more that the algorithm, the machine itself has the agency now.

• 1:32:32Speaker 01

  It's the thing that's making the decisions and it's the thing that has what we would call values.

• 1:32:39Lex Fridman?

  Yes.

• 1:32:39Speaker 01

  So whether that's just a matter of degree, you know, it's hard to say, right? Because, but I would say that's sort of qualitatively different than a face recognition neural network.

• 1:32:52Lex Fridman?

  And to broadly linger on that point, if you look at Elon Musk or Stuart Russell or Bostrum, people who are worried about existential risks of AI, however far into the future, their argument goes is it eventually happens. We don't know how far, but it eventually happens. Do you share any of those concerns? And what kind of concerns in general do you have about AI that approach anything like existential threat to humanity?

• 1:33:24Speaker 01

  So I would say, yes, it's possible. But I think there's a lot more closer in existential threats to humanity.

• 1:33:33Lex Fridman?

  Because you said like 100 years for, so your time's

• 1:33:36Speaker 01

  More than 100 years.

• 1:33:37Lex Fridman?

  More than 100 years.

• 1:33:38Speaker 01

  Maybe even more than 500 years. I don't, I don't know. I mean, it's

• 1:33:42Lex Fridman?

  So the existential threats are so far out that the future is, I mean, there'll be a million different technologies that we can't even predict now that will fundamentally change the nature of our behavior, reality, society, and so on, before then.

• 1:33:57Speaker 01

  Yeah, I think so. I think so. And, you know, we have so many other pressing existential threats going on right now.

• 1:34:04Lex Fridman?

  Nuclear weapons even.

• 1:34:05Speaker 01

  Nuclear weapons, climate problems, you know, poverty, possible pandemics, that you can go on and on. And I think though, you know, worrying about existential threat from AI is, is not the best priority for what we should be worried about. That's kind of my view because we're so far away. But, you know, I'm not, I'm not necessarily criticizing Russell or Bostrom or whoever for worrying about that. And I'm, I think it's, some, some people should be worried about it. It's, it's certainly fine.

• 1:34:48Speaker 01

  But I, I was more sort of getting at their, their view of what intelligence is. So I was more focusing on like their view of the superintelligence than just the fact of them worrying. And the, the title of the article was, was written by the, the New York Times editors. I wouldn't have called it that.

• 1:35:13Lex Fridman?

  We shouldn't be scared by superintelligence.

• 1:35:16Speaker 01

  No.

• 1:35:17Lex Fridman?

  If you wrote it, it'd be like, we should redefine what you mean by superintelligence.

• 1:35:21Speaker 01

  I actually said, said, you know, something like, superintelligence is not, is, is not a sort of coherent idea.

• 1:35:33Speaker 01

  But that doesn't, that's not like something New York Times would put in.

• 1:35:36Lex Fridman?

  And the, the follow-up argument that Yoshio makes also, not argument, but a statement. And I've heard him say it before, and I think I, I agree. He's kind of has a very friendly way of phrasing it as it's good for a lot of people to believe different things.

• 1:35:53Speaker 01

  He's such a nice guy.

• 1:35:54Lex Fridman?

  Yeah, well, but he's, it's also practically speaking, like, we shouldn't be like, while your article stands, like Stuart Russell does amazing work, Basra does a lot amazing work, you do amazing work. And even when you disagree about the definition of superintelligence or the usefulness of even the term, it's still useful to have people that like use that term, right? And then argue.

• 1:36:20Speaker 01

  Sure. I, I absolutely agree with Benjo there. And I think it's great that, you know, and it's great that New York Times will publish all this stuff.

• 1:36:28Lex Fridman?

  That's right. It's an exciting time to be here. What, what do you think is a good test of intelligence? Like, is, is natural language ultimately a test that you find the most compelling? Like the, the original or the, what, you know, the higher levels of the Turing test? Kind of, yeah.

• 1:36:46Speaker 01

  Yeah, I, I still think the original idea of the Turing test is a good test for intelligence. I mean, I can't think of anything better. You know, the Turing test, the way that it's been carried out so far has been very impoverished, if you will. But I think a real Turing test that really goes into depth, like the one that I mentioned, I talk about in the book, I talk about Ray Kurzweil and Mitchell Kuper have this bet, right? That, that in 2029, I think is the date, they're, a machine will pass the Turing test. And Turing says, and they have a very specific, like, how many hours, expert judges and all

• 1:37:31Speaker 01

  of that. And, you know, Kurzweil says yes, Kuper says no. We can, we only have like nine more years to go to see. But I, I, you know, if something, a machine could pass that, I would be willing to call it intelligent.

• 1:37:48Lex Fridman?

  Of course, nobody will. They will say that's just a language model, right? If it does. So you would be comfortable. So language, a long conversation that's... Well, yeah. I mean, you're right, because I think probably to carry out that long conversation, you would literally need to have deep commonsense understanding of the world.

• 1:38:09Speaker 01

  I think so. I think so.

• 1:38:11Lex Fridman?

  And the conversation is enough to reveal that.

• 1:38:14Speaker 01

  I think so.

• 1:38:15Lex Fridman?

  Perhaps it is. So another super fun topic of complexity that you have worked on, written about. Let me ask the basic question. What is complexity?

• 1:38:29Speaker 01

  So complexity is another one of those terms, like intelligence. It's perhaps overused. But my book about complexity was about this wide area of complex systems, studying different systems in nature, in technology, in society, in which you have emergence, kind of like I was talking about with intelligence. You know, we have the brain, which has billions of neurons. And each neuron individually could be said to be not very complex compared to the system as a whole.

• 1:39:11Speaker 01

  But the system, the interactions of those neurons and the dynamics creates these phenomena that we call intelligence or consciousness, you know, that are, we consider to be very complex. So the field of complexity is trying to find general principles that underlie all these systems that have these kinds of emergent properties.

• 1:39:36Lex Fridman?

  And the emergence occurs from, like, underlying the complex system is usually simple, fundamental interactions.

• 1:39:45Speaker 01

  Yes.

• 1:39:46Lex Fridman?

  And the emergence happens when there's just a lot of these things interacting.

• 1:39:52Speaker 01

  Yes.

• 1:39:53Lex Fridman?

  Sort of what... And then most of science to date, can you talk about what is reductionism?

• 1:40:03Speaker 01

  Well, reductionism is when you try and take a system and divide it up into its elements, whether those be cells, or atoms, or subatomic particles, whatever your field is, and then try and understand those elements, and then try and build up an understanding of the whole system by looking at sort of the sum of all the elements.

• 1:40:31Lex Fridman?

  Yeah, so what's your sense, whether we're talking about intelligence, or these kinds of interesting, complex systems, is it possible to understand them in a reductionist way? Which is probably the approach of most of science today, right?

• 1:40:48Speaker 01

  I don't think it's always possible to understand the things we want to understand the most. So I don't think it's possible to look at single neurons and understand what we call intelligence, you know, to look at sort of summing up, and the sort of the summing up is the issue here that we're, you know, the one example is that the human genome, right? So there was a lot of work on excitement about sequencing the human genome, because the idea would be that

• 1:41:24Speaker 01

  we'd be able to find genes that underlies diseases. But it turns out that, and it was a very reductionist idea, you know, we figure out what all the the parts are, and then we would be able to figure out which parts cause which things. But it turns out that the parts don't cause the things that we're interested in. It's like the interactions, it's the networks of these parts. And so that kind of reductionist approach didn't yield the the explanation that we wanted.

• 1:41:55Lex Fridman?

  What is the most beautiful, complex system that you've encountered?

• 1:42:01Speaker 01

  Most beautiful?

• 1:42:03Lex Fridman?

  That you've been captivated by? Is it sort of, I mean, for me, is the simplest to be cellular automata?

• 1:42:12Speaker 01

  Oh, yeah. So I was very captivated by cellular automata and worked on cellular automata for several years.

• 1:42:19Lex Fridman?

  Do you find it amazing or is it surprising that such simple systems, such simple rules in cellular automata can create sort of seemingly unlimited complexity?

• 1:42:32Speaker 01

  Yeah, that was very surprising to me.

• 1:42:34Lex Fridman?

  How do you make sense of it? How does that make you feel? Is it just ultimately humbling? Or is there a hope to somehow leverage this into a deeper understanding and even able to engineer things like intelligence?

• 1:42:47Speaker 01

  It's definitely humbling. How humbling in that, also kind of awe inspiring that it's that awe inspiring, like, part of mathematics that these incredibly simple rules can produce this very beautiful, complex, hard to understand behavior. And that's, it's mysterious, you know, and surprising still. But exciting because it does give you kind of the hope that you might be able to engineer complexity just from.

• 1:43:25Lex Fridman?

  From simple, from the beginnings. Can you briefly say what is the Santa Fe Institute? Its history, its culture, its ideas, its future? So I've never, as I mentioned to you, I've never been, but it's always been this, in my mind, this mystical place where brilliant people study the edge of chaos.

• 1:43:42Speaker 01

  Yeah, exactly.

• 1:43:45Speaker 01

  So the Santa Fe Institute was started in 1984. And it was created by a group of scientists, a lot of them from Los Alamos National Lab, which is about a 40 minute drive from the Santa Fe Institute.

• 1:44:04Speaker 01

  They were mostly physicists and chemists. But they were frustrated in their field because they felt so that their field wasn't approaching kind of big interdisciplinary questions, like the kinds we've been talking about. And they wanted to have a place where people from different disciplines could work on these big questions without sort of being siloed into physics, chemistry, biology, whatever. So they started this institute. And this was people like George Cowan, who was a chemist in the Manhattan Project,

• 1:44:44Speaker 01

  and Nicholas Metropolis, who mathematician, physicist, Marie-Gail Mann, physicist. So some really big names here. Ken Arrow, an economist, Nobel Prize winning economist. And they started having these workshops. And this whole enterprise kind of grew into this research institute that's itself has been kind of on the edge of chaos its whole life because it doesn't have any, it doesn't have a significant endowment.

• 1:45:20Lex Fridman?

  Right.

• 1:45:21Speaker 01

  And it's just been kind of living on whatever funding it can raise through donations and grants and however it can, you know, business, business associates and so on. But it's a great place. It's a really fun place to go think about ideas from, that you wouldn't normally encounter.

• 1:45:45Lex Fridman?

  I saw Sean Carroll, so physicists.

• 1:45:50Speaker 01

  Yeah, he's on the external faculty.

• 1:45:52Lex Fridman?

  And you mentioned that there's, so there's some external faculty and there's people that are.

• 1:45:55Speaker 01

  A very small group of resident faculty. Maybe, maybe about 10. Who are there for on five year terms that can sometimes get renewed. And then they have some post docs and then they have this much larger on the order of 100 external faculty or people come like me who come and visit for various periods of time.

• 1:46:17Lex Fridman?

  So what do you think is the future of the Saffin Institute? Like what, and if people are interested, like what, what's there in terms of the public interaction or students or so on, that that could be a possible interaction with Saffin Institute or its ideas?

• 1:46:32Speaker 01

  Yeah, so there's a, there's a few different things they do. They have a complex system, summer school for graduate students and post docs, and sometimes faculty attend too. And that's a four week, very intensive residential program where you go and you listen to lectures and you do projects and people, people really like that. I mean, it's a lot of fun. They also have some specialty summer schools. There's one on computational social science. There's one on climate and sustainability, I think it's called.

• 1:47:10Speaker 01

  There's a few. And then they have short courses where just a few days on different topics.

• 1:47:17Speaker 01

  They also have an online education platform that offers a lot of different courses and tutorials from SFI faculty, including an introduction to complexity course that I taught.

• 1:47:31Lex Fridman?

  Awesome. And there's a bunch of talks too on, online from there's guest speakers and so on. They, they host a lot of.

• 1:47:37Speaker 01

  Yeah, they, they, they have a sort of technical seminars and colloquia. They all, and they have a community lecture series, like public lectures, and they put everything on their YouTube channel. So you can see it all.

• 1:47:51Lex Fridman?

  Watch it. Douglas Haastader, author of Getl, Escherbach, was your PhD advisor. He mentioned a couple of times and collaborator. Do you have any favorite lessons or memories from your time working with him? That continues to this day, I guess, but just even looking back throughout, throughout your time working with him.

• 1:48:12Speaker 01

  So one of the things he taught me was that when you're looking at.

• 1:48:18Speaker 01

  A complex problem to to idealize it as much as possible to try and figure out what are really, what is the essence of this problem? And this is how like the copycat program came into being was by taking analogy making and saying, how can we make this as idealized as possible, but still retain really the important things we want to study? And that's really kept, you know, been a core theme of my research, I think. And I continue to try and do that. And it's really very much kind of physics inspired. Hofstadter was a PhD in physics. That was his background.

• 1:49:01Lex Fridman?

  It's like first principles kind of thinking like you're reduced to the most fundamental aspect of the problem.

• 1:49:06Speaker 01

  Yeah.

• 1:49:07Lex Fridman?

  So that you can focus on solving that fundamental aspect.

• 1:49:09Speaker 01

  Yeah. And in AI, you know, that was, people used to work in these micro worlds, right? Like the blocks world was very early important area in AI. And then that got criticized because they said, oh, you know, you can't scale that to the real world. And so people started working on much like more real world like problems. But now there's been kind of a return even to the blocks world itself. You know, we've seen a lot of people who are trying to work on more of these very idealized problems for things like natural language and common sense. So that's an interesting evolution of those ideas.

• 1:49:49Lex Fridman?

  So the perhaps the blocks world represents the fundamental challenges of the problem of intelligence more than people realize.

• 1:49:56Speaker 01

  It might. Yeah.

• 1:49:58Lex Fridman?

  Is there sort of, when you look back at your body of work in your life, you've worked in so many different fields. Is there something that you're just really proud of in terms of ideas that you've gotten a chance to explore, create yourself?

• 1:50:12Speaker 01

  So I am really proud of my work on the copycat project. I think it's really different from what almost everyone has done in AI. I think there's a lot of ideas there to be explored. And I guess one of the happiest days of my life.

• 1:50:32Speaker 01

  You know, aside from like the births of my children, was the birth of copycat, when it actually started to be able to make really interesting analogies. And I remember that very clearly. That was a very exciting time.

• 1:50:48Lex Fridman?

  Well, you kind of gave life.

• 1:50:50Speaker 01

  Yes.

• 1:50:50Lex Fridman?

  An artificial system.

• 1:50:51Speaker 01

  That's right.

• 1:50:52Lex Fridman?

  What, in terms of what people can interact, I saw there's like a, I think it's called Meta Copy.

• 1:50:58Speaker 01

  Metacat.

• 1:50:58Lex Fridman?

  Metacat. And there's a Python 3 implementation. If people actually wanted to play around with it and actually get into it and study it and maybe integrate into whether it's with deep learning or any other kind of work they're doing, what

• 1:51:13Lex Fridman?

  what would you suggest they do to learn more about it and to take it forward in different kinds of directions?

• 1:51:18Speaker 01

  Yeah, so that there's Douglas Hofstadter's book called Fluid Concepts and Creative Analogies. Talks in great detail about copycat. I have a book called Analogy Making as Perception, which is a version of my PhD thesis on it. There's also code that's available that you can get it to run. I have some links on my webpage to where people can get the code for it. And I think that that would really be the best way to get into it.

• 1:51:46Lex Fridman?

  Dive in and play with it. Well, Melanie, it was an honor talking to you. I really enjoyed it. Thank you so much for your time today.

• 1:51:53Speaker 01

  Thanks, it's been really great.

• 1:51:56Lex Fridman?

  Thanks for listening to this conversation with Melanie Mitchell. And thank you to our presenting sponsor, Cash App. Download it, use code LEXPODCAST, you'll get $10 and $10 will go to First, a STEM education nonprofit that inspires hundreds of thousands of young minds to learn and to dream of engineering our future. If you enjoy this podcast, subscribe on YouTube, give it five stars on Apple Podcasts, support it on Patreon, or connect with me on Twitter. And now, let me leave you with some words of wisdom from Douglas Hofstadter and Melanie Mitchell. Without concepts, there can be no thought. And without analogies, there can be no concepts. And Melanie adds,

• 1:52:37Lex Fridman?

  "How to form and fluidly use concepts is the most important open problem in AI."

• 1:52:45Lex Fridman?

  Thank you for listening and hope to see you next time.

• 1:53:06Lex Fridman?

  Thank you.