#115 – Dileep George: Brain-Inspired AI

Dileep George is a researcher at the intersection of neuroscience and artificial intelligence, co-founder of Vicarious, formerly co-founder of Numenta. From the early work on Hierarchical temporal memory to Recursive Cortical Networks to today, Dileep’s always sought to engineer intelligence that is closely inspired by the human brain. Support this channel by supporting our sponsors. Click links, get discount: – Babbel: https://babbel.com and use code LEX – MasterClass: https://masterclass.com/lex – Raycon: https://buyraycon.com/lex If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter https://twitter.com/lexfridman, LinkedIn https://www.linkedin.com/in/lexfridman/, Facebook https://www.facebook.com/lexfridman, Medium https://medium.com/@lexfridman, or YouTube https://www.youtube.com/lexfridman where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts https://podcasts.apple.com/us/podcast/artificial-intelligence/id1434243584, follow on Spotify https://open.spotify.com/show/2MAi0BvDc6GTFvKFPXnkCL, or support it on Patreon https://www.patreon.com/lexfridman. Here’s the outline of the episode. On some podcast players you should be able to click the timestamp to jump to that time. OUTLINE: 0:00 – Introduction 4:50 – Building a model of the brain 17:11 – Visual cortex 27:50 – Probabilistic graphical models 31:35 – Encoding information in the brain 36:56 – Recursive Cortical Network 51:09 – Solving CAPTCHAs algorithmically 1:06:48 – Hype around brain-inspired AI 1:18:21 – How does the brain learn? 1:21:32 – Perception and cognition 1:25:43 – Open problems in brain-inspired AI 1:30:33 – GPT-3 1:40:41 – Memory 1:45:08 – Neuralink 1:51:32 – Consciousness 1:57:59 – Book recommendations 2:06:49 – Meaning of life

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Lex Fridman?

  The following is a conversation with Dileep George, a researcher at the intersection of neuroscience and artificial intelligence, co-founder of Vacarious with Scott Phoenix, and formerly co-founder of Numenta with Jeff Hawkins, who's been on this podcast, and Donna Dubinsky. From his early work on hierarchical temporal memory to recursive cortical networks to today, Dileep's always sought to engineer intelligence that is closely inspired by the human brain. As a side note, I think we understand very little about the fundamental principles underlying the function of the human brain. But the little we do know gives hints that may be more useful for engineering intelligence

• 0:43Lex Fridman?

  than any idea in mathematics, computer science, physics, and scientific fields outside of biology. And so the brain is a kind of existence proof that says it's possible. Keep at it. I should also say that brain-inspired AI is often overhyped and use this fodder just as quantum computing for a marketing speak. But I'm not afraid of exploring these sometimes overhyped areas since where there's smoke, there's sometimes fire. Quick summary of the ads. Three sponsors: Babbel, Raycon earbuds, and MasterClass. Please consider supporting this podcast by clicking the special links in the description to get the discount.

• 1:25Lex Fridman?

  It really is the best way to support this podcast. If you enjoy this thing, subscribe on YouTube, review it with five stars on Apple Podcasts, support on Patreon, or connect with me on Twitter at Lex Fridman. As usual, I'll do a few minutes of ads now and never any ads in the middle that can break the flow of the conversation. This show is sponsored by Babbel, an app and website that gets you speaking in a new language within weeks. Go to babbel.com and use code Lex to get three months free. They offer 14 languages, including Spanish, French, Italian, German, and yes, Russian. Daily lessons are 10 to 15 minutes, super easy, effective, designed by over 100 language experts.

• 2:10Lex Fridman?

  Let me read a few lines from the Russian poem "Noć ulica, fanari, apteka", by Alexander Bloch, that you'll start to understand if you sign up to Babbel. Ночь, улица, фонарь, аптека. Бессмысленный и тусклый свет. Живи еще хоть четверть века. Все будет так, исхода нет. Now, I say that you'll only start to understand this poem because Russian starts with a language and ends with vodka. Now, the latter part is definitely not endorsed or provided by Babbel and will probably lose me this sponsorship. But once you graduate from Babbel, you can enroll in my advanced course

• 2:53Lex Fridman?

  of late-night Russian conversation over vodka. I have not yet developed a nap for that. It's in progress. So get started by visiting babbel.com and use code LEX to get three months free. This show is sponsored by Raycon earbuds. Get them at buyraycon.com/lex. They become my main method of listening to podcasts, audiobooks and music when I run, do push-ups and pull-ups, or just living life. In fact, I often listen to Brown Noise with them when I'm thinking deeply about something. It helps me focus. They're super comfortable, pair easily, great sound, great bass, six hours of playtime. I've been putting in a lot of miles to get ready for a potential ultra marathon

• 3:38Lex Fridman?

  and listening to audiobooks on World War II. The sound is rich and really comes in clear. So again, get them at buyraycon.com/lex. This show is sponsored by MasterClass. Sign up at masterclass.com/lex to get a discount and to support this podcast. When I first heard about MasterClass, I thought it was too good to be true. I still think it's too good to be true. For 180 bucks a year, you get an all-access pass to watch courses from, to list some of my favorites. Chris Hadfield on space exploration, Neil deGrasse Tyson on scientific thinking and communication, Will Wright, creator of Simcity and Sims on game design. Every time I do this read,

• 4:21Lex Fridman?

  I really want to play a city builder game. Carlos Santana on guitar, Gera Kasparov on chess, Daniel Negrono on poker, and many more. Chris Hadfield explaining how rockets work and the experience of being launched into space alone is worth the money. By the way, you can watch it on basically any device. Once again, sign up at masterclass.com to get a discount and to support this podcast. And now here's my conversation with the leap George.

• 5:08Lex Fridman?

  Do you think we need to understand the brain in order to build it?

• 5:11Speaker 01

  Yes, if you want to build the brain, we definitely need to understand how it works. So Blue Brain or Henry Markram's project is trying to build a brain without understanding it like you know, just trying to put details of the brain from neuroscience experiments into a giant simulation by putting more and more neurons, more and more details. But that is not going to work because when it doesn't perform as what you expect it to do, then what do you do? You do, you just keep adding more details? How do you debug it? So it's a, so unless you understand, unless you have a theory about how the system is supposed to work,

• 5:54Speaker 01

  how the pieces are supposed to fit together, what they're going to contribute, you can't, you can't build it.

• 5:59Lex Fridman?

  At the functional level, understand. So can you actually linger on and describe the Blue Brain project? It's kind of fascinating principle and idea to try to simulate the brain. We're talking about the human brain, right?

• 6:15Speaker 01

  Right. Human brains and rat brains or cat brains have lots in common that the cortex, the neocortex structure is very similar. So initially they were trying to just simulate a cat brain and.

• 6:32Lex Fridman?

  To understand the nature of evil.

• 6:34Speaker 01

  Understand the nature of evil. Or as it happens in most of these simulations, you, you easily get one thing out, which is oscillations. You know, yeah, if you, if you simulate a large number of neurons, they oscillate. And you can adjust the parameters and say that oscillations match the rhythm that we see in the brain, et cetera. But.

• 6:57Lex Fridman?

  I see. So like, so the idea is. Uh. Is the simulation at the level of individual neurons?

• 7:04Speaker 01

  Yeah, so the Blue Brain Project, the original idea as proposed was uh you you put very detailed biophysical neurons uh biophysical models of neurons. And you interconnect them according to the statistics of connections that we have found from real neuroscience experiments and then uh turn it on and see what happens. Uh and and these neural models are, you know, incredibly complicated in themselves, right? Because these neurons are a model using uh this uh idea called Hodgkin-Huxley models, which are about how signals propagate in a cable.

• 7:45Speaker 01

  And there are a active dendrites, all those phenomena which those phenomena themselves, we don't understand that well. Uh and then uh we put in connectivity, which is part guesswork, part, you know, observed. And of course, if we do not have any theory about how it is supposed to work, uh we you you know, we just have to take whatever comes out of it as, okay, this is something interesting.

• 8:12Lex Fridman?

  But in your sense, like these models of the way signal travels along or like with the axons and all the basic models, that's they're too crude.

• 8:21Speaker 01

  Oh, well, actually, they are pretty detailed and pretty sophisticated. And they do replicate the neural dynamics. If you take a single neuron and you you try to uh turn on the different channels, the calcium channels and uh the different receptors, uh and see what the effect of uh turning on or off those channels are in the neuron's spike output. People have built pretty sophisticated models of that and and they are, I I would say, um, you know, in the regime of correct.

• 8:58Lex Fridman?

  Well, see, the correctness, that's interesting because you've mentioned it several levels. The correctness is measured by looking at some kind of aggregate statistics?

• 9:06Speaker 01

  It would be more the spiking dynamics of a signal.

• 9:10Lex Fridman?

  Spiking dynamics of signal, yeah, okay.

• 9:13Speaker 01

  And, yeah, these models, because they are, they are going to the level of mechanism, right? So they are basically looking at, okay, what, what is the effect of turning on an ion channel? And, and you can, you can model that using electric circuits. And then, so they are model, so it is not just a function fitting. It is, people are looking at the mechanism underlying it and putting that in terms of electric circuit theory, signal propagation theory and, and modeling that. And so those models are sophisticated, but getting a single neurons model 99% right

• 9:52Speaker 01

  does not still tell you how to, you know, it would be the analog of getting a transistor model right and now trying to build a microprocessor. And if you, if you just observe, you know, if you did not understand how a microprocessor works, but you say, oh, I have, I now can model one transistor well. And now I will just try to interconnect the transistors according to whatever I could, you know, guess from the, the experiments and try to simulate it. Then it is very unlikely that you will produce a functioning microprocessor. You want to, you know, when you want to produce a functioning microprocessor, you want to understand Boolean logic, how does, how do the, the gates work, all those things.

• 10:37Speaker 01

  And then, you know, understand how do those gates get implemented using transistors?

• 10:41Lex Fridman?

  Yeah, there's actually, I remember, this reminds me, there's a paper, maybe you're familiar with it, that I remember going through in a reading group that approaches a microprocessor from a perspective of neuroscientist. I think it, it basically, it uses all the tools that we have of neuroscience to try to understand, like as if we just aliens showed up to study computers.

• 11:05Speaker 01

  Yeah.

• 11:05Lex Fridman?

  And, and to see if, if those tools can be used to get any kind of sense of how the microprocessor works. I think the final, the takeaway from at least this initial exploration is that we're screwed. There's no way that the tools of neuroscience would be able to get us to anything, like not even Boolean logic. I mean, it's. Just a any aspect of the architecture of the function of the processes involved, the clocks, the the timing, all that, you can't figure that out from the tools of neuroscience.

• 11:42Speaker 01

  Yes, I'm very familiar with this this particular paper. I think it was called Can a neuroscientist understand a microprocessor?

• 11:51Lex Fridman?

  Yeah.

• 11:51Speaker 01

  Something like that. Following the methodology in that paper, even an electrical engineer would not understand microprocessors. So I could, so I so I I don't think it is that bad in the sense of saying neuroscientists do find valuable things by observing the brain. They they do find good insights, but those insights cannot be put together just as a simulation. You have to you have to investigate what are the computational under findings findings of those findings? How do do all of them fit together from an information processing perspective?

• 12:33Speaker 01

  You have to you have to somebody has to painstakingly put those things together and build hypothesis. So I don't want to this all of neuroscience is saying, oh, they are not finding anything. No, that, you know, that that paper almost went to that level of neuroscientists will never understand. No, that that's not true. I think they do find lots of useful things, but it has to be put together in a in a computational framework.

• 12:57Lex Fridman?

  Yeah, I mean, but, you know, just the AI systems will be listening to this podcast a hundred years from now, and they will probably it there's some non-zero probability they'll find your words laughable. There's like, I remember humans humans thought they understood something about the brain. They're totally clueless. There's a sense about neuroscience that we may be in the very, very early days of understanding the brain. But, I mean, that's one perspective. I mean, in your perspective, how far are we into understanding?

• 13:32Lex Fridman?

  Any aspect of the brain, so the the dynamics of the individual neuron communication, to the how in a collective sense, how they're able to store information, transfer information, how the intelligence then emerges, all that kind of stuff. Where are we on that timeline?

• 13:52Speaker 01

  Yeah. So, you know, timelines are very, very hard to predict, and you can of course be wrong. And it can be wrong on either side. You know, we know that, you know, when we look back, the first flight was in nineteen oh three. In nineteen hundred, there was a New York Times article on flying machines that do not fly. And, and, you know, humans might not fly for another hundred years. That was what that article stated. And so, but no, they flew three years after that. So it is, you know, it's very hard to, so.

• 14:28Lex Fridman?

  Well, and on that point, one of the Wright brothers, I think, two years before, said that, like he said, like some number, like fifty years, he has become convinced that it's, it's, it's impossible.

• 14:47Speaker 01

  Even during their experimentation. Yeah.

• 14:48Lex Fridman?

  Yeah, I mean, that's a tribute to when, that's like the entrepreneurial battle of like depression of going through just like thinking this is impossible. But there, yeah, there's something, even the person that's in it is not able to see, estimate correctly.

• 15:04Speaker 01

  Exactly. But I can, I can tell from the point of, you know, objectively, what are the things that we know about the brain and how that can be used to build AI models, which can then go back and inform how the brain works. So my way of understanding the brain would be to basically say, look at the insights neuroscientists have found. Understand that from a computational angle, information processing angle. Build models using that. And then, building that model which functions, which is a functional model, which is doing the task that we want the model to do. It is not just trying to model a phenomena in the brain. It is trying to do what the brain is trying to do on the whole functional level.

• 15:49Speaker 01

  And building that model will help you fill in the missing pieces. That, you know, biology just gives you the hints. And building the model, you know, fills in the rest of the pieces of the puzzle. And then you can go and connect that back to biology and say, okay, now it makes sense that this part of the brain is doing this, or this layer in the cortical circuit is doing this. And then continue this iteratively. Because now, that will inform new experiments in neuroscience. And of course, you know, building the model and verifying that in the real world will also tell you more about, does the model actually work?

• 16:30Speaker 01

  And you can refine the model, find better ways of putting these neuroscience insights together. So, so I would say it is, you know, it, so neuroscientists alone, just from experimentation, will not be able to build a model of the, of the brain, or a functional model of the brain. So we, you know, there's lots of efforts, which are very impressive efforts in collecting more and more connectivity data from the brain, you know, how are the microcircuits of the brain connected with each other? Those are beautiful, by the way. Those are beautiful. And at the same time, those, those do not itself, by themselves, convey the story of how does it work? Yeah.

• 17:14Speaker 01

  And, and somebody has to understand, okay, why are they connected like that? And what, what are those things doing? And, and we do that by building models in AI, using hints from neuroscience, and, and repeat the cycle.

• 17:29Lex Fridman?

  So what aspect of the brain are useful in this whole endeavor? Which, by the way, I should say, you're both a neuroscientist and AI person. I guess the dream is to both understand the brain and to build AGI systems. So you're, it's like an engineer's perspective of trying to understand the brain. So what aspects of the brain, functionally speaking, like you said, do you find interesting?

• 17:56Speaker 01

  Yeah, quite a lot of things, alright. So one is, you know, if you look at the visual cortex and, and, you know, the visual cortex is, is a large part of the brain. I forgot the exact fraction, but it is, it's a, it's a huge part of our brain area is occupied by just, just vision. So vision, visual cortex is not just a feed forward cascade of neurons. There are a lot more feedback connections in the brain compared to the feed forward connections. And, and it is surprising to the level of detail neuroscientists have actually studied this. If you, if you go into neuroscience literature and poke around and ask, you know,

• 18:37Speaker 01

  have they studied, what will be the effect of poking a neuron in a level IT in level V1? And have they studied that? And you will say, yes, they have studied that.

• 18:52Lex Fridman?

  So every possible combination.

• 18:55Speaker 01

  I mean, it's, it's, it's not a random exploration at all. It's very hypothesis driven, right? They, they are very, experimental neuroscientists are very, very systematic in how they probe the brain. Because experiments are very costly to conduct. They take a lot of preparation, they, they need a lot of control. So they, they are very hypothesis driven in how they probe the brain. And often what I find is that when we have a question in, in AI about, have, has anybody probed, probed how lateral connections in the brain works? And when you go and read the literature, yes, people have probed it and people have probed it very systematically.

• 19:32Speaker 01

  And, and they have hypothesis about how those lateral connections are supposedly contributing to visual processing. But of course, they haven't built very, very functional detailed models of it.

• 19:46Lex Fridman?

  By the way, how do you know studies, sorry to interrupt, do they stimulate like a neuron in one particular area of the visual cortex and then see how the travel, the signal travels kind of thing?

• 19:57Speaker 01

  Fascinating, very, very fascinating experiments, right? So I can, I can give you one example I was impressed with. This is, so before going to that, let me, like, let me give you, you know, a overview of how the, the layers in the cortex are organized, right? Visual cortex is organized into roughly four hierarchical levels. Okay, so V1, V2, V4, IT. And in V1-

• 20:20Lex Fridman?

  What happened to V3?

• 20:21Speaker 01

  Uh, well, yeah, there's another pathway.

• 20:23Lex Fridman?

  Okay.

• 20:24Speaker 01

  Okay, so there is, this is, this, I'm, I'm talking about just object recognition pathway.

• 20:27Lex Fridman?

  All right, cool.

• 20:28Speaker 01

  Uh, and then, um, in V1 itself, um, so it's, there is a very detailed micro circuit in V1 itself, that is, that is organization within a level itself. The cortical sheet is organized into, uh, you know, multiple layers and there are columnar structure. And, and this, this layer wise and columnar structure is repeated in V1, V2, V4, uh, uh, IT, all of them, right? Uh, and, and the connections between these layers within a level, you know, in V1 itself, there are six layers roughly, and the connections between them, there is a particular structure to them.

• 21:05Speaker 01

  Uh, and, um, now, so one example of an experiment, uh, people did is, when I, when you present a stimulus, uh, which is, um, let's say, requires, um, separating the foreground from the background of an object. So it is, it's a textured triangle on a textured background. Uh, and, um, you can check, does the surface settle first or does the contour settle first?

• 21:36Lex Fridman?

  Settle?

• 21:37Speaker 01

  Settle in the sense that the, so when you fi- finally form the percept of the, of the, uh, triangle, you understand where the contours of the triangle are and you also know where the inside of the triangle is, right? That's when you form the final percept. Uh, now, you can- Ask what is the dynamics of forming that final person? Um, do the, uh, do the neurons, um, first find the edges and converge on where the edges are? And then they find the inner surfaces or does it go the other way?

• 22:13Lex Fridman?

  The other way around.

• 22:13Speaker 01

  Uh, so.

• 22:14Lex Fridman?

  So what's the answer?

• 22:15Speaker 01

  Uh, in this case, it, it turns out that it fi- first settles on the edges. It, it converges on the edge hypothesis first and then the, the surfaces are filled in from the edges to the inside.

• 22:28Lex Fridman?

  That's fascinating.

• 22:29Speaker 01

  Uh, and, and the detail to which you can study this, it's, it's amazing that you can actually not only find um, the temporal dynamics of when this happens. Uh, uh, and then you can also find which layer in the, you know, in V one, which layer is encoding, uh, the edges? Which layer is encoding the surfaces and, um, which layer is encoding the feedback? Which layer is encoding the feed forward and what, what's the combination of them that produces the final person? Um, and these kinds of experiments stand out when you try to explain illusions. Uh, one, one example of a favorite illusion of mine is the Kanizza triangle. I don't know that you are familiar with this one.

• 23:10Speaker 01

  So this is, um, uh, this is an example where it's a triangle, uh, but you know, the, the corners of the, only the corners of the triangle are shown in the stimuli, the stimulus.

• 23:20Lex Fridman?

  Uh huh.

• 23:21Speaker 01

  Uh, so they, they look like kind of Pac-Man.

• 23:23Lex Fridman?

  Oh, the, the black Pac-Man.

• 23:25Speaker 01

  Exactly.

• 23:25Lex Fridman?

  Uh, and then. You start to see.

• 23:27Speaker 01

  Your visual system hallucinates the edges.

• 23:30Lex Fridman?

  Yeah.

• 23:30Speaker 01

  Um, and you can, you, you know, you, when you look at it, you will see a faint edge, right? And you can go inside the brain and look, you know, do actually neurons signal the presence of this edge? And, and if they signal, how do they do it? Because they are not receiving anything from the input. The, in the, the input is blank. For those neurons, right? Uh, so how do they signal it? When does the signaling happen? You know, does it, you know, so, so if a real contour is present in the input, then the sig- the neurons immediately signal, okay, there is a, there is an edge here. When, when it is an illusory edge, um, it is clearly not in the input. It is coming from the context. So those neurons fire later.

• 24:15Speaker 01

  And, and you can say that, okay, these are, it's the feedback connections that is causing them to fire. Uh, and, and they happen later. And you can fi- I'll find the dynamics of them. Oh, so, so these studies are pretty impressive and, and very detailed.

• 24:30Lex Fridman?

  So, by the way, just, uh, just to take, step back, you said, uh, that there may be more feedback connections than feed forward connections.

• 24:38Speaker 01

  Yeah.

• 24:39Lex Fridman?

  Uh, first of all, if it's just for like a machine learning folks.

• 24:44Speaker 01

  Yeah.

• 24:44Lex Fridman?

  I mean, that, that's crazy that there's all these feedback connections. I mean, we often think about, I think, thanks to deep learning, you, you start to think about, um, the, the human brain as a kind of feed forward mechanism.

• 25:00Speaker 01

  Right.

• 25:01Lex Fridman?

  Uh, so what the heck are these feedback connections?

• 25:06Speaker 01

  Yeah.

• 25:06Lex Fridman?

  What's their, what's the dynamics? Well, what are we supposed to think about them?

• 25:11Speaker 01

  Yeah, so this is, this fits into a very beautiful picture about how the brain works, right? Um, so the, the beautiful picture of how the brain works is that our brain is building a model of the world. Uh, I know, so our visual system is building a model of how objects behave in the world. And, and we are constantly projecting that model back onto the world. So what we are seeing is not just a feed forward thing that just gets interpreted in, in a feed forward part. It, we are constantly projecting our expectations onto the world. And, and what the final percept is a combination of what we project onto the world. Combined with what the actual sensory input is.

• 25:53Lex Fridman?

  Uh huh. Almost like trying to calculate the difference and then trying to interpret the difference.

• 25:58Speaker 01

  Yeah, it's it's um I wouldn't put it as calculating the difference. It's more like what is the best explanation for the the input stimulus based on the model of the world I have.

• 26:09Lex Fridman?

  Got it. Got it. And that's where all the illusions come in and that's but that's that's incredibly efficient. So uh efficient process. So the feedback mechanism, it just helps you constantly uh yeah. So hallucinate how the world should be based on your world model and then just looking at uh if there's novelty uh like trying to explain it.

• 26:33Speaker 01

  Yeah.

• 26:33Lex Fridman?

  Hence, that's why movement, we detect movement really well. There's all these kinds of things. And this is like at all different levels of the cortex you're saying that this happens at the lowest level, the highest level.

• 26:46Speaker 01

  Yes. Yeah. In fact, feedback on actions are more prevalent in everywhere in the cortex and and um. So one way to think about it, and there's a lot of evidence for this, is inference. Uh so you know, so basically, if you have a model of the world and when when some evidence comes in, what you are doing is inference, right? You are trying to now explain this evidence using your model of the world.

• 27:12Lex Fridman?

  Yep.

• 27:12Speaker 01

  And this inference includes projecting your model onto the evidence and taking the evidence uh back into the model and and doing an iterative procedure. Um and this iterative procedure is what happens using the feed forward feedback propagation. Uh and feedback affects what you see in the world and you know, it also affects feed forward propagation. And examples are everywhere. We we see these kinds of things everywhere. The idea that there can be multiple competing hypothesis uh in our model, trying to explain the same evidence. And then you have to kind of make them compete and one hypothesis will explain away the other hypothesis

• 27:58Speaker 01

  through this competition process.

• 27:59Lex Fridman?

  Wait, what? So you you have competing models of the world that try to explain. What do you mean by explain away?

• 28:07Speaker 01

  So this is a classic example in a uh graphical models, probabilistic models. Um so if you.

• 28:14Lex Fridman?

  What are those?

• 28:15Speaker 01

  Uh, okay.

• 28:18Lex Fridman?

  I think it's useful to mention because we'll talk about them more.

• 28:21Speaker 01

  Yeah, yeah. So neural networks are one class of machine learning models. You know, you have distributed set of nodes which are called the neurons. You know, each one is doing a dot product and you can you can approximate any function using this multi-level network of neurons. So that's a class of models which are useful useful for function approximation. There is another class of models in machine learning called probabilistic graphical models. And you can think of them as each node in that model is variable which is which is talking about something. You know, it can be a variable representing is is an edge present in the input or not.

• 29:05Speaker 01

  And at the top of the network, a node can be representing is there an object present in the world or not? And and then so it can it is it is another way of encoding knowledge. And and then you, once you encode the knowledge, you can do inference in the right way, you know, how what is the best way to you know, explain some set of evidence using this model that you encoded, you know. So when you encode the model, you are encoding the relationship between these different variables. How is the edge connected to my the model of the object? How is the surface connected to the model of the object?

• 29:47Speaker 01

  And then, of course, this is a very distributed, complicated model. And inference is how do you explain a piece of evidence? When when a set of stimulus comes in, if somebody tells me there is a 50% probability that there is an edge here in this part of the model. How does that affect my belief on whether I should think that there should be is the square present in the image? So this is the process of inference. So one example of inference is having this experience of effect between multiple causes. So graphical models can be used to represent causality in the world.

• 30:26Speaker 01

  So let's say, you know, your alarm at home can be triggered by a burglar getting into your house, or it can be triggered by an earthquake. Both can be causes of the alarm going off. So now, you're in your office, you heard burglar alarm going off, you are heading home, thinking that there's a burglar. But while driving home, if you hear on the radio that there was an earthquake in the vicinity, now your strength of evidence for a burglar getting into their house is diminished.

• 31:07Speaker 01

  Because now that that piece of evidence is explained by the earthquake being present. So if you think about these two causes explaining at lower level variable which is alarm, now what we're seeing is that increasing the evidence for some cause, you know, there is evidence coming even from below for alarm being present. And initially it was flowing to a burglar being present. But now, since somebody, there is side evidence for this other cause, it explains away this evidence and evidence will now flow to the other cause. This is, you know, two competing causal things trying to explain the same evidence.

• 31:45Lex Fridman?

  And the brain has a similar kind of mechanism for doing so. That's kind of interesting. And that, how's that all encoded in the brain? Like, where's the storage of information? Are we talking, just maybe to get it a little bit more specific? Is it in the hardware of the actual connections? Is it in chemical communication? Is it electrical communication? Do we know?

• 32:11Speaker 01

  So this is, you know, a paper that we are bringing out soon.

• 32:15Lex Fridman?

  Which one is this?

• 32:16Speaker 01

  This is the cortical microcircuits paper that I sent you a draft of. Of course, this is. A lot of it is still hypothesis, one hypothesis that you can think of a cortical column as encoding a concept, a concept, you know, think of it as a, a, um, an example of a concept is, um, is an edge present or not, or is, is an object present or not. Okay. So it can, you can think of it as a binary variable, a binary random variable, the presence of an edge or not, or the presence of an object or not. So each cortical column can be thought of as representing that one concept, one variable. And then the connections

• 32:57Speaker 01

  between these cortical columns are basically encoding the relationship between these random variables. And then there are connections within the cortical column. There are, each cortical column is implemented using multiple layers of neurons with very, very, very rich um, structure there, you know, there are thousands of neurons in a cortical column.

• 33:17Lex Fridman?

  But, but that structure is similar across the different cortical columns.

• 33:21Speaker 01

  Correct.

• 33:21Lex Fridman?

  Yeah.

• 33:21Speaker 01

  Correct. And also these cortical columns connect to a substructure called thalamus in the, uh, you know, so all, all cortical columns pass through this substructure. So our hypothesis is that, yeah, the connections between the cortical columns implement this, uh, you know, that's where the knowledge is stored about, you know, how these different concepts, concepts connect to each other. And then the, the neurons inside this cortical column and in thalamus in combination, implement this, um, actual computations in there for inference, which includes explaining away and competing between the different hypotheses.

• 34:00Speaker 01

  Um, and it is all very, so what is amazing is that, um, neuroscientists have actually done experiments to the tune of showing these things. Uh, they might not be putting it in the overall inference framework, but they will show things like, if I poke this higher level neuron, uh, it will inhibit through this complicated loop through the thalamus, it will inhibit this other column. So they will do such experiments.

• 34:29Lex Fridman?

  But do they use terminology of concepts, for example? So you're, I mean,

• 34:36Lex Fridman?

  is it, is it something where it's easy to anthropomorphize? And think about concepts like you start moving into logic based kind of reasoning systems. So are we to think of concepts in that kind of way? Or is it, is it a lot messier, a lot more gray area? You know, even, even more gray, even more messy than the artificial neural network kinds? Types of abstractions?

• 35:05Speaker 01

  Easiest way to think of it is a variable, right? It's a binary variable, which is showing the presence or absence of something.

• 35:12Lex Fridman?

  But I guess what I'm asking is, is that something that we're supposed to think of something that's human interpretable of that something?

• 35:21Speaker 01

  It doesn't need to be. It doesn't need to be human interpretable. There is no need for it to be human interpretable. But it's, it's almost like

• 35:31Speaker 01

  you, you will be able to find some interpretation of it. Because it is connected to the other things that you know about.

• 35:39Lex Fridman?

  And the, the point is it's useful somehow.

• 35:42Speaker 01

  Yeah.

• 35:42Lex Fridman?

  It's useful as an entity in the graph that in connecting to the other entities that are, let's call them concepts.

• 35:51Speaker 01

  Right.

• 35:52Lex Fridman?

  Okay, so by the way, what's, are these the cortical microcircuits?

• 35:57Speaker 01

  Correct, these are the cortical microcircuits. You know, that's what neuroscientists use to talk about the circuits in, in within a level of the cortex. So you can think of, you know, let's think of in neural network, you know, artificial neural network terms, you know? People talk about the architecture of the, you know, so how many, how many layers they build? You know, what is the fan in fan out, etc. That is the macro architecture. So, and then within a layer of the neural network, you can, you know, the cortical neural network is much more structured within, you know, within a level. There's a lot more intricate structure there. But even, even within an artificial neural network, you can think of in feature detection plus pooling as one, one level.

• 36:42Speaker 01

  And so that is kind of a micro circuit. Uh it it's much more uh complex in the real brain. Uh and and so within a level, whatever is that circuitry within a column of the cortex and between the layers of the cortex, that's the micro circuitry.

• 36:58Lex Fridman?

  Yeah, I love that terminology. Uh machine learning people don't use the circuit terminology.

• 37:03Speaker 01

  Right.

• 37:03Lex Fridman?

  But they should. It's a it's a nice. So, okay. Uh, okay, so that's uh that that's the the cortical micro circuit. So what's interesting about uh what what can we say? What is the paper that you're working on uh propose about the ideas around these cortical micro circuits?

• 37:21Speaker 01

  So this is a fully functional model for the micro circuits of the visual cortex.

• 37:28Lex Fridman?

  So the the paper focuses and your idea and our discussion now is focusing on vision.

• 37:33Speaker 01

  Yeah.

• 37:33Lex Fridman?

  The uh visual cortex. Okay.

• 37:36Speaker 01

  Yeah.

• 37:36Lex Fridman?

  So this is a model. This is a full model. This is this is how vision works.

• 37:39Speaker 01

  Uh but this is this is a.

• 37:42Lex Fridman?

  A hypothesis, a hypothesis.

• 37:43Speaker 01

  Okay, so let me let me step back a bit. Um so we looked at neuroscience for insights on how to build a vision model. Right. And and and we synthesized all those insights into a computational model. This is called the recursive cortical network model that we we used for breaking capture and and we are using the same model for robotic picking and uh tracking of objects.

• 38:08Lex Fridman?

  And that again is a vision system.

• 38:09Speaker 01

  That's a vision system.

• 38:10Lex Fridman?

  Computer vision system.

• 38:11Speaker 01

  That's a computer vision system.

• 38:12Lex Fridman?

  Takes in images and outputs what?

• 38:16Speaker 01

  On one side, it outputs the class of the image uh and also segments the image. Uh and you can also ask it further queries. Where is the edge of the object? Where is the interior of the object? So so it's it's a model that you build to answer multiple questions. So you are not trying to build a model for just classification or just segmentation, etc. It's a it's a it's a joint model that can do multiple things. Um and um so so that's the model that we built using insights from neuroscience. And some of those insights are what is the role of feedback connections? What is the role of lateral connections? Uh so all those things went into the model. The the model actually uses feedback connections.

• 38:58Lex Fridman?

  All these ideas from your from your science.

• 39:00Speaker 01

  Yeah.

• 39:00Lex Fridman?

  Uh so what what what the heck is a recursive cortical network? Like what? What are the architecture approaches, interesting aspects here? Which is essentially a brain-inspired approach to a computer vision.

• 39:13Speaker 01

  Yeah, so there are multiple layers to this question. I can go from the very, very top and then zoom in, okay. So one important thing, constraint that went into the model is that you should not think vision, think of vision as something in isolation. We should not think perception as something, as a preprocessor for cognition. Perception and cognition are interconnected. And so you should not think of one problem in separation from the other problem. And so that means if you finally want to have a system that understands concepts about the world and can learn a very conceptual model of the world and can reason and connect to language, all of those things,

• 39:56Speaker 01

  you need to, you need to have, think all the way through and make sure that your perception system is compatible with your cognition system and language system and all of them. And one aspect of that is top-down controllability.

• 40:11Lex Fridman?

  What does that mean?

• 40:11Speaker 01

  So that means, you know, so, so think of, you know, you can close your eyes and think about the details of one object, right? I can, I can zoom in further and further. I can, you know, so, so think of the bottle in front of me, right? And, and now you can think about, okay, what the cap of that bottle looks. Uh, I know, we can think about what's the texture on that bottle of the, of the cap. You know, you can think about, you know, what will happen if something hits that. Uh, so you can, you can, you can manipulate your visual knowledge in a cognition-driven ways.

• 40:49Lex Fridman?

  Yes.

• 40:50Speaker 01

  Uh, and so this top-down controllability and being able to simulate scenarios in the world.

• 40:58Lex Fridman?

  So you're not just a passive, uh... Player in this perception game, you you can you can control it.

• 41:04Speaker 01

  Correct.

• 41:05Lex Fridman?

  You you you have imagination.

• 41:06Speaker 01

  Correct, correct. So so so basically, you know, basically having a generative network.

• 41:11Lex Fridman?

  Yeah.

• 41:12Speaker 01

  Uh, which is a model and and it is not just some arbitrary generated network. It has to be, it has to be built in a way that it is controllable top down. It is, it is not just trying to generate a whole picture at once. Uh, you know, it's not trying to generate photorealistic things of the world. You, you know, you don't have good photorealistic models of the world. Human brains do not have. If I, if I, for example, ask you the question, uh, what is the color of the letter E in the Google logo?

• 41:39Speaker 01

  You have no idea.

• 41:40Lex Fridman?

  No idea.

• 41:41Speaker 01

  Although you have seen it millions of times, or not millions of times, hundreds of times. So.

• 41:45Lex Fridman?

  Yeah.

• 41:46Speaker 01

  Uh, so it's not, our model is not photorealistic. But but it it is, but it has other properties that we can manipulate it. Uh, in the, uh, and you can think about filling in a different color in that logo. You can think about expanding the the letter E.

• 41:59Lex Fridman?

  Yeah.

• 41:59Speaker 01

  Uh, you you know, you you can see what in, so you can imagine the consequence of, you you know, actions that you have never performed. So so these are the kind of characteristics the generative model need to have. So this is one constraint that went into our model, like, you know, so this is, when you read the, just the perception side of the paper, it is not obvious that this was a constraint into the, it that went into the model, this top down controllability of the generative model.

• 42:23Lex Fridman?

  Uh, so what what is the top down controllability in a model? Look like? It's a really interesting concept, fascinating concept. What is that? Is that the recursive, the recursiveness gives you that? Or how how do you?

• 42:37Speaker 01

  It's um, quite a few things. It's like, what what does the model factor, uh, factorize? You know, what are the, what is the model representing as different pieces in the puzzle? Like, you know, so, so in the R C N uh, network, it it thinks of the world, you know, so for example, the background of an image is modeled separately from the foreground of the image.

• 42:58Lex Fridman?

  Got it.

• 42:59Speaker 01

  So the the objects are separate from the background. They are different entities.

• 43:02Lex Fridman?

  So there's a kind of segmentation that's built in fundamentally to this.

• 43:06Speaker 01

  And and and then even that object is composed of parts. And also, and another one is the the shape of the object uh, is differently modeled from the texture of the object.

• 43:19Lex Fridman?

  Got it. So there's like these, um, I've been, you know who Francois Chollet is?

• 43:25Speaker 01

  Yeah.

• 43:26Lex Fridman?

  He's, so there's, uh, he developed this like IQ test type of thing for arc challenge for, and it's kind of cool that there's, um, these concepts, priors that he defines that you bring to the table in order to be able to reason about basic shapes and things in IQ test.

• 43:45Speaker 01

  Right.

• 43:46Lex Fridman?

  So here you're making it quite explicit that here, here are the things that you should be, these are like distinct things that you should be able to, um, model in this.

• 43:57Speaker 01

  Keep in mind that you, you can derive this from much more general principles. It doesn't, you don't need to explicitly put it as, oh, objects versus foreground versus background, uh, the surface versus texture. No, these are, these are derivable from more fundamental principles of how, you know, what's the property of continuity of natural signals?

• 44:19Lex Fridman?

  What's the property of continuity of natural signals?

• 44:22Speaker 01

  Yeah.

• 44:23Lex Fridman?

  By the way, that sounds very poetic, but yeah. Uh, so you're saying that's a, there, there's some low level properties from which emerges the idea that shapes should be different than, like, uh, there should be a parts of an object, there should be, I mean,

• 44:37Speaker 01

  Exactly.

• 44:37Lex Fridman?

  Kind of like Francois, I mean, there's objectness, there's all these things that it's kind of crazy that we humans, uh, I guess evolved to have because it's useful for us to perceive the world. Yeah.

• 44:49Speaker 01

  Correct. And it derives mostly from the properties of natural signals. And, and so, um,

• 44:56Lex Fridman?

  Natural signals. So natural signals are the kind of things we'll perceive in the, in the natural world.

• 45:02Speaker 01

  Correct.

• 45:02Lex Fridman?

  I don't know. I don't, I don't know why that sounds so beautiful. Natural signals. Yeah.

• 45:06Speaker 01

  As opposed to a QR code, right? Which is an artificial signal that we created. Humans are not very good at classifying QR codes. We are very good at saying something is a cat or a dog, but not very good at, you know, the classification. The computers are very good at classifying QR codes. Um, so our, our visual system is tuned for, Natural signals and there are fundamental assumptions in the architecture that are derived from natural signals properties.

• 45:32Lex Fridman?

  I wonder when you take hallucinogenic drugs, does that go into natural? Or is that closer to QR code?

• 45:39Speaker 01

  It's still natural.

• 45:41Lex Fridman?

  It's still natural?

• 45:42Speaker 01

  Yeah, because it is still operating using your brains.

• 45:45Lex Fridman?

  By the way, on that, on that topic, I mean, I haven't been following. I think they're becoming legalized in certain, I can't wait until they become legalized to the degree that you like vision scientists could study it.

• 45:57Speaker 01

  Yeah.

• 45:57Lex Fridman?

  Just like through, through medical, chemical ways modify. There could be ethical concerns, but modify, that's another way to study the brain is to be, be able to chemically modify it. There's probably, um, probably very long a way to, to figure out how to do it ethically.

• 46:16Speaker 01

  Yeah, but I, I think there are studies on that already.

• 46:19Lex Fridman?

  Already?

• 46:20Speaker 01

  Yeah, I think so, uh, because it's, it's not unethical to give it to rats.

• 46:25Lex Fridman?

  Oh, that's true. That's true.

• 46:29Lex Fridman?

  There's a lot of drugged up rats out there. Okay, cool. Sorry, sorry to, it's okay. So there's a, so there's these, uh, low level, uh, things from natural signals that, uh, that, that, uh, that can.

• 46:43Speaker 01

  From which these properties will emerge.

• 46:45Lex Fridman?

  Yes.

• 46:46Speaker 01

  Uh, but it is still a very hard problem on how to encode that, you know? So you don't, you know, there is no, uh, so, uh, you mentioned, um, the, the, the priors, uh, Francher wanted to encode in, uh, in the, uh, abstract reasoning challenge, but it is not straightforward how to encode those priors. Um, so, so some of those, uh, challenges, like, you know, the object record or completion challenges are things that we purely use our visual system to do. It is, uh, it looks like abstract reasoning, but it is purely an output of the, the vision system. For example, completing the corners of that Kynza triangle, completing the lines of that Kynza triangle. It's a purely a visual system property. It, you know, it's, there is no abstract reasoning involved.

• 47:28Speaker 01

  It, it uses all these priors, but it is stored in our visual system in a particular way that is amenable to inference. And and and that is one of the things that we tackled in the, you know, basically saying, okay, these are the prior knowledge uh which which will be derived from the world. But then how is that prior knowledge represented in the model such that inference, when when some piece of evidence comes in can be done very efficiently and in a very distributed way? Um because it is very, there are so many ways of representing knowledge, which is not amenable to very quick inference, you know, quick lookups.

• 48:07Speaker 01

  Uh and so that's one um core part of what we tackled in uh the RCN model. Um uh how do you encode visual knowledge to uh do very quick inference and yeah?

• 48:20Lex Fridman?

  Can you maybe comment on uh so folks listening to this and in general may be familiar with different kinds of architectures of neural networks? What what are we talking about with the RCN? Uh what what does the architecture look like? What are the different components? Is it close to neural networks? Is it far away from neural networks? What does it look like?

• 48:40Speaker 01

  Yeah, so so you can uh think of the delta between the model and a convolutional neural network. If if people are familiar with convolutional neural networks. So convolutional neural networks have this feed forward processing cascade, which is called uh feature detectors and pooling. And that is repeated in the in the hierarchy, in in a multi-level uh system. Um and if you if you want to, and intuitive idea of what what is happening, feature detectors are uh you know, detecting interesting co-occurrences in the input. It can be a line, a corner, a an eye, or a piece of texture, etc. And the pooling neurons are doing some local transformation of that

• 49:23Speaker 01

  and making it invariant to local transformations. So this is what the structure of convolutional neural network is. Um recursive cortical network has a similar structure when you look at just the feed forward pathway. But in addition to that, it is also structured in a way that it is generative.

• 49:41Speaker 01

  So that it can run it backward and combine the forward with the backward. Another aspect that it has is it has lateral connections. These lateral connections, which is between, so if you have an edge here and an edge here, it has connections between these edges. It is not just feed forward connections. It is something between these edges, which is the nodes representing these edges, which is to enforce compatibility between them. So otherwise, what will happen is that

• 50:11Lex Fridman?

  A constraints?

• 50:12Speaker 01

  It's a constraint. It's basically, if you, if you do just feature detection followed by pooling, then your, your transformations in different parts of the visual field are not coordinated. And so you can, you will create jagged, when you, when you generate from the model, you will create jagged things and uncoordinated transformations. So these lateral connections are enforcing the transformations.

• 50:39Lex Fridman?

  Is the whole thing still different, Shabu?

• 50:41Speaker 01

  Uh, no.

• 50:42Lex Fridman?

  Okay.

• 50:43Speaker 01

  No. It's not, it's not trained using backprop.

• 50:47Lex Fridman?

  Okay, that's really important. So, uh, so there's these feed forward, there's feedback mechanisms. There's some interesting connectivity things. It's still layered, like a

• 50:56Speaker 01

  Yes, there are multiple levels, yes.

• 50:58Lex Fridman?

  Multiple, multiple layers. Okay, very, very interesting. And yeah, okay, so the interconnection between adjacent, so connections across service constraints.

• 51:10Speaker 01

  Yeah.

• 51:10Lex Fridman?

  That, that keep the thing stable.

• 51:12Speaker 01

  Correct.

• 51:13Lex Fridman?

  Okay, so what else?

• 51:15Speaker 01

  And then there's this idea of doing inference. A neural network does not do inference on the fly. So an example of why this inference is important is, you know, so one of the first applications that we showed in the paper was to crack text-based captures.

• 51:34Lex Fridman?

  What are captures, by the way?

• 51:36Speaker 01

  Uh, yeah.

• 51:37Lex Fridman?

  By the way, one of the most awesome, like, the people don't use this term anymore as human computation, I think. Uh, I love this term. The guy who created captures, I think, came up with this term.

• 51:48Speaker 01

  Yeah.

• 51:48Lex Fridman?

  I love it. Anyway.

• 51:49Speaker 01

  Yeah.

• 51:50Lex Fridman?

  Uh, what, what are captures?

• 51:53Speaker 01

  So captures are those strings that you fill in, uh, when you're, you know, when, if you're open, opening a new account in Google, they show you a picture, uh, you know, usually it used to be set of garbled letters, uh, that you have to kind of, uh, figure out what, what, what is that string of characters and type in. And the reason captures exist is because, you know, um, Google or Twitter do not want automatic creation of accounts. You can use a computer to create millions of accounts, uh, and, uh, use that for nefarious purposes. Uh, so you want to make sure that to the extent possible, the interaction that, yeah, their system is having is with a human.

• 52:35Speaker 01

  So it's a, it's called a human interaction proof. A capture is a human interaction proof.

• 52:40Lex Fridman?

  Yeah.

• 52:40Speaker 01

  So, so this is, a captures are by design, things that are easy for humans to solve, but hard for computers.

• 52:48Lex Fridman?

  Hard for robots, yeah.

• 52:49Speaker 01

  Yeah. Uh, so, and text-based captures were, was the one which is prevalent and around two thousand fourteen. Because at that time, text-based captures were hard for computers to crack. Even now, they are actually, in the sense of an arbitrary text-based capture will be unsolvable even now. But with the techniques that we have developed, it can be, you know, you can quickly develop a mechanism that solves, uh, the capture.

• 53:16Lex Fridman?

  They, they've probably gotten a lot harder too. The people, they've, they've, they've been getting clever and clever generating these text captures, yeah.

• 53:23Speaker 01

  Correct, right.

• 53:23Lex Fridman?

  So, okay, so that was one of the things you've tested it on is these kinds of captures in two thousand fourteen, fifteen, that kind of stuff.

• 53:30Speaker 01

  Right, right.

• 53:31Lex Fridman?

  So what, uh, I mean, why, by the way, why captures?

• 53:35Speaker 01

  Yeah, yeah. Even now, I would say capture is a very, very good challenge problem. Uh, if you want to understand how human perception works and if you want to build, uh, systems that work like the human brain. Uh, and I wouldn't say capture is a solved problem. We have cracked the fundamental defense of captures, but it is not solved in the way that humans solve it. Um, so I can give an example. I can, um, take a five-year-old child who has just learned characters, uh, and, uh, show them any new capture that we create.

• 54:10Lex Fridman?

  Mm-hmm.

• 54:10Speaker 01

  They will be able to solve it. I can show you pretty much any new capture from any new website. You'll be able to solve it without getting any training examples from that particular style of capture.

• 54:23Lex Fridman?

  You're assuming I'm human, yeah.

• 54:24Speaker 01

  Yes, yeah, that's right. So if you are human, otherwise I will be able to figure that out using this one.

• 54:33Lex Fridman?

  But. This whole podcast is just a touring test, a long touring test. Anyway, I'm sorry. So yeah, so humans can figure it out with very few examples.

• 54:43Speaker 01

  Or no training examples, like no training examples from that particular style of capture. And so you can, you know, so even now, this is unreachable for the current deep learning system. So basically, there is no, I don't think a system exists where you can basically say, train on whatever you want. And then now say, hey, I will show you a new captcha, which I did not show you in the in the training setup. Will the system be able to solve it? It still doesn't exist. So that is the magic of human perception. Yeah, and Doug Hafstadter put this very beautifully in one of his talks. The the central problem in AI is what is the letter A?

• 55:27Speaker 01

  If you can, if you can build a system that reliably can detect all the variations of the letter A, you don't even need to go to the.

• 55:37Lex Fridman?

  The V and the C, yeah.

• 55:38Speaker 01

  Yeah, you don't even know to go to the V and the C or the strings of characters. And so that that is the spirit at which, you know, with which we tackle that problem.

• 55:46Lex Fridman?

  What does it mean by that? I mean, is it like without training examples, try to figure out the fundamental elements that make up the letter A in all of its forms?

• 56:01Speaker 01

  In all of its forms. It can be, A can be made with the two humans standing leaning against each other, holding the hands. And it can be made of leaves, it can be.

• 56:09Lex Fridman?

  Yeah, you might have to understand everything about this world in order to understand the letter A. Yeah, so it's common sense reasoning essentially, yeah.

• 56:17Speaker 01

  Right, so so to finally, to really solve, finally to say that you have solved captcha, you have to solve the whole problem.

• 56:28Lex Fridman?

  Yeah, okay, so what, how does this kind of the R C N architecture help us to get do a better job of that kind of thing?

• 56:36Speaker 01

  Yeah, so as I mentioned, one of the important things was being able to do inference, being able to dynamically do inference.

• 56:44Lex Fridman?

  Can you, can you, can you clarify what you mean? Because you said like neural networks don't do inference.

• 56:50Speaker 01

  Yeah.

• 56:50Lex Fridman?

  So what do you mean by inference in this context then?

• 56:53Speaker 01

  So, okay, so in captures what they do to confuse people is to make these characters crowd together.

• 57:00Lex Fridman?

  Yes.

• 57:01Speaker 01

  Okay, and when you make the characters crowd together, what happens is that you will now start seeing combinations of characters as some other new character or or an existing character. So you would, you would put an R and N together, it will start looking like an M. And, and so locally, they are, you know, there is very strong evidence for it being some incorrect character. But globally, the only explanation that fits together is something that is different from what you find locally.

• 57:33Lex Fridman?

  Yes.

• 57:33Speaker 01

  So, so, so this is inference. You are basically taking local evidence and putting it in the global context and often coming to a conclusion locally, which is conflicting with the local information.

• 57:47Lex Fridman?

  So actually, so you mean inference like in the way it's used when you talk about reasoning, for example, as opposed to like inference, which is this with neural, with artificial neural networks, which is a single pass to the network.

• 58:01Speaker 01

  Correct.

• 58:01Lex Fridman?

  Okay, so like you're basically doing some basic forms of reasoning.

• 58:05Speaker 01

  Correct.

• 58:06Lex Fridman?

  Like integration of like how local things fit into the the global picture.

• 58:11Speaker 01

  And, and, and things like explaining away coming into this one, because you are, you are explaining that piece of evidence as something else, because globally, that's the only thing that makes sense. So now, you can.

• 58:27Speaker 01

  Amortize this inference by, you know, in a neural network, if you want to do this, what you can, you can brute force it. You can just show it all combinations of things that you want to, you want to, your reasoning to work over. And you can, you know, like, just train the help out of that neural network. And it will look like it is doing, you know, inference on the fly, but it is, it is really just doing amortized inference. It is because you, you have shown it a lot of these combinations during training time. So what you want to do is be able to do dynamic inference, rather than just being able to show all those combinations in the training time. And that's something we emphasized in the model.

• 59:09Lex Fridman?

  What does it mean dynamic inference? Is that, that has to do with the feedback thing?

• 59:13Speaker 01

  Yes.

• 59:14Lex Fridman?

  Like, what, what is dynamic? I'm trying to visualize what dynamic inference would be in this case, like, what is it doing with the input? It's shown the input the first time.

• 59:25Speaker 01

  Yeah.

• 59:25Lex Fridman?

  Is, is like, what's changing over temporally over, what's the dynamics of this inference process?

• 59:32Speaker 01

  So, so you can think of it as you have, at the top of the model, the characters that you are trained on.

• 59:38Lex Fridman?

  Yeah.

• 59:38Speaker 01

  They are the causes. You're trying to explain the pixels using the characters as the causes. The, you know, the characters are the things that cause the pixels.

• 59:50Lex Fridman?

  Yeah, so there's this causality thing. So the reason you mentioned causality, I guess, is because there's a temporal aspect to this whole thing.

• 59:58Speaker 01

  In this particular case, the temporal aspect is not important. It is more like when, if, if I turn the character on, the, the pixels will turn on. Yeah, it will be after this a little bit, but.

• 1:00:08Lex Fridman?

  Okay, so that is causality in the sense of like a logic causality, like hence inference, okay.

• 1:00:14Speaker 01

  The dynamics is that even though locally, it will look like, okay, this is an A. And, and locally, just when I look at just that patch of the image, it looks like an A. But when I look at it in the context of all the other causes, it might not, A is not the something that makes sense. So that is something you have to kind of, you know, recursively figure out.

• 1:00:38Lex Fridman?

  Yeah. So, okay, so, and this thing performed pretty well on the captures.

• 1:00:43Speaker 01

  Correct.

• 1:00:44Lex Fridman?

  And.

• 1:00:46Lex Fridman?

  I mean, is there some kind of interesting intuition you can provide why I did well? Like, what did it look like? Is there visualizations that could be human interpretable to us humans?

• 1:00:57Speaker 01

  Yes, yeah, so the good thing about the model is that it is extremely, so it is not just doing a classification, right? It is, it is, it is, it is providing a full explanation for the scene. So when, when it, when it operates on a scene, it is coming at back and saying, look, this is the part is the A, and these are the pixels that turned on. These are the pixels in the input that tells, makes me think that it is an A. And also, these are the portions I hallucinated. It, it, you know, it, it provides a complete explanation of that form. And then, this is, these are the contours, this is, this is the interior. And this is in front of this other object.

• 1:01:40Speaker 01

  So that, that's the kind of explanation it, the inference network provides. So, so that, that is useful and interpretable. And then the kind of errors it makes are also, I don't want to read too much into it, but the kind of errors the network makes are very similar to the kinds of errors humans would make in a, in a similar situation.

• 1:02:07Lex Fridman?

  So there's something about the structure that feels reminiscent of the way humans' visual system works. Well, I mean, how hard-coded is this to the capture problem, this idea?

• 1:02:21Speaker 01

  Not really hard-coded, because it's the, the assumptions, as I mentioned, are general, right? It is more, and, and those assumptions can be applied in many situations which are natural signals. So it's, it's the foreground versus background factorization, and the factorization of the surfaces versus the contours. So these are all generally applicable assumptions.

• 1:02:45Lex Fridman?

  In, in all vision. So why, why capture?

• 1:02:49Lex Fridman?

  Why attack the capture problem, which is quite unique in the computer vision context versus like the traditional benchmarks of ImageNet and all those kinds of image classification or even segmentation tasks, all that kind of stuff? Do you feel like that's, I mean, what's your thinking about those kinds of benchmarks in this context?

• 1:03:10Speaker 01

  I mean, those benchmarks are useful for deep learning kind of algorithms where you, you know, so the settings that deep learning works in are here is my huge training set and here is my test set. So the, the training set is almost, you know, 100x, 1000x bigger than the test set in many, many cases. What we wanted to do was invert that. The training set is way smaller than the, the test set.

• 1:03:39Lex Fridman?

  Yes.

• 1:03:40Speaker 01

  And, and, you know, capture is a problem that is by definition hard for computers. And it has these good properties of strong generalization, strong out of training distribution generalization. If you are interested in studying that and putting, having your model have that property, then it's a, it's a good data set to tackle.

• 1:04:04Lex Fridman?

  So is there, have you attempted to, which I think, I believe there's quite a growing body of work on looking at MNIST and ImageNet without training. So like taking, like the basic challenge is how, what tiny fraction of the training set can we take in order to do a reasonable job of the classification task? Have, have you explored that angle on these classic benchmarks?

• 1:04:32Speaker 01

  Yes, so, so we did do MNIST, so, you know, so it's not just capture. We, so there was also versions of, multiple versions of MNIST, including the, the standard version, which where we inverted the problem, which is basically saying rather than train on 60,000 training data, you know, how quickly can you get to high level accuracy with very little training data?

• 1:04:57Lex Fridman?

  Was, is there some performance that you remember, like how well, how well did it do? How many examples did it need?

• 1:05:04Speaker 01

  Yeah, I, I, like, you know, I remember that it was, you know, on the order of

• 1:05:12Speaker 01

  Tens or hundreds of examples to get into ninety five percent accuracy and it was it was definitely better than the systems, other systems out there at that time.

• 1:05:21Lex Fridman?

  At that time.

• 1:05:21Speaker 01

  Yeah.

• 1:05:22Lex Fridman?

  Yeah, they're really pushing. I think that's a really interesting space, actually. I think there's an actual name for MNIST that like there's different names, the different sizes of training sets. I mean, people are like attacking this problem. I think it's super interesting.

• 1:05:40Speaker 01

  Yeah.

• 1:05:40Lex Fridman?

  It's funny how like the MNIST will probably be with us all the way to AGI.

• 1:05:46Speaker 01

  Yes.

• 1:05:47Lex Fridman?

  It's a data set that just sits by. It is. It's a clean, simple data set to to study the fundamentals of learning with, just like CAPTCHES. It's interesting. Not enough people. I don't know, maybe you can correct me, but I feel like CAPTCHES don't show up as often in papers as they probably should.

• 1:06:05Speaker 01

  That's correct. Yeah, because, you know, usually these things have a momentum, you know, once, once something gets established as a standard benchmark.

• 1:06:16Lex Fridman?

  Yeah.

• 1:06:16Speaker 01

  There is a, there is a, there is a dynamics of how graduate students operate and how academic, academic system works that pushes people to track that benchmark. So.

• 1:06:28Lex Fridman?

  Yeah.

• 1:06:29Speaker 01

  Yeah.

• 1:06:30Lex Fridman?

  Nobody wants to think outside the box. Okay.

• 1:06:32Speaker 01

  Yes.

• 1:06:34Lex Fridman?

  Okay, so good performance on the CAPTCHES. What else is there interesting on the RCN side before we talk about the cortical micro?

• 1:06:43Speaker 01

  Yeah, so the same model, so the, the, the important part of the model was that it trains very quickly with very little training data and it's, you know, quite robust to out of distribution perturbations. And, and we are using that very fruitfully in, at various in many of the robotic tasks we are solving.

• 1:07:05Lex Fridman?

  Solving that. Well, let me ask you this kind of touchy question. I have to, I've spoken with your friend, colleague Jeff Hawkins, too. I mean, he's, I have to kind of ask, there is a bit of. Whenever you have brain-inspired stuff.

• 1:07:21Speaker 01

  Yeah.

• 1:07:22Lex Fridman?

  And you make big claims.

• 1:07:23Speaker 01

  Yeah.

• 1:07:24Lex Fridman?

  Big sexy claims.

• 1:07:25Speaker 01

  Yeah.

• 1:07:25Lex Fridman?

  There's a, you know, uh, there's critics, I mean, machine learning subreddit. Don't get me started on those people. They're hard, I mean, criticism is good, but they're a bit, they're a bit over the top. There is quite a bit of sort of skepticism and criticism, you know, is this work really as good as it promises to be?

• 1:07:47Speaker 01

  Yeah.

• 1:07:47Lex Fridman?

  What, do you have thoughts on that kind of skepticism? Do you have comments on the kind of criticism we might have received about, you know, is this approach legit? Is this, is this a promising approach?

• 1:08:01Speaker 01

  Yeah.

• 1:08:02Lex Fridman?

  Or at least as promising as it seems to be, you know, advertised as?

• 1:08:07Speaker 01

  Yeah, I can comment on it. So, you know, our, our scene paper is published in Science, which I would argue is, is a very high-quality journal, very hard to publish in. And you, you know, usually it is indicative of the, of the quality of the work. And, um, uh, I can, I can, I, I am very, very certain that the ideas that we brought together in that paper, uh, in terms of the importance of feedback connections, uh, recursive inference, lateral connections, uh, coming to best explanation of the scene as the problem to solve, trying to solve, uh, recognition, segmentation, uh, all jointly in a way that is compatible with higher-level cognition, top-down attention, all those ideas that we brought together into something,

• 1:08:52Speaker 01

  you know, coherent and workable in the, uh, in the world and solving a challenging, tackling a challenging problem. I think that will, that will stay and that, that contribution I stand by, right? Now, uh, I can, I can tell you a story which is funny in the, in the context of this, right? So if you read the abstract of the paper and like, you know, the argument we are putting in, you know, we are putting in, look, current deep learning systems take a lot of training data, uh, they don't use these insights. And here is a new model, which is not a deep neural network, it's a graphical model, it does inference, this is how the paper is, right? Now, once the paper was accepted and everything, um, it went to the press department in, in Science, you know, to play as Science Office. We, we didn't do any press release when it was published. It was, it went to the press department.

• 1:09:36Speaker 01

  What did the, what was the press release that they wrote up? A new deep learning model.

• 1:09:42Lex Fridman?

  Solves captures.

• 1:09:43Speaker 01

  Solves captures. And so, so you can see where was, you know, what, what was being hyped in that thing, right? So, so it's like, um, there is the, there is a dynamic in the, uh, in the community of, you know, so, um, that's especially happens when there are lots of new people coming into the field and they get attracted to one thing. And some people are trying to think different compared to that. So there is, there is some, uh, I, I think skepticism is science is important and it is, uh, um, you know, very much, uh, required. But it's also, it's not, uh, skepticism usually, it's mostly bandwagon effect that is happening rather than.

• 1:10:22Lex Fridman?

  Well, well, but that's not even that. I mean, I'll tell you what they react to, which is like, I'm sensitive to as well. If you, if you look at just companies, OpenAI, DeepMind.

• 1:10:32Speaker 01

  Yeah.

• 1:10:33Lex Fridman?

  Um, Vicarious, I mean, they just, there's a, there's a little bit of a race to the top in hype, right? It's, it's like, it doesn't pay off to be humble.

• 1:10:47Speaker 01

  Right.

• 1:10:48Lex Fridman?

  So like, uh, and, and the press is just irresponsible often. They, they just, I mean, don't get me started on the state of journalism today. Like, it seems like the people who write articles about these things, they literally have not even spent an hour on the Wikipedia article about what is neural networks. Like, they haven't like invested just even the language, the laziness. It's like, uh, robots beat humans. Like, they, they write this kind of stuff that just, uh, and then, and then of course, the researchers are quite sensitive to that, uh, because it gets a lot of attention. They're like, why did this work get so much attention?

• 1:11:32Lex Fridman?

  Uh, you know, that's, that's over the top and people get really sensitive, you know, the same kind of criticism with, um, OpenAI did work with the Rubik's Cube with the robot that people criticized. Uh, same with GPT two and three. They criticize, uh, same thing with, uh, DeepMinds with AlphaZero. I mean, yeah, I, I'm sensitive to it, uh, but.

• 1:11:55Lex Fridman?

  And of course, with your work, you mentioned deep learning, but there's something super sexy to the public about brain inspired. I mean, that immediately grabs people's imagination, not even like neural networks, but like really brain inspired, like like brain like neural networks. That seems really compelling to people and to me as well to the world as a narrative. And so people hook up, hook on to that. And sometimes you, the skepticism engine turns on in the research community and they're skeptical. But I think putting aside the ideas of the actual

• 1:12:37Lex Fridman?

  performance on captures or performance any data set. I mean, to me, all these data sets are useless anyway. It's nice to have them. But in the grand scheme of things, they're silly toy examples. The point is, is there intuition about the ideas, just like you mentioned, bringing the ideas together in a unique way? Is there something there? Is there some value there? And is it going to stand the test of time?

• 1:13:03Speaker 01

  Yes.

• 1:13:03Lex Fridman?

  And that's the hope. That's the hope.

• 1:13:05Speaker 01

  I'm, my confidence there is very high. I, you know, I don't treat brain inspired as a marketing term. You know, I am looking into the details of biology and I'm puzzling over those things and I am grappling with those things. And so it is, it is not a marketing term at all. It, you know, you can use it as a marketing term and, and people often use it. And you can get combined with them. And when, when people don't understand how we are approaching the problem, it is, it is easy to be misunderstood and, you know, think of it as, you know, purely marketing, but that's not the way we are.

• 1:13:43Lex Fridman?

  So you really. I mean, as a scientist, you believe that if we kind of just stick to really understanding the brain, that's going to, that's the right, like, you should constantly meditate on the, how does the brain do this? Because that's going to be really helpful for engineering and intelligent systems.

• 1:14:03Speaker 01

  Yes, you need to, so I think it is, it's one input and it is, it is helpful, but you, you should know when to deviate from it, too. So, an example is convolutional neural networks, right? Convolution is not an operation brain implements. The visual cortex is not convolutional. Visual cortex has local receptive fields, local connectivity, but, you know, the, there is no translation invariance in the, the network weights in, in the visual cortex. That is a computational trick, which is a very good engineering trick that we

• 1:14:44Speaker 01

  use for sharing the training between the different nodes. So, and, and that trick will be with us for some time. It will go away when we have robots with eyes and heads that move. And so then that trick will go away. It will not be useful at that time.

• 1:15:05Lex Fridman?

  So the brain doesn't, so the brain doesn't have translational invariance. It has the focal point, like it has a thing it focuses on.

• 1:15:13Speaker 01

  Correct. It has, it has a phobia. And because of the phobia, the receptive fields are not like the copying of the weights, like the, the weights in the center are very different from the weights in the periphery.

• 1:15:25Lex Fridman?

  Yes, at the periphery. I mean, I did this, actually wrote a paper and just got the chance to really study peripheral, peripheral vision, which is a fascinating thing. Very under, understood thing of what the, you know, at the every level, the brain does with the periphery. It does some funky stuff. So it's, it's another kind of trick than convolutional, like it does. It's, you know, convolution in neural networks is a trick for efficiency, is efficiency trick. And the brain does a whole nother kind of thing, I guess.

• 1:16:04Speaker 01

  Correct, correct. So you need to understand the principles of processing so that you can still apply engineering tricks.

• 1:16:12Lex Fridman?

  Yeah.

• 1:16:13Speaker 01

  Where you want it to. You don't want to be slavishly mimicking all the things of the brain. And so, yeah, so it should be one input, and I think it is extremely helpful. But it should be the point of really understanding so that you know when to deviate from it.

• 1:16:28Lex Fridman?

  So, okay, that's really cool. That's work from a few years ago. So you did work in Umenta with Jeff Hawkins.

• 1:16:36Speaker 01

  Yeah.

• 1:16:37Lex Fridman?

  With hierarchical temporal memory. How is your, just if you could give a brief history, how is your view of the way the models of the brain changed over the past few years leading up to now? Is there some interesting aspects where there was an adjustment to your understanding of the brain? Or is it all just building on top of each other?

• 1:17:01Speaker 01

  In terms of the higher level ideas, especially the ones Jeff wrote about in the book, if you blur out, right?

• 1:17:08Lex Fridman?

  Yeah, unintelligence.

• 1:17:09Speaker 01

  Right, unintelligence. If you blur out the details and if you just zoom out and at the higher level idea, things are, I would say, consistent with what he wrote about. But many things will be consistent with that because it's a blur. You know, when you, you know, deep learning systems are also, you know, multi-level, hierarchical, all of those things, right? So, but in terms of the detail, a lot of things are different. And those details matter a lot. So, so one point of difference I had with Jeff was how to approach, you know, how much of biological possibility and realism do you want in the learning algorithms?

• 1:17:53Speaker 01

  Um. So uh when I was there, uh this was, you know, almost ten years ago now. So yeah.

• 1:18:00Lex Fridman?

  It flies when you're having fun.

• 1:18:01Speaker 01

  I don't know. I don't know what just thinks now, but ten years ago, uh the difference was that I did not want to be so constrained on saying uh my learning algorithms want to need to be biologically plausible uh based on some filter of biological possibility available at that time. To me, that is a dangerous cut to make because we are, you know, discovering more and more things about the brain all the time. New biophysical mechanisms, new channels uh are being discovered all the time. So I don't want to upfront kill off an uh a learning algorithm just.

• 1:18:38Lex Fridman?

  Because we don't really understand the full uh the full uh biophysics or whatever of how the brain learns.

• 1:18:45Speaker 01

  Exactly, exactly.

• 1:18:46Lex Fridman?

  But let me ask that, Sergeant Trump, like, what's our, what's your sense, what's our best understanding of how the brain learns?

• 1:18:54Speaker 01

  So things like backpropagation, credit assignment. So so many of these algorithms have, learning algorithms have things in common, right? It is, uh backpropagation is one way of credit assignment. There is another algorithm called expectation maximization, which is, you know, uh another weight adjustment algorithm.

• 1:19:13Lex Fridman?

  But is it your sense the brain does something like this?

• 1:19:16Speaker 01

  Has to. There is no way around it in the sense of saying that you do have to adjust the the connections.

• 1:19:23Lex Fridman?

  So yeah, and you're saying credit assignment, you have to reward the connections that were useful in making a correct prediction and not. Yeah, I guess what else? But yeah, it doesn't have to be differentiable. I mean.

• 1:19:35Speaker 01

  Yeah, it doesn't have to be differentiable.

• 1:19:36Lex Fridman?

  Yeah.

• 1:19:36Speaker 01

  Yeah, but you have to have a, you know, you have a model that you start with, you have data comes in, and you have to have a way of adjusting the model such that it better fits the data.

• 1:19:49Lex Fridman?

  Yeah.

• 1:19:49Speaker 01

  So that that is all of learning, right? And and some of them can be using backprop to do that. Some of it can be using uh, you know, very local uh graph changes to do that. Uh, there can be, you know, uh, many of these learning algorithms have similar update properties locally in terms of what the neurons need to do locally.

• 1:20:14Lex Fridman?

  I wonder if small differences in learning algorithms can have huge differences in the actual effect. So the dynamics of. I mean, sort of the reverse, like spiking, like if credit assignment is like a lightning versus like a rainstorm or something like, whether, whether there's a, like a looping local type of situation with the credit assignment.

• 1:20:42Speaker 01

  Yeah.

• 1:20:43Lex Fridman?

  Whether there is like regularization, like how, how, how it injects robustness into the whole thing, like whether it's chemical or electrical or mechanical.

• 1:20:57Speaker 01

  Yeah.

• 1:20:58Lex Fridman?

  All those kinds of things.

• 1:20:59Speaker 01

  Yes.

• 1:21:00Lex Fridman?

  I feel like it, that, yeah, I feel like those differences could be essential, right?

• 1:21:06Speaker 01

  It could be, it's just that you don't know enough to, on the learning side, you don't know enough to say, that is definitely not the way the brain does it.

• 1:21:18Lex Fridman?

  Got it. So you don't want to be stuck to it.

• 1:21:20Speaker 01

  Correct.

• 1:21:20Lex Fridman?

  So that, yeah, so you, you've been open minded on that side of things.

• 1:21:24Speaker 01

  Correct. On the inference side, on the recognition side, I am much more amenable to being constrained because it's much easier to do experiments because, you know, it's like, okay, here's a stimulus, you know, how many steps did it get to take the answer? I can trace it back, I can, I can understand the speed of that computation, et cetera, much more readily on the inference side.

• 1:21:45Lex Fridman?

  Got it. And then you can't do good experiments on the learning side.

• 1:21:48Speaker 01

  Correct.

• 1:21:50Lex Fridman?

  So that, let's, let's go right into cortical microcircuits right back. So what, what are these ideas beyond recursive cortical network that you're looking at now?

• 1:22:02Speaker 01

  So we have made a, you know, pass through, you know, multiple of the steps that, you know, as I, as I mentioned earlier, you know, we were looking at perception from the angle of cognition, right? It was not just perception for perception's sake. How do you, how do you connect it to cognition? How do you learn concepts? And how do you learn abstract reasoning, similar to some of the things Francois talked about, right? So.

• 1:22:29Speaker 01

  So we have taken one pass through it, basically saying, what is the basic cognitive architecture that you need to have, which has a perceptual system, which has a system that learns dynamics of the world, and then has something like a routine program learning system on top of it to learn concepts. So we have, we have built one, you know, the version point one of that system. Uh, this was another uh, science robotics paper. Uh, it is, it's the title of that paper was, you know, something like cognitive programs. How do you build cognitive programs? Uh, and?

• 1:23:06Lex Fridman?

  And the application there was on manipulation, robotic manipulation?

• 1:23:10Speaker 01

  It was, um, so think of it like this. Suppose you wanted to tell a new person uh, that you met. You don't know the language uh, that person uses. You want to communicate to that person uh, to achieve some task, right? So I want to say, hey, um, you need to pick up all the, the red cups from the kitchen counter and put it here, right? Uh, how do you communicate that, right? You can show pictures. You can basically say, look, this is the starting state. The, the things are here. This is the ending state. And, and what does the person need to understand from that? The, the person need to understand what conceptually happened in those pictures from the input to the output, right?

• 1:23:52Speaker 01

  Um, so, um, so we are looking at pre-verbal conceptual understanding. Without language, how do you, how do you have a set of concepts that you can manipulate in your head? Uh, and from a, you know, set of images of input and output, can you infer what is happening in those images?

• 1:24:14Lex Fridman?

  Got it. With concepts that are pre-language, okay. So what, what does it mean to, for a concept to be pre-language, like?

• 1:24:21Speaker 01

  Yeah.

• 1:24:21Lex Fridman?

  Why, why so, why, why is language so important here?

• 1:24:27Speaker 01

  So, I, I want to make a distinction between concepts that are just learned from text. By, by just, just feeding brute force text. Uh, you can, you can start extracting things like, okay, a cow is likely to be on grass. Uh, in, in, uh, so those kinds of things you can extract purely from text. Um, uh, but that's kind of a simple association or thing rather than a concept as an abstraction of something that happens in the real world, you know, in a grounded way that I can, I can simulate it in my mind and connect it back to the real world.

• 1:25:06Lex Fridman?

  And, and you think kind of the visual, uh, the visual world, concepts in the visual world are somehow lower level than just the language?

• 1:25:16Speaker 01

  The lower level kind of makes it feel like, okay, that's like an unimportant, like it's more like, uh, I would say, uh, the concepts in the visual and motor system and, you know, the, uh, the concept learning system, which if you cut off the language part, just the, just what we learn by interacting with the world and abstractions from that, that is a prerequisite for any real language understanding.

• 1:25:44Lex Fridman?

  So you're, uh, so you disagree with Chomsky, because he says language is at the bottom of everything.

• 1:25:49Speaker 01

  No, I, I, I, yeah, I disagree with Chomsky completely on a whole new level, level, from, from universal grammar to, yeah.

• 1:25:57Lex Fridman?

  So that was a paper in Science beyond the recursive cortical network. Uh, what, what other interesting problems are there, the open problems and brain-inspired approaches that you're thinking about?

• 1:26:09Speaker 01

  I mean, everything is open, right? Like, no, no, no problem is, uh, solved, solved, all right? Uh, first, uh, I think of perception as kind of the, the, the first thing that you have to build. But the last thing that you will be actually solved.

• 1:26:27Speaker 01

  So because if you do not build perception system in the right way, you cannot build concept system in the right way. So you have to build a perception system, however wrong that might be, you have to still build that and learn concept from there and then, you know, keep iterating. And finally, perception will get solved fully when perception, cognition, language, all those things work together finally.

• 1:26:51Lex Fridman?

  So what, and so great, we've talked a lot about perception, but then maybe on the concept side and like common sense or just general reasoning side, is there some intuition you can draw from the brain about how we could do that?

• 1:27:08Speaker 01

  So I have, I have this classic example I give. So suppose I give you a few sentences and then ask you a question following that sentence. This is a natural language processing problem, right? So here it goes, I'm telling you Sally pounded a nail on the ceiling. Okay, that's a sentence. Now I am asking you a question, was the nail horizontal or vertical?

• 1:27:34Lex Fridman?

  Vertical.

• 1:27:35Speaker 01

  Okay, how did you answer that?

• 1:27:39Lex Fridman?

  Well, I imagined Sally, it was kind of hard to imagine what the hell she was doing, but

• 1:27:47Lex Fridman?

  but I imagined I had a visual of the whole situation.

• 1:27:51Speaker 01

  Exactly, exactly. So here, you know, I posed a question in natural language. The answer to that question was you, you got the answer from actually simulating the scene. Now I can go more and more detail about, okay, was Sally standing on something while doing this? You know, could she have been standing on a light bulb to do this? You know, I could, I could ask more and more questions about this and I can ask, make you simulate the scene in more and more detail, right? Where is all that knowledge that you're accessing stored? It is not in your language system. It is not, it was not just by reading text you got that knowledge. It is stored from the everyday experiences that you have had from,

• 1:28:34Speaker 01

  and by the, by the age of five, you, you have pretty much all of this, right? And it is stored in your visual system, motor system in a way such that it can be accessed through language.

• 1:28:47Lex Fridman?

  Got it. I mean, right. So you're, the language is just almost serves as a query into the whole visual cortex and then does the whole feedback thing. But I mean, is all reasoning kind of connected to the perception system in some way?

• 1:29:02Speaker 01

  You can do a lot of it, you know, you can still do a lot of it by quick associations without having to go into the depth. And, and most of the time you will be right, right? You can just do quick associations. But I can easily create tricky situations for you where that quick associations is wrong and you have to actually run the simulation.

• 1:29:22Lex Fridman?

  So figuring out how these concepts connect, do we have a good idea of how to do that?

• 1:29:29Speaker 01

  That's exactly what.

• 1:29:30Lex Fridman?

  That's the.

• 1:29:31Speaker 01

  One of the problems that we are working on and, and, and the, the way we are approaching that is basically saying, okay, you need to. So the, the takeaway is that language is simulation control. And your perceptual plus motor system is building a simulation of the world. And so, so that's basically the way we are approaching it. And the first thing that we built was a controllable perceptual system. And we built a schema networks, which was a controllable dynamic system. Then we built a concept learning system that puts all these things together into programs, as abstractions that you can run and simulate.

• 1:30:12Speaker 01

  And now we are taking the step of connecting it to language and, and it will be very simple examples initially. It will not be the G P T three like examples, but it will be grounded simulation based language.

• 1:30:26Lex Fridman?

  And for like the, the querying would be like question answering kind of thing?

• 1:30:31Speaker 01

  Correct, correct. And it will be in some simple world initially on, you know, but it will be about, okay, can the system connect the language? And ground it in the right way and run the right simulations to come up with the answer.

• 1:30:45Lex Fridman?

  And the goal is to try to do things that, for example, G P T three couldn't do.

• 1:30:49Speaker 01

  Correct.

• 1:30:50Lex Fridman?

  Speaking of which, if we could talk about GPT-3 a little bit, I think it's an interesting, thought-provoking set of ideas that OpenAI is pushing forward. I think it's good for us to talk about the limits and the possibilities in neural networks. So in general, what are your thoughts about this recently released, very large, 175 billion parameter language model?

• 1:31:17Speaker 01

  So I haven't directly evaluated it yet. From what I have seen on Twitter and, you know, other people evaluating it, it looks very intriguing. You know, I am very intrigued by some of the properties it is displaying. And, of course, the text generation part of that was already evident in GPT-2, you know, that it can generate coherent text over long distances. But, of course, the weaknesses are also pretty visible in saying that, okay, it is not really carrying a world state around. And, you know, sometimes you get sentences like, "I went up the hill to reach the valley," or something, you know,

• 1:31:58Speaker 01

  completely incompatible statements. Or when you're traveling from one place to the other, it doesn't take into account the time of travel, things like that. So those things, I think, will happen less than GPT-3 because it is trained on even more data, and so it can do even more longer distance coherence. But it will still have the fundamental limitations that it doesn't have a world model, and it can't run simulations in its head to find whether something is true in the world or not.

• 1:32:30Lex Fridman?

  Do you think within, so it's taking a huge amount of text from the internet and forming a compressed representation, do you think in that could emerge?

• 1:32:41Lex Fridman?

  Something that's an approximation of a world model, which essentially could be used for reasoning. And it's a, it's a, it's a, so I'm not talking about G P T three, I'm talking about G P T four, five and G P T ten.

• 1:32:54Speaker 01

  Yeah, I mean, they will look more impressive than G P T three. So you can, if you take that to the extreme, then a Markov chain of just first order, and if you, if you go to, I'm, I'm taking the other extreme. If you read Shannon's book, right? He has a model of English text, which is based on first order Markov chains, second order Markov chains, third order Markov chains, and think that, okay, third order Markov chains look better than first order Markov chains. So does that mean a first order Markov chain has a model of the world? Yes, it does. So, yes, in that level, when you go higher order models

• 1:33:36Speaker 01

  or more sophisticated structure in the model like the transformer networks have, yes, they have a model of the text world. But that is not a model of the world. It's, it's a model of the text world and it will have in, interesting properties and it will be useful. But just scaling it up is not going to give us A G I or natural language understanding or meaning.

• 1:34:06Lex Fridman?

  The, the question is whether being forced to compress a very large amount of text

• 1:34:13Speaker 01

  Yeah.

• 1:34:14Lex Fridman?

  forces you to construct things that are very much like,

• 1:34:20Lex Fridman?

  because the ideas of concepts and meaning is, is a, is a spectrum.

• 1:34:25Speaker 01

  Sure, yeah.

• 1:34:26Lex Fridman?

  So in order to form that kind of compression, maybe you will be forced to figure out.

• 1:34:36Lex Fridman?

  Abstractions which look awfully a lot like the kind of things that we think about as as concepts, as world models, as common sense. Is that possible?

• 1:34:48Speaker 01

  No, I don't think it is possible because the information is not there.

• 1:34:51Lex Fridman?

  Well, the the information is uh is there behind the text, right?

• 1:34:56Speaker 01

  No, because unless somebody has written down all the details about how everything works in the world to the the absurd amounts like, okay, it is easier to walk forward than backward. Uh, that you have to open the door to go out of the thing. Uh, doctors wear underwear. You know, uh, uh, unless all these things somebody has written down somewhere or, you know, somehow the program found it to be useful for compression from some other text. Uh, the information is not there.

• 1:35:22Lex Fridman?

  So that's an argument that like text is a lot lower fidelity than the, you know, the experience of our physical world.

• 1:35:30Speaker 01

  Right, right. Pictures worth a thousand words, like that kind of thing.

• 1:35:34Lex Fridman?

  Well, in this case, pictures aren't really. So the the richest aspect of the physical world isn't even just pictures. It's the uh, it's the interactivity with the world.

• 1:35:45Speaker 01

  Exactly.

• 1:35:46Lex Fridman?

  Yeah, it's being able to um uh interact. It's almost like

• 1:35:54Lex Fridman?

  it's almost like if you could interact. So I I I disagree. Well, maybe I agree with you that pictures were a thousand words, but a thousand.

• 1:36:03Speaker 01

  It's still, yeah, you could say you could capture it with a GPT.

• 1:36:06Lex Fridman?

  So I wonder if there's some interactive element where a system could live in text world where it could um be part of the chat, be part of, you know, talking to people. It's it's interesting. I mean, fundamentally, so you you're making a statement about the limitation of text. I, okay, let's so let's say we have a text corpus that includes basically.

• 1:36:33Lex Fridman?

  Every experience we could possibly have, I mean, just a very large corpus of text and also interactive components. I guess the question is whether the neural network architecture, these very simple transformers, but if they had like hundreds of trillions or whatever comes after trillion parameters, whether that could store the information needed. That's architecturally. Do you have thoughts about the limitation on that side of things with neural networks?

• 1:37:06Speaker 01

  I mean, so transformer is still a feed forward neural network. It has a very interesting architecture, which is good for text modeling and probably some aspects of video modeling, but it is still a feed forward architecture.

• 1:37:20Lex Fridman?

  You believe in the feedback mechanism, the recursion?

• 1:37:24Speaker 01

  Oh, and also causality, being able to do counterfactual reasoning, being able to do interventions, which is actions in the world. So all those things require different kinds of models to be built. I don't think transformers captures that family. It is very good at statistical modeling of text and it will become better and better with more data, bigger models, but that is only going to get so far. Finally, when you, so I had this joke on Twitter saying that, hey, this is a model

• 1:38:04Speaker 01

  that has read all of quantum mechanics and theory of relativity and we are asking it to do text completion or we are asking it to solve simple puzzles. That, you know, when you have AGI, that's not what you ask a system to do. If it does, you know, we ask, we'll ask the system to do experiments, you know, what and come up with hypotheses and revise the hypothesis based on evidence from experiments, all those things, right? Those are the things that we want the system to do when we have AGI, not solve with simple puzzles.

• 1:38:36Lex Fridman?

  Like impressive demo, somebody generating a red button in HTML.

• 1:38:41Speaker 01

  Right. Which are all useful, like, you know, there's no dissing the usefulness of.

• 1:38:47Lex Fridman?

  So I get, by the way, I'm playing a little bit of a devil's advocate, so calm down, internet. So I just, I'm curious almost.

• 1:38:58Lex Fridman?

  In which ways will a dumb but large neural network will surprise us?

• 1:39:05Speaker 01

  Yeah.

• 1:39:06Lex Fridman?

  So like, um, it's kind of your, I completely agree with your intuition. It's just that I don't want to dogmatically, like a hundred percent, put all the chips there.

• 1:39:18Speaker 01

  Right, right.

• 1:39:18Lex Fridman?

  We've been surprised so much, even the current GPT two and three are so surprising.

• 1:39:25Speaker 01

  Yeah.

• 1:39:26Lex Fridman?

  Uh, the self-play mechanisms of Alpha Zero are really surprising. And I reinforcement, the fact that reinforcement learning works at all to me is really surprising. The fact that neural networks work at all is quite surprising, given how nonlinear the space is. The fact that it's able to find local minima that are at all reasonable, it's very surprising. So it, uh, I, I wonder sometimes whether us humans just want it to not, the, for AGI not to be such a dumb thing.

• 1:40:05Lex Fridman?

  So I just, cause exactly what you're saying is like, um, the ideas of concepts and be able to reason with those concepts and, and connect those concepts in, uh, like hierarchical ways. And then to be able to have, uh, world models, I mean, just everything we're describing in human language in this poetic way, uh, seems to make sense that that is what intelligence and reasoning are like. I, I wonder if at the core of it, it could be much dumber.

• 1:40:32Speaker 01

  Uh, well, finally it is still connections and messages passing over them, right?

• 1:40:37Lex Fridman?

  Right.

• 1:40:38Speaker 01

  So in that way it's dumb.

• 1:40:41Lex Fridman?

  So I guess the recursion, the, the feedback mechanism, that, that does seem to be a fundamental kind of thing. Um, yeah, yeah. The idea of concepts, also memory.

• 1:40:53Speaker 01

  Correct. Yeah, having an episodic memory.

• 1:40:56Lex Fridman?

  Yeah.

• 1:40:56Speaker 01

  Yeah.

• 1:40:56Lex Fridman?

  That seems to be an important thing. So how do we get memory?

• 1:41:00Speaker 01

  So, yeah, we have another piece of work that which came on recently on how do you form episodic memories and and form abstractions from them. Uh, and we haven't figured out, you know, all the connections of that to the overall cognitive architecture. But, um.

• 1:41:16Lex Fridman?

  Well, yeah, what are your ideas about how you could have episodic memory?

• 1:41:20Speaker 01

  So at least it's very clear that there you need to have two kinds of memory, right? That that's very, very clear, right? Which is there are things that happen, uh, as statistical patterns in the world. Uh, but then there is the the one timeline of things that happen only once in your life, right? Uh, uh, and this day is not going to happen ever again. And and so and that needs to be stored as a, as a, you know, just a stream of, uh, string, right? This is, this is my experience. And then then the question is about how do you take that experience and connect it to the statistical part of it? How do you now say that, okay, I experienced this thing? Now I want to be careful about similar situations.

• 1:42:04Lex Fridman?

  Mm hmm.

• 1:42:05Speaker 01

  Uh, and so so you need to be able to index that similarity using your other giant statistic, you know, the the model of the world that you have learned. Although the situation came from the episode, you need to be able to index the other one. So, uh, the episodic memory being implemented as an indexing over the other, uh, model that you're building.

• 1:42:32Lex Fridman?

  So the memories remain and they, uh, they they're indexed into this, like the statistical thing that you formed.

• 1:42:42Speaker 01

  Yeah, statistical cost structural model that you've built over over time. So so it's basically the idea is that, uh, the hippocampus is, uh, just storing or sequencing, uh, a, in a set of pointers that happens over time.

• 1:42:59Lex Fridman?

  Mm hmm.

• 1:42:59Speaker 01

  And then whenever you want to reconstitute that memory and evaluate the different, uh, uh, aspects of it, whether it was good, bad, do I need to encounter the situation again? You need the cortex, uh, to re-instantiate, to replay that memory.

• 1:43:16Lex Fridman?

  So how do you find that memory? Like, which direction is the important direction?

• 1:43:20Speaker 01

  Both directions, again, it's bi-directional.

• 1:43:24Lex Fridman?

  I guess, how do you retrieve the memory?

• 1:43:27Speaker 01

  So this is again hypothesis, right? We're making this up. So when you, when you come to a new situation, right? Your, your cortex is doing inference over in the new situation. And then, of course, hippocampus is connected to different parts of the cortex. And, and you have this deja vu situation, right? Okay, I have seen this thing before. And, and then, in the hippocampus, you can have an index of, okay, this is when it happened, as a timeline. And, and, and then, then you can use the hippocampus to drive the similar timelines to say, now I am, I am, rather than being driven by my current input stimuli,

• 1:44:10Speaker 01

  I am going back in time and rewinding my experience from there.

• 1:44:14Lex Fridman?

  Replying it.

• 1:44:15Speaker 01

  But putting back into the cortex. And then, putting it back into the cortex, of course, affects what you're going to see next in your current situation.

• 1:44:22Lex Fridman?

  Got it. Yeah, so that's, that's the whole thing, having a world model and then, yeah, connecting to the perception. Yeah, it does seem to be that that's what's happening. It'd be, on the neural network side, it's, it's interesting to think of how we actually do that.

• 1:44:39Speaker 01

  Yeah, yeah.

• 1:44:40Lex Fridman?

  To have a knowledge base.

• 1:44:42Speaker 01

  Yes. It is possible that you can put many of these structures into neural networks and we will find ways of combining properties of neural networks and graphical models. So, I mean, it's already started happening.

• 1:44:58Lex Fridman?

  Yes.

• 1:44:58Speaker 01

  Graph neural networks are kind of emerged between them. And there will be more of that thing. So, but to me, it is, the direction is particularly, in looking at biology and the history, history of evolutionary history of intelligence, it is pretty clear that, okay, what it'll need is more structure in the models and modeling of the world and supporting dynamic inference.

• 1:45:26Lex Fridman?

  Well, let me ask you, there's a guy named Elon Musk, there's a company called Neuralink, and there's a general field called brain-computer interfaces.

• 1:45:34Speaker 01

  Yeah.

• 1:45:35Lex Fridman?

  It's kind of an interface between your two loves.

• 1:45:40Speaker 01

  Yes.

• 1:45:40Lex Fridman?

  The brain and the intelligence. So there's like very direct applications of brain-computer interfaces for people with different conditions, more in the short term.

• 1:45:50Speaker 01

  Yeah.

• 1:45:51Lex Fridman?

  But there's also these sci-fi futuristic kinds of ideas of AI systems being able to communicate in a high bandwidth way with the brain, bidirectional.

• 1:46:03Speaker 01

  Yeah.

• 1:46:03Lex Fridman?

  What are your thoughts about Neuralink and BCI in general as a possibility?

• 1:46:09Speaker 01

  So I think BCI is a cool research area, and in fact, when I got interested in brains initially, when, you know, so I was enrolled at Stanford, and when I got interested in brains, it was, it was through a brain-computer interface talk that Krishna Shenoy gave. That's when I even started thinking about the problem. So, so it is definitely a fascinating research area, and it is, the applications are enormous, right? So, you know, there is the science fiction scenario of, you know, brains directly communicating. Let's, you know, let's keep that aside for the time being. Even just the, the intermediate milestones they're pursuing, which are very reasonable

• 1:46:49Speaker 01

  as far as I can see, being able to control an external limb using, you know, direct connections from the brain and being able to write things into the brain. So, so those are all good steps to take, and they have enormous applications. You know, people losing limbs, being able to control prosthetics, quadriplegics, being able to control something, so, and therapeutics. And, you know, I also know about another company working in this space called Paradromics. They're doing, you know, it's based on a different electrode array, but trying to attack some of the same problems. So I think it's a very-

• 1:47:29Lex Fridman?

  Also surgery.

• 1:47:31Speaker 01

  Correct surgically implanted electrodes, yeah. Um, so, uh, yeah, I, I think of it as a, uh, very, very promising field, especially when it is helping people overcome, uh, some limitations now. At some point, of course, it will advance the level of being able to communicate, uh.

• 1:47:48Lex Fridman?

  How hard is that problem, do you think? Like, so, so, okay, let's say we magically solve what I think is a really hard problem of doing all of this safely.

• 1:47:59Speaker 01

  Yeah.

• 1:48:00Lex Fridman?

  So, so like, being able to, uh, connect electrodes and not just thousands, but like millions to the brain.

• 1:48:08Speaker 01

  I, I think it's very, very hard because you also do not know what the, what will happen to the brain with that, right? In the sense of how does the brain adapt to something like that?

• 1:48:17Lex Fridman?

  And it's, you know, as we're learning, it's, the brain is quite, uh, in terms of neuroplasticity, it's pretty malleable.

• 1:48:24Speaker 01

  Correct.

• 1:48:25Lex Fridman?

  So it's gonna adjust.

• 1:48:26Speaker 01

  Correct.

• 1:48:27Lex Fridman?

  So the machine learning side, the computer side is gonna adjust and then the brain's gonna adjust.

• 1:48:31Speaker 01

  Exactly. And then what, what soup does this land us into is.

• 1:48:35Lex Fridman?

  The kind of, the kind of hallucinations you might get from this. They might be pretty intense.

• 1:48:40Speaker 01

  Yeah, yeah.

• 1:48:41Lex Fridman?

  Uh, just connecting to all of Wikipedia. It's interesting whether we need to be able to figure out the basic protocol of the brain's communication schemes in order to get them to, the machine and the brain to talk. Cause another possibility is the brain actually just adjusts to whatever the heck the computer is doing.

• 1:49:01Speaker 01

  Exactly. That's the way I think that I find that to be a more promising way. It's basically saying, you know, okay, attach electrodes to some part of the cortex. Okay. And make sure, maybe if it is done from birth, uh, the brain will adapt. It says that, you know, that part is not damaged. It was not used for anything. These electrodes are attached there, right? And now you, you train that part of the brain to do this high bandwidth communication between something else, right? And, and, uh, if you do it like that. Then it is brain adapting to and of course your external system is designed such that it is adaptable, just like we, you know, design computers or mouse, keyboard, all of them to be interacting with humans. So of course, that feedback system is designed

• 1:49:46Speaker 01

  to be human compatible, but now it is not trying to record from the all of the brain and you know, now you know, two system trying to adapt to each other. It's a brain adapting to one way.

• 1:50:01Lex Fridman?

  Fascinating. The brain is connected to like the internet. It's connected.

• 1:50:06Speaker 01

  Yeah.

• 1:50:06Lex Fridman?

  Just imagine it's connecting you to Twitter and just, just, just taking that stream.

• 1:50:12Speaker 01

  Yep.

• 1:50:12Lex Fridman?

  Of information. Yeah. But again, if we take a step back, I don't know what your intuition is. I feel like that is not as hard of a problem as the doing it safely. There's, there's a huge barrier to surgery.

• 1:50:32Speaker 01

  Right.

• 1:50:32Lex Fridman?

  That because, because the biological system, it's, it's, it's a mush of like weird stuff.

• 1:50:38Speaker 01

  Correct. So that the surgery part of it, biology part of it, the, the long-term repercussions part of it, again, I don't know what else will, you know, we, we often find after a long time in biology that, okay, that idea was wrong, right? You know, so people used to cut off this, the gland called the thymus or something. And then they found that, oh no, that actually causes cancer.

• 1:51:05Lex Fridman?

  Yeah. And then there's a subtle, like millions of variables involved. But this whole process, the nice thing, just like, again, with Elon, just like colonizing Mars, seems like a ridiculously difficult idea. But in the process of doing it, we might learn a lot about the biology of the neurobiology of the brain, the neuroscience side of things. It's like, if you want to learn something, do the most difficult version of it.

• 1:51:33Speaker 01

  Yeah.

• 1:51:34Lex Fridman?

  And see what you learn.

• 1:51:35Speaker 01

  The intermediate steps that they are taking sounded all very reasonable to me.

• 1:51:39Lex Fridman?

  Yeah. It's great. Well, but like everything with Elon is the timeline seems insanely fast. So that's, that's the only awful question. Well, one, we've been talking about cognition a little bit. So like reasoning.

• 1:51:56Lex Fridman?

  We haven't mentioned the other C word, which is consciousness. Do you ever think about that one? Do you, is that useful at all in this whole context of what it takes to create an intelligent reasoning being? Or is that completely outside of your, like the engineer perspective of intelligence?

• 1:52:15Speaker 01

  It is not outside the realm, but it doesn't on a day to day way, you know, basis inform what we do. But it's more, so in, in many ways, the company name is connected to this idea of consciousness.

• 1:52:29Lex Fridman?

  What's, what's the company name?

• 1:52:31Speaker 01

  Vicarious, you know, so Vicarious is the company name. And, and so what does Vicarious mean, right? It's, at the first level, it is about modeling the world. And, and it is internalizing the external actions. So, so you interact with the world and learn a lot about the world. And now, after having learned a lot about the world, you, you can run those things in your mind without actually having to act in the world. So you can run things vicariously, just in your, in your, in your brain. And similarly, you can experience another person's thoughts by, you know, having a model of how that person works.

• 1:53:12Speaker 01

  And, and running their, you know, putting yourself in some other person's shoes. So that is being vicarious. Now, it's the same modeling apparatus that you're using to model the external world or some other person's thoughts. You can turn it to yourself. You can up, you know, if that same modeling thing is applied to your own modeling apparatus, then that is what gives rise to consciousness, I think.

• 1:53:38Lex Fridman?

  Well, that's more like self-awareness. There's the hard problem of consciousness, which is like when the model becomes, when, when the model feels like something, when this whole process is like, it act, it's like you really are in it. It, you feel like an entity in this world, not just you know that you're an entity, but it feels like something to be that entity. It, it, you know, and thereby we attribute this, you know, then it starts to be where in something that has consciousness can suffer. You start to have these kinds of things that we can reason about that.

• 1:54:19Speaker 01

  Yes.

• 1:54:20Lex Fridman?

  Much.

• 1:54:22Lex Fridman?

  Much heavier. It seems like there's much greater cost as your your decisions. And like mortality is tied up into that, like the fact that these things end.

• 1:54:34Speaker 01

  Right.

• 1:54:35Lex Fridman?

  That first of all, I end at some point and then other things end. And, you know, that that somehow seems to be, at least for us humans, a deep motivator.

• 1:54:47Speaker 01

  Yes.

• 1:54:48Lex Fridman?

  And that, you know, that that idea of motivation in general, we talk about goals in AI, but

• 1:54:55Lex Fridman?

  the goals aren't quite the same thing as like the our mortality. It feels like it feels like, first of all, humans don't have a goal. And they just kind of create goals at different levels. They like make up goals. Because we're terrified by the mystery of the thing that that gets us all. So we we we make these goals up. So we're like a goal generation machine. As opposed to a machine which optimizes the trajectory towards singular goal. So it feels like that's an important part of cognition, that whole mortality thing.

• 1:55:34Speaker 01

  Well, it is. It is a part of human cognition. But there is no reason for that mortality to come to the question for a

• 1:55:48Speaker 01

  artificial system, because we can copy the artificial system. The the the problem with humans is that we can't I can't clone you. I can't, I, you know, I can, I can clone, even if I clone you as a, you know, the the hardware, your experience that was stored in your brain, your episodic memory, all those will not be captured in the in the new clone. So but that is not the same with an AI system, right? So.

• 1:56:14Lex Fridman?

  But it's also possible that the the thing that you mentioned with with us humans is actually of fundamental importance for intelligence. So like the fact that you can copy an AI system.

• 1:56:27Speaker 01

  Yeah.

• 1:56:27Lex Fridman?

  Means that that AI system is not yet an AGI. So like, so if you look at existence proof.

• 1:56:35Speaker 01

  Yeah.

• 1:56:35Lex Fridman?

  If we reason.

• 1:56:37Speaker 01

  Yeah.

• 1:56:37Lex Fridman?

  Based on existence proof, you could say that it doesn't feel like death is a fundamental property of an intelligent system.

• 1:56:44Speaker 01

  Correct.

• 1:56:44Lex Fridman?

  But we don't yet, give me an example of an immortal intelligent being. We don't have those. It's very possible that, you know, that's, that is a fundamental property of intelligence is a thing that has a deadline for itself.

• 1:57:05Speaker 01

  Well, you can think of it like this. So suppose you invent a way to freeze people for a long time. It's not dying, right? So, so you can be frozen and woken up thousands of years from now. So it's no fear of death.

• 1:57:22Lex Fridman?

  Well, no, you're still, it's, it's, it's not, it's not about time. It's about the knowledge that it's temporary. And the, that aspect of it, the finiteness of it, I think, creates a kind of urgency.

• 1:57:39Speaker 01

  Correct. For us, for humans.

• 1:57:41Lex Fridman?

  Yeah, for humans.

• 1:57:41Speaker 01

  Yes, and that, that is part of our drives. But, and that's why I'm not too worried about AI, you know, having motivations to kill all humans and those kinds of things. Why? Just wait, you know? So, so why do you need to do that?

• 1:58:02Lex Fridman?

  Yeah, I've never heard that before. That's a good, that's a good point.

• 1:58:08Lex Fridman?

  Yeah, just murder seems like a lot of work. We'll just wait it out.

• 1:58:13Lex Fridman?

  They'll probably hurt themselves. Let me ask you, people often kind of wonder, world-class researchers such as yourself, what kind of books, technical fiction, philosophical were?

• 1:58:28Lex Fridman?

  Had an impact on you in your life and maybe ones you could possibly recommend that others read, maybe if you have three books that pop in the mind.

• 1:58:40Speaker 01

  Yeah, so I definitely liked Judea Pearl's book probabilistic reasoning and intelligent systems. It's it's a very deep, technical book, but what I liked is that in so there are many places where you can learn about probabilistic graphical models from. But throughout this book, Judea Pearl kind of sprinkles his philosophical observations and and he thinks about connects us to how the brain thinks and attentions and resources, all those things. So so that whole thing makes it more interesting to read.

• 1:59:12Lex Fridman?

  He emphasizes the importance of causality.

• 1:59:15Speaker 01

  So that was in his later book. So this was the first book probabilistic reasoning and intelligence systems. He mentions causality, but he hadn't really sunk his teeth into like, you know, how do you actually formalize?

• 1:59:26Lex Fridman?

  Got it.

• 1:59:27Speaker 01

  Yeah. And the second book causality is two thousand, the one in two thousand, that one is really hard, so I wouldn't recommend that.

• 1:59:35Lex Fridman?

  Oh, yes, so that looks at the like the mathematical, like his model of.

• 1:59:40Speaker 01

  Do calculus.

• 1:59:40Lex Fridman?

  Do calculus, yeah, it's pretty dense mathematics.

• 1:59:43Speaker 01

  Right, right, right. The Book of Why is definitely more enjoyable.

• 1:59:46Lex Fridman?

  Oh, for sure.

• 1:59:46Speaker 01

  Yeah. So, yeah, so I would I would recommend probabilistic reasoning in intelligent systems. Another book I liked was one from Doug Hofstadter. This is a long time ago, he has a book, he had a book, I think, called it was called A Mind's Eye. It was probably Hofstadter and Daniel Dennett together.

• 2:00:06Lex Fridman?

  Yeah.

• 2:00:07Speaker 01

  So.

• 2:00:07Lex Fridman?

  And it actually was, I bought that book, it's on my shelf, I haven't read it yet, but I I couldn't get an electronic version of it, which is annoying because I read everything on Kindle.

• 2:00:19Speaker 01

  Oh, okay.

• 2:00:20Lex Fridman?

  So you had to actually purchase the physical, it's like one of the only physical books I have because anyway, there's a lot of people recommend it highly, so.

• 2:00:28Speaker 01

  Yeah. And the third one I would definitely recommend reading is, this is not a technical book, it is history. It's called, it's the name of the book, I think, is Bishop's Boys. It's about Wright brothers and, and their, their, their path and how it was. There are multiple books on this topic and all of them are, are great. It's fascinating how flight was, you know, treated as an unsolvable problem. And, and, and also, you know, what aspects did people emphasize?

• 2:01:07Speaker 01

  You know, people thought, oh, it is all about just powerful engines, you know, just need to have powerful lightweight engines. And so, you know, some people thought of it as, how far can we just throw the thing, you know, just throw it.

• 2:01:22Lex Fridman?

  Yeah, catapult.

• 2:01:23Speaker 01

  Yeah, so, so it's, it's a very fascinating, and even after they made the invention, like, you know, people not believing it and.

• 2:01:32Lex Fridman?

  Ah, the social aspect of it, yeah.

• 2:01:34Speaker 01

  The social aspect, it's, it's, you know, very fascinating.

• 2:01:36Lex Fridman?

  I mean, do you, do you draw any parallels between, you know, birds fly? So there's the natural approach to, to flight and then there's the engineered approach. Do you, do you see the same kind of thing with the brain and are trying to engineer intelligence?

• 2:01:54Speaker 01

  Yeah, it's, it's a good analogy to have. Of course, all analogies have their, you know, so, so, so.

• 2:02:02Lex Fridman?

  For sure.

• 2:02:02Speaker 01

  There is, so people in, you know, AI often use airplanes as an example of, hey, we didn't learn anything from birds. Look, yeah, but the, the funny thing is that, and, and the, the saying is airplanes don't flap wings. Yeah. Right, this is what they say. The funny thing and ironic thing is that, that you don't need to flap to fly is something Wright brothers found by observing birds.

• 2:02:32Lex Fridman?

  Yeah.

• 2:02:32Speaker 01

  So they, they have in their notebook, like, you know, in some of these books, they show their, their notebook drawings, right? They, they make detailed notes about buzzards just soaring over thermals. And they basically say, look, flapping is not the important, propulsion is not the important problem to solve here. We want to solve control and once you solve control, propulsion will fall into place. All of, all of these are people, you know, they realize this by observing birds.

• 2:03:01Lex Fridman?

  Beautifully put. That's actually brilliant, because people do use that analogy a lot. I'm gonna have to remember that one. Do you have advice for people interested in artificial intelligence, like young folks today? I talk to undergraduate students all the time, interested in neuroscience, interested in understanding how the brain works. Is there advice you would give them about their career? Maybe about their life in general?

• 2:03:26Speaker 01

  Sure. I think every, you know, every piece of advice should be taken with a pinch of salt. Because, you know, each person is different, their motivations are different. But I can, I can definitely say, if your goal is to understand the brain from the angle of wanting to build one, you know, then being an experimental neuroscientist might not be the way to go about it.

• 2:03:53Speaker 01

  It might, a better way to pursue it might be through computer science, electrical engineering, machine learning and AI. And of course, you have to study the neuroscience, but that you can do on your own. If you are more attracted by finding something intriguing about, discovering something intriguing about the brain, then of course, it is better to be an experimentalist. So find that motivation. What are you intrigued by? And of course, find your strengths too. Some people are very good experimentalists and they enjoy doing that.

• 2:04:27Lex Fridman?

  And it's interesting to see which department, if you're, if you're picking in terms of like your education path, whether to

• 2:04:37Lex Fridman?

  to go with like, in MIT, it's brain and computer, no, BCS.

• 2:04:44Speaker 01

  Yeah.

• 2:04:44Lex Fridman?

  Brain and cognitive sciences, yeah. Or, or the CS side of things. And actually, the brain folks, the neuroscience folks are more and more now embracing of, you know, learning TensorFlow and PyTorch, right? They, they see the power of trying to engineer ideas that, that they get from the brain into, and then explore how those could be used to, to create intelligent systems. So that might be the right department, actually.

• 2:05:16Speaker 01

  Yeah.

• 2:05:18Speaker 01

  So this was a question in, you know, one of the Red Bull Neuroscience Institute workshops that Jeff Hawkins organized almost ten years ago. This question was put to a panel, right? What should be the undergrad major? You should take if you want to understand the brain. And the majority opinion in that one was electrical engineering.

• 2:05:39Lex Fridman?

  Interesting.

• 2:05:41Speaker 01

  Because, I mean, I'm a double undergrad, so I got lucky in that way. But I think it does have some of the right ingredients because you learn about circuits. You learn about how you can construct circuits to, you know, do functions. You learn about microprocessors. You learn information theory. You learn signal processing. You learn continuous math. So in that way, it's a good step to, if you want to go to computer science or neuroscience, it's a good step.

• 2:06:12Lex Fridman?

  The downside, you're more likely to be forced to use MATLAB.

• 2:06:22Lex Fridman?

  One of the interesting things about, I mean, this is changing. The world is changing, but like certain departments lagged on the programming side of things, on developing good habits into the software engineering. But I think that's more and more changing and students can take the answer of their own hands, like learn to program. I feel like everybody should learn to program because it's like everyone in the sciences because it empowers, it puts the data at your fingertips. So you can organize it, you can find all kinds of things in the data. And then you can also, for the appropriate sciences, build systems that like based on that. So like then engineer intelligent systems.

• 2:07:08Lex Fridman?

  We already talked about mortality, so we hit a ridiculous point, but let me ask you the, you know.

• 2:07:21Lex Fridman?

  One of the things about intelligence is it's goal driven. And you study the brain. So the question is like, what's the goal that the brain is operating under? What's the meaning of it all for us humans in your view? What's the meaning of life?

• 2:07:39Speaker 01

  The meaning of life is whatever you construct out of it. It's completely open.

• 2:07:43Lex Fridman?

  It's open? So there's not, there's nothing.

• 2:07:48Lex Fridman?

  Like you mentioned, you like constraints. So what's, it's wide open. Is there, is there some useful aspect that you think about in terms of like

• 2:08:01Lex Fridman?

  the openness of it and just the basic mechanisms of generating goals? In studying cognition in the brain that you think about? Or is it just about, because everything we've talked about, kind of the perception system is to understand the environment. There's like to be able to like not die.

• 2:08:19Speaker 01

  Got it. Exactly.

• 2:08:21Lex Fridman?

  Like not fall over and like be able to, you don't think we need to think about anything bigger than that?

• 2:08:30Speaker 01

  Yeah, I think so, because it's, it's basically being able to understand the machinery of the world. Such that you can pursue whatever goals you want, right?

• 2:08:40Lex Fridman?

  So the machinery of the world is really ultimately what we should be striving to understand. And the rest is just, the rest is just whatever the heck you want to do or whatever.

• 2:08:49Speaker 01

  Whatever fun you want. It's culturally popular.

• 2:08:54Lex Fridman?

  I think that's, that's beautifully put. I don't think there's a better way to end it. Delete, I'm so honored that you, you would show up here and waste your time with me. It's been an awesome conversation. Thanks so much for talking today.

• 2:09:09Speaker 01

  Oh, thank you so much. This was, this was so much more fun than I expected. Thank you.

• 2:09:16Lex Fridman?

  Thanks for listening to this conversation with Delete George, and thank you to our sponsors, Babbel, Raycon earbuds, and MasterClass. Please consider supporting this podcast by going to babbel.com and use code LEX, going to buyraycon.com/lex, and signing up at masterclass.com/lex. Click the links, get the discount. It really is the best way to support this podcast. If you enjoy this thing, subscribe on YouTube, review the Five Stars and Apple Podcasts, support on Patreon, or connect with me on Twitter at @LexFriedman, spelled S without the E, just F R I D M A N. And now, let me leave you with some words from Marcus Aurelius.

• 2:10:02Lex Fridman?

  "You have power over your mind, not outside events. Realize this, and you will find strength." Thank you for listening and hope to see you next time.