Getting AI To Think And Learn Like Humans — With Daniel Kahneman and Yann LeCun

Daniel Kahneman is a Nobel prize-winning psychologist and economist and author of Thinking, Fast and Slow, a landmark book that decodes human decision-making. Yann LeCun is the chief AI scientist at Meta (Facebook) and a pioneer in the field of deep learning, which the cutting edge of AI is based on today. The two come together on Big Technology Podcast this week to discuss how machines and humans learn, whether there are parallels, and what each field can learn from each other. Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay?

• 0:43Alex Kantrowitz?

  Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 1:17Alex Kantrowitz?

  Hello and welcome to the Big Technology Podcast, a show for cool-headed nuanced conversations of the tech world and beyond. But let me tell you, I'm pinching myself that we're actually about to have this conversation because I think it's going to be the most fascinating we've had on the podcast, bar none. And let me tell you why. So in 2017, I got a chance to spend some time with Jan LeCun. And Jan was the head of Facebook AI Research. Now you might know the company as Meta. He's the chief AI scientist there. And we spoke a lot about how you can actually take the human mind and make it artificial, turn it into something that's an AI. And I think that that's making a thinking machine is Jan's long-term goal.

• 2:02Alex Kantrowitz?

  Then this summer, I had some time off and decided that I was going to read a book I wanted to get around to for a long time. It was Thinking Fast and Slow by the Nobel Prize-winning Professor Daniel Kahneman. And as I'm reading this book, I start thinking, wow, Professor Kahneman describes the way the mind works in so many interesting ways that I hadn't thought about and ways that you'd imagine Jan would have to think about if he wants to build a thinking machine. And so why don't we just get the two of them together? Obviously, it was a pipe dream, right? Anyway, about a month ago, I'm in the same room as both of them by some stroke of luck. And I propose, why don't we have this conversation?

• 2:46Alex Kantrowitz?

  So of course, I go up to Professor Kahneman, we'll call him Danny for this conversation, and say, hey, have you ever spoken with Jan? And it turns out that they've spoken many times. And this is discussions they've had in private a handful of times, but they've never had it in public. And so we'll have it in public today. So first of all, I want to say Jan and Danny, welcome to the show. It's great to have you both. So I think we should just do the foundational stuff about what Jan's mission is, a little bit on Danny's research, and then actually stay tuned for the second half because Danny brought a list of questions that he wants to ask Jan, and I'll just take backseat for that. But why don't we start here? Jan, I'd love to hear a little bit more about, if you could briefly

• 3:29Alex Kantrowitz?

  share your ambition to build a thinking machine, and what does that actually look like? Do you want to replicate the human mind, one for one?

• 3:36Speaker 01

  No, I don't want to replicate the human mind. I want to understand intelligence. I think one of the most fascinating questions of our time, scientific questions of our time, is what is intelligence? How does the brain work? The other two fascinating questions of our times are what is life all about? And how does the universe work? So these are like big questions. And as a

• 4:04Speaker 01

  As a scientist and an engineer, I consider that I don't really understand how something works unless I build it myself. So, or have some understanding of how to build it. And so the purpose of AI is, uh, there's a dual purpose. One is understand intelligence, perhaps approach models of human intelligence if we want. And the other one, of course, is to construct interesting artifacts that can, you know, help people in the daily lives and make the world better and everything.

• 4:38Alex Kantrowitz?

  Yeah.

• 4:39Speaker 01

  So, um, I think it's really two purposes, the scientific one and the technological one.

• 4:45Alex Kantrowitz?

  Right, and just to follow up on that. I mean, I remember sitting with you in your office, talking about, you were talking about, you know, how, um, you know, you want to teach computers how to predict. And if they can predict, they can plan. And if they plan, they can start to have some functionality that, you know, we have as human beings. And there is this rush towards general intelligence, um, where folks like you, uh, and researchers in your, in your stratosphere are attempting to build a thinking machine. That's right.

• 5:17Speaker 01

  Well, yeah, so the, I mean, the first question is, or the first thing you can, you can notice in the animal world is that there is no intelligence without learning. And, uh, in the engineering world, it's almost true as well. And it's probably because I'm either lazy or not smart enough to, that I think that as human engineers, we cannot actually directly conceive and construct an intelligent machine. We have to build a machine that can make itself intelligent through learning, right? So that's how I got interested in learning, that was very early on, like when I was an undergrad or something. And, uh, and then the next question is, um, how, how to get machines to learn?

• 5:58Speaker 01

  And of course, you know, there is a long history of machine learning, you know, supervised learning, starting with the perceptron and and things like that. And what what has become really clear over the last couple of decades, it was clear, it was even clear for people like Jeff Hinton before that, but maybe just for me is is that the type of learning that we are currently able to reproduce in machine, which is supervised learning and reinforcement learning, do not seem to reflect what we observe in humans and animals. There is another type of learning, another paradigm of learning that seems to take place in humans and animals that allows humans and animals to learn how the world works, you know, mostly by observation, a little bit by interaction, but mostly by

• 6:43Speaker 01

  observation. And we accumulate enormous amounts of background knowledge about how the world works. And that connects with what with with things that, you know, Danny talks about. If if we if we have a model of the world, this model of the world can, we can use it to plan because we can imagine the result of the the consequences of actions we're taking. That allows us to plan. So this is what Danny calls system two. Okay. And but currently, what we can do in machine learning is more like the system one, the stuff that, you know, here is an input, here is an output, that does not require reasoning, if you want. So we're sort of, you know, I'm interested in sort of trying to get

• 7:25Speaker 01

  machine to learn models of the world so that we can get them to reason, essentially.

• 7:30Alex Kantrowitz?

  Yeah, it's so amazing to have you both on on the same show because now we get to go to Danny to talk a little bit about these concepts in terms of the way that the human mind thinks. And then we can go back to you, Jan, and think about how we might be able to get that into AI. So Danny, at the at the risk of having you go through, you know, the Free Bird speech of that you make often, but I think it's important for listeners, especially those who haven't read the book. Are you able to just describe a little bit more what Jan is talking about in terms of the way human beings think and you know, talk a little bit, I know it's sort of a shortcut, but talk a little bit about system one and system two and give us an overview there?

• 8:12Speaker 02

  I would say that when we describe human intelligence, we describe, we speak of a representation of the world. And it's the representation that leads to prediction. That is, there is no shortcut to the prediction from the data. You go through a representation, which includes how the system works. It includes the causal relations. This is what enables you to predict. And it turns out that we do have such a model of the world. And we, it's not so much that we have specific expectations about what's going to happen next.

• 8:57Speaker 02

  What is happening most of the time is that things happen, and then we make sense of them. That is, we actually go back and fit them into what happened before. And quite often, you know, right now, you're not predicting what I'm going to say next, but I'm not going to surprise you by what I say. Or, you know, if I said lumber, or of a sudden, I wouldn't surprise you because it doesn't fit. But so this is the way, the most interesting part of this works. And what is remarkable from the point of view of the psychologist,

• 9:36Speaker 02

  there are many, many things, but that touch on what Jan is doing. What's remarkable is how little it takes, how quickly people learn, and whether, that I think is a fundamental puzzle. And it turns out that the representation of the world that we have, it's hard to imagine it completely without symbols. That is, in, you know, we do think symbolically. And how you represent symbols without using symbols is sort of a puzzle.

• 10:21Speaker 02

  So, you know, I would, I would ask Jan the...

• 10:27Speaker 02

  Question I've asked him before, which is whether solving a generic learning problem is enough to get all of that. That is, to get the system that we learn quickly and to get a system that that will have the kind of logic that enables us that there is a certain kind of mistakes that people just don't do. And, you know, so, so we have it that an object is not in those two places at once, or basic facts about the world, that, and, and there is sort of a certainty about it

• 11:08Speaker 02

  that seems to be difficult to achieve with just a system that learns approximately. So I was wondering, you know, how Jan is going to deal with that.

• 11:20Alex Kantrowitz?

  Yeah, and, yeah, before we get into the answer, just a quick table setting. So the system one is the automatic, sort of the things that we do without, without, you know, thinking and, and the system two is, or we would feel like we're doing without thinking, even though we might be in system two is more complex.

• 11:37Speaker 02

  Yeah, so far I was talking, everything I was saying, you know, the system one, basically.

• 11:43Alex Kantrowitz?

  Yes.

• 11:43Speaker 02

  Because I think the representation of the world that we have and our ability to anticipate or to, or to, to feel, to feel unsurprised by, by what happens, which is, I think, more than anticipating, that is all system one. That's all, you know, that's all automatic. It's, it's effortless and it's very quick. System two is, we, we'll get to system two, but solving system one seems to be a big one.

• 12:11Alex Kantrowitz?

  Yeah, yeah, so, so, yeah, we'll start on system one. Jan, your thoughts?

• 12:15Speaker 01

  Well, I mean, I agree with, with Danny that current, current AI systems are

• 12:22Speaker 01

  You know, very specialized, and that makes them very brittle because they're trained for one task or maybe a collection of tasks. There's a sort of a motion towards training relatively large systems for multiple tasks at once, not just one, because they tend to work better and require less data. But it is astonishing how fast humans and animals can learn new tasks when faced with a sort of a new situation. How is it that, you know, a teenager can learn to drive a car in about, you know, ten or twenty hours of practice, or learn to fly an airplane in ten or twenty hours of practice? It's incredibly fast.

• 13:04Speaker 01

  If we were to use, let's say, reinforcement learning to train a self-driving car to drive itself, it would have to drive itself for millions of hours and cause, you know, untold thousands of accidents and destroy itself multiple times before it learns to drive probably not nearly as reliably as a human. So what's the difference? Now, of course, you know, obviously, we can say humans rely on their background knowledge about the world, and that basically is the answer to that. You know, we learn enormous amounts of background knowledge about how the world works, so we don't have to, when we learn to drive, we don't have to learn that if we drive next

• 13:45Speaker 01

  to a cliff and we turn the wheel to the right, the car will run off the cliff and nothing good will come out of it. We don't need to try because we know that from our model of how the world works, of intuitive physics and things like that. And this type of model is what gives us, in my opinion, some sort of common sense. I mean, what we call common sense comes out of this, that prevents us from making the really stupid mistakes that Danny was talking about, that AI systems are doing currently. So so how do we get machines to learn that? You know, you you look at how, at what stage in their life, human babies learn basic concepts? Like, what is the difference between an animate and inanimate object?

• 14:27Speaker 01

  Or I'm going to put this object on the table, is it going to stay stable or is it going to fall? That's around around the age of three months, so, three, four months. Difference between animate and inanimate also comes pretty early. Object permanence comes very early as well. Some some people claim it's innate, not clear. And then there are concepts that, you know, we take for granted, the fact that objects that are not supported fall, they're subject to gravity.

• 14:57Speaker 01

  Babies learn this around the age of eight months, eight to nine months. It takes a long time to understand that an object that is not supported will fall. It takes a long time to understand momentum and things like that.

• 15:11Speaker 01

  But then, you know, by by the age of nine months, and so in the first few months, you know, babies have very little ability to act on the world, right? They pretty much everything they learn is through observation only.

• 15:26Speaker 01

  And, you know, by by the time they are eight or nine months, they've they've they've pretty much understood, you know, the physics of the world. And and babies are relatively slow. You take a baby cat, they understand intuitive physics incredibly quickly. And and their own, the dynamics of their own body and everything. So, of course, you know, it's not clear how much of this is hardwired, but but there is clearly a lot of learning taking place. This type of learning is the one that we don't yet know how to reproduce with machines. And I think, I mean, that's where I focus my research, you know, what what is this type of learning? That allows us to learn representations of the world and predictive models of the world? That

• 16:08Speaker 01

  you know, then allows us to learn new tasks very quickly with very you know, very few trials or very few examples. And it's it's a very hot topic in machine learning at the moment, actually.

• 16:21Speaker 02

  Well, I'm curious about one thing. I mean, there are that I know about and I'm quite ignorant about it, but there are two tentative answers that I know about to solve the problem of the speed of learning. And one is that a lot is built in, so.

• 16:43Speaker 02

  And that looks very appealing, because actually, some animals really come out of the womb, and they're ready to go. I mean, goats, mountain goats, they know how not to fall as soon as they are born. So there it's difficult to think of learning, so nativism is part of the answer. Then there is something that I've been learning about, the hierarchical Bayesian models, which is sort of a different, that there are logical structures that we are prepared for to organize the world that we see, and that this is part of what enables us to learn things very quickly. Now, I think you reject both of these.

• 17:28Speaker 01

  Well, yes. The first one only partially, because, you know, it's obvious that, at least in certain species, including in humans, you know, a lot of things are not necessarily completely hardwired, but at least we have sort of intrinsic motivation to learn them quickly, and it drives us to learn them quickly, essentially. And it's certainly true for, you know, baby ibexes and mountain goat, whatever. And certainly for simpler animals, you know, like spiders and things like that. But, you know, even for an ibex or a mountain goat, they still need to kind of learn a lot about, you know, the dynamics of their own body. They are not hardwired with that. They are hardwired maybe with the geometry, but not with the details of it.

• 18:11Speaker 01

  And, you know, you can think of this as just being a few parameters to adjust, but I think it's much more, what goes on in the brain of the animal is much more complex than that. So I think there is a lot of learning in there, even if in the end we can observe that the learning takes place really quickly. So then, and, you know, there is some evidence.

• 18:39Speaker 01

  You know, for the things that a lot of people in cognitive science think or have thought in the past, perhaps, that were hardwired for those things to be learned, because they can be learned really quickly. So, for example, it's relatively simple, it would be relatively simple for genetics to encode the fact that the visual cortex needs to be built with neurons that can detect oriented edges, for example, you know, the kind of stuff that we find in the primary visual cortex area. But in fact, we have now, you know, a dozen different learning algorithms that if we were to run them in real time in sort of a simulated animal, if you want, would learn those oriented feature detectors from almost random images within minutes.

• 19:22Speaker 01

  And so there is no point in hardwiring this, because it can be learned within minutes. Same for face detection. So, you know, you look at the brain and there are areas in the brain that light up when people are shown faces. And so an easy conclusion of that is, oh, you know, this area of the brain is specialized in face detection and it's hardwired and, you know, face detection is innate or face recognition is innate. But again, face detection can be learned in minutes. If you're a baby, your virulence is bad, your focus is basically fixed at a relatively short range. So the only thing you see during the first weeks of your life are faces and nipples, essentially.

• 20:03Speaker 01

  So, you know, and then you have a hardwired thing to pay attention to motion. And so within minutes, you're going to have a face detector in your visual cortex. You know, any reasonable learning algorithm will learn this extremely quickly. So you don't need this to be innate if it can be learned. So that's the first thing about the innate, you know, is it the case that the fact that the world is three dimensional is innate, for example? It would make sense, right? Because a world is three dimensional, all of evolution has always taken place in the three dimensional world. So it would make sense to kind of hardwire this in the cortex. Then if you're an AI scientist or engineer, it's like, how would I even define

• 20:46Speaker 01

  an architecture that has that hardwired? I don't even know how to do it. And so it's not clear you can actually encode this in the genome. On the other hand, you can learn it really quickly. Because the fact that every point in the world has a depth is the best explanation for how your view of the world changes when you move your head. Or where or if you, you know, correlate the two views from your left eye and right eye. And so learning things like very basic things like this is very simple for whatever learning algorithm our brain uses. So that's the first part about the sort of nature and nurture debate, if you want. And then, and then there is the second question of symbols,

• 21:29Speaker 01

  about symbols. So is it necessary to have explicitly hardwired mechanisms in our brain that allows us to do things like symbolic manipulation or reasoning? And I, I find that hard to believe, that, that they need to be hardwired. First of all, the, the question, the first question to you, Danny, perhaps, which I've asked to, you know, many people who've come up with that question, is, do you view the type of reasoning that, let's say, great apes do as symbolic? Or do you view what monkeys

• 22:08Speaker 01

  do as, as symbolic? Or dogs, or cats, or octopus? Do they do symbolic reasoning?

• 22:19Alex Kantrowitz?

  And can we also just, for table setting, define the symbolic, what symbolic reasoning is?

• 22:25Speaker 01

  I don't know how to define it. Like, I'm, I'm not a big, you know, advocate of the very existence of symbolic reasoning. So.

• 22:32Alex Kantrowitz?

  Okay. Well, toss it over to Danny, yeah?

• 22:37Speaker 02

  I think that when we're talking about symbols, we're really talking about logical relations. That's

• 22:48Speaker 02

  that's not an easy one to explain, to understand each other, but it's hard for me to explain.

• 22:57Speaker 02

  But I think a characteristic of symbolic reasoning is a certain discreteness.

• 23:04Speaker 01

  Yeah.

• 23:06Speaker 02

  And it's not, so it's not completely compatible with what you have talked about. I mean, this is interesting, what is happening here, because I would push you with your idea that everything has to be learned or can be learned very easily. I would push you down the animal chain, you know, where I think it becomes highly plausible. And you are doing the same thing to me with symbols, by asking me, you know, how far down do symbols go? And it's clear that the answer, that, you know, the sort of standard answer,

• 23:46Speaker 02

  that it's connected with language and that the symbolic system and the language system are related, that therefore other animals don't have symbols in the way that we do. Whether that is really compelling, I'm not sure. That is, in the sense, for example, that in the hierarchical Bayesian models, that I'd like to get your evaluation of. But those are supposed to take care of perceptual learning. And with the symbolic element or the logical element is sort of upstream of that

• 24:27Speaker 02

  in what categories of things they are, you know, what categories of structure they are, that you're about to learn.

• 24:36Speaker 01

  Right, so I think the... There's a number of very interesting questions there. So the first one is, what do we mean by symbols, really? And if by symbol we mean the ability to form sort of discrete categories, represent sort of discrete categories in our, you know, mental representation system, then I think pretty much every brain has some sort of symbolic representation. And the reason I think this is that discretizing concepts, like having identified categories.

• 25:20Speaker 01

  Makes memory more important, right? Because it makes representation have the ability to be error corrected. Right? So if I, if I want to, if I write a zip code or a credit card number, and you make a mistake on one digit, that mistake would be detected. Because when you make a single error on a credit card number, there is some self-consistency, right? And that's because credit card numbers not only are discrete, you know, because they are numbers, but they are actually some distant apart, like two credit card numbers are some distance apart from each other. So if you make a small error, you can snap it back to the correct one. You can detect the mistake and you can, you know,

• 26:02Speaker 01

  that's the basics of error correction. So just for the purpose of being able to do error correction and for the purpose of being able to store concepts in memory, discrete concepts in memory efficiently, you need discrete representations. But now the big conundrum is, so in a way, you need symbols. And I think if that's the definition of symbols, then I think dogs have symbols. You know, I don't know if they can reason logically. I guess they have some logic, certainly. I mean, I think, you know, cats and dogs and probably mice even have this kind of, these kind of symbols.

• 26:37Alex Kantrowitz?

  And what's a discrete representation?

• 26:40Speaker 01

  You know, it's the difference between a real number and a natural number, right? So if I...

• 26:48Alex Kantrowitz?

  Ah, yeah, yeah, go ahead.

• 26:50Speaker 01

  Right, so let's say I send, I want to, I want to send a number to you. And the only thing I have is a, is a electric wire, and I can send a voltage, right? So I can send a voltage, you know, three volts, let's say. But then at your end, it may not be exactly three volts, because there is resistance, impedance, you know, parasites, noise, whatever. So what you're going to observe is some sort of fluctuating thing that may be around three volts, but maybe a little shifted. But it's not going to be shifted all the way to four volts or two volts. It's going to be around three volts. So because you know that the signals I want to transmit to you are either zero, one, two, or three, or four, you know it's going to be three, right? So despite the fact that there is noise in the transmission

• 27:35Speaker 01

  and despite the fact that the signal is continuous, you snap it back to its correct value. Okay.

• 27:41Alex Kantrowitz?

  Yeah.

• 27:41Speaker 01

  So, um, and then to store that value somewhere, because there is only, you know, five different possible values, zero, one, two, three, five, one, two, three, four, five, uh, you can store it with a small number of bits. Um, whereas if you had to store a precise voltage value, it would require many bits, right? So discretization makes memory more efficient. And it makes, uh, transmission either inside the brain or between, between, uh, agents, uh, uh, reliable. Um, it also, uh, you know, give the possibility of associative memory. So if you have, uh, uh, in the brain, you know, if you, you know, the brain represents things with voltages, right, to, uh, you know, to first order approximation.

• 28:25Speaker 01

  Now, you know, there may be noise again in this rap- in this representation and, uh, you know, you, you may see a, a, a partial view of, of an image and you can reconstruct the whole image because of your, you know, knowledge of, uh, uh, you know, what, what the, what the system is, is supposed to look like. You may not, never have seen my, my left side, okay, the left side of my face, but you, even if you'd never seen it, you, you probably would have a pretty good idea of what it looks like because of your general model of, of a, a human face and the fact that they are mostly symmetrical. Um, so that, that's an example of kind of snapping, uh, what's, what's a noisy signal

• 29:05Speaker 01

  to, uh, you know, a, a perfect signal because of the, maybe discrete, maybe not discrete, but at least the structure, the internal structure of, uh, of, of what you, of what, what you're looking at, which your, your brain has, has captured. So, uh, so discrete, you know, that's, that's what discrete, um, why discrete symbols are, are, are interesting. It's the same reason, by the way, why all of modern communication use, is digital. It used to be analog. It used to be that to communicate with each other, we would call each other on the phone and it would just transmit a voltage directly, you know, from, from iPhone to your phone. But now we replace this with digital communication with bits.

• 29:50Speaker 01

  And the reason is it's more efficient and you know more robust to noise and there's all kinds of advantages. Our brains actually use digital communication internally. A neuron, you know, neurons send spike to communicate with each other. And the reason is it's easier to regenerate a binary signal than it is to regenerate an analog signal. And it's more efficient energetically. I mean, there's all kinds of good reasons for this. So, you know, let me explain why symbols may emerge in our representations of the world, just for reasons of efficiency. And including for animals that do not have language.

• 30:30Speaker 01

  Now, of course, if you, if you, if you do want language, language is, you know, basically a sort of approximate way of representing the, the sort of complex data structures that we have in our, in our minds and to serialize it so that we can, you know, put it on the single, one dimensional signal, which is, you know, sound, sound pressure or, or sequences of words which are, which are symbols. And again, if, if, if I'm, you know, I'm talking to you through a microphone, it goes through the internet and, you know, there's some noise attached to that process. It's pretty high quality, but it's still some noise.

• 31:10Speaker 01

  Because, because our language uses discrete words, we can have error correcting communication. So even if you don't completely understand all of the syllables or phones that I'm pronouncing, you can still kind of recover what I meant. Because there is only a finite number of, of words, right? And, and you may not have heard all the syllables I pronounced, but you can snap it back to whatever makes sense in the context. In fact, you know, every speech recognition system works this way. They, they, they have sort of a language model and even if they don't understand every sound, they can sort of reconstruct really what, what was meant. In my case, it's even harder because of my accent.

• 31:54Speaker 01

  So, so, so language has to be discrete because it needs to be, because it needs to be noise resistant essentially.

• 32:06Speaker 01

  And perhaps, you know, language was, the appearance of language, of symbolic language was facilitated by the fact that, you know, we need to form the equivalent of symbols or discrete categories in our brain for efficient storage and sort of recovery, associative memory recovery possibly. And so it's quite possible that, you know, in the right condition, animals would have kind of more linguistic capabilities than we give them. And there's a number of experiments, of course, on this that have been done with monkeys and parrots and whatever.

• 32:50Speaker 02

  So in some sense, it's the the discretization itself is built in. That has to be, I mean, that. But what you're saying is there is no, no contents or very little content that is built in.

• 33:06Alex Kantrowitz?

  To the human brain at inception, is that, is that the, yeah, okay.

• 33:10Speaker 01

  Well, I mean, the big question is where do the meaning of those, of those discrete entities come from, right? Those, and, and I certainly do not believe that those, the, the, the meaning of those discrete entities are, are predetermined in the, the human mind, certainly, and the animal mind either. Those are completely learned. And so here is, here, here now comes the conundrum. So you're talking about hierarchical Bayesian systems. The, in a sense, multi-layer, you know, deep neural nets are hierarchical Bayesian systems. If you kind of view them the right way, and there are certain forms of them that are actually explicitly Bayesian.

• 33:50Speaker 01

  The main question in, in sort of the, the, the classical approaches to sort of Bayesian modeling is that the, the concepts in a hierarchical kind of Bayesian graph, you know, a Bayesian network or a graphical model, you know, people give them different names. Those concepts have, have to be designed by the, the human designer.

• 34:14Speaker 01

  When they build those those systems. And it clearly it clearly is not happening that way in the brain. Those concepts are learned, right? So the the basic entities that that are manipulated are are those those kind of, you know, discrete concepts are are learned.

• 34:34Alex Kantrowitz?

  Yeah, can you, sorry to ask, but can you just for the general audience, describe hierarchical Bayesian systems?

• 34:42Speaker 01

  Well, okay.

• 34:46Speaker 01

  I mean, it it means it means different things to different people. But we have a classical view of it called Bayesian networks, where you you have, basically, it's a graph. So graph in the mathematical term is a collection of nodes that are linked by by edges. And the the the the structure of this graph, so each node represents a variable. For example, so here's a classical example that, you know, people cite in courses on on AI. Is a node that represents whether your your your house is

• 35:21Speaker 01

  jumping, okay, it was moving, all right? Is another node that indicates whether a truck has hit your house. And then there is another node that indicates whether, you know, you're in California and whether there was an earthquake. Okay, so you could establish a causal a causal relationship, or at least a dependency between those nodes. You can say, well, if my if my house just jolted, it's either because a truck hit it or because there was an earthquake, and there's probably not many other reasons for that to happen. Now, if I look at the window and I see that

• 35:57Speaker 01

  a truck just hit the just hit the house, it immediately lowers my estimate of the likelihood that there was some continuous earthquake, right? Because the likelihood of those two happening at the same time is is very low. So that's called explaining away. So you could imagine sort of building a network of those nodes where each node has a particular meaning. You know, for example, some nodes are would be symptoms that you can observe in a person. And then underlying nodes that are causally, causally influenced would be sort of, you know, infection by a particular bug or whatever, right? And you could, you could imagine building those things and then doing inference.

• 36:41Speaker 01

  So you know the value of some nodes and you can infer the probability that all the other nodes that you don't observe have a particular value. I'm observing this and this and this and this symptom and I have this graph. And now I've measured the blood pressure and the, you know, whether the person's tummy hurts or in particular way or whatever. And I conclude that, you know, it's probably appendicitis or whatever, you know? So, so you could imagine, and in fact, you know, systems in the 1970s were built exactly on that model for, for, you know, so-called expert systems or probabilistic expert systems. And what we now call Bayesian networks and hierarchical Bayesian models and whatever are kind of the descendants of this, if you want. Now, this does not involve any learning.

• 37:26Speaker 01

  All those things would be built entirely by hand, okay, completely engineered. And that was essentially that, that necessity was one of the reason of the, the kind of decrease in interest in sort of good old fashioned AI.

• 37:40Alex Kantrowitz?

  Yeah, these were your competitors.

• 37:41Speaker 01

  In the 1980s.

• 37:42Alex Kantrowitz?

  Bayesian, yeah, the Bayesian.

• 37:44Speaker 01

  It could be completely logic, logical and

• 37:48Alex Kantrowitz?

  Right, right.

• 37:49Speaker 01

  Hard, hard decisions, but the fact that you had to hand engineer those entire systems from scratch is kind of a little bit way to kill them. So the, the alternative is running, but now you have the question, you know, how, how do you learn those concepts? How do you learn that, you know, the, the, you want to do computer vision, the, the nodes at the bottom level are pixel values, okay? What would be the right node just, just above that represent good combinations of pixel values? How do you learn that? Turns out those combinations would be things that, you know, detect oriented edges, which is what you observe in the brain and what, you know, convolutional neural nets actually learn. But that has to be learned.

• 38:30Speaker 01

  So, so then the, the, the big conundrum is if a system is to learn and manipulate discrete symbols.

• 38:40Speaker 01

  And learning essentially requires things to be kind of continuous. How do you kind of make those two seem compatible with each other?

• 38:50Speaker 02

  And the answer?

• 38:53Speaker 01

  I don't know. Maybe the answer is that we are giving perhaps a little too much importance to logical reasoning. So it's one thing that Jeff Hinton is saying, which is, which I agree with to a large extent, which is a lot of reasoning, certainly in animals and in humans, is not logical reasoning. It's basically simulation or analogical reasoning, which kind of similar. So, you know, you are a lion in the savannah and you're kind of chasing a wildebeest or something. And you have to do some prediction about the trajectory of the wildebeest and, you know, work with the other lion or lionesses usually to kind of chase the animal in the right way.

• 39:39Speaker 01

  And, you know, that requires a kind of, you know, a simulation of the animal you're chasing. The best way to predict how the animal you're chasing is going to act is to have an eternal model of that that you can simulate. And it's the same like, so in a more kind of human situation of, you know, you want to build a widget, like a box out of planks or whatever. You have to have sort of a model in your mind of what would be the result of assembling those planks and, you know, how solid it is going to be if you use glue or nails or screws or kind of more complex carpentry.

• 40:22Speaker 01

  And so the key element in this form of intelligence is your ability to build models of the world, predictive models of the world. And that's what we're missing, okay? That's what we need to figure out how to do with machines.

• 40:36Speaker 02

  And you can do this, and I find this very convincing actually. I mean, so you can...

• 40:45Speaker 02

  You can do this without symbols, you can do this without an explicit representation of causality. You just observe and from the observation you can go directly to prediction.

• 40:58Speaker 01

  Right, so there is going to be some causality involved. The question is whether you again need an explicit mechanism for causal inference, but if if your model of the world includes the prediction of the next state of the world, given the previous state and given your action, then you're going to build a causal model, right? Because you know that your action or the action you observe from other people or other agent will tell you, if I take that action, I will get this result from this state, right? So you will be able to establish causal models

• 41:33Speaker 01

  if you can act or if you can determine that another agent has acted and you have observed the result.

• 41:38Speaker 02

  Mechanism of that kind certainly exists, you know, in human physiology, so any movement of the eye involves a prediction of what the world looks, will look like when your eye moves, and there are those beautiful old experiments where

• 41:59Speaker 02

  people, in fact you use curare to paralyze their eyes, so they can't move their eyes. But when they intend to move their eyes, the world moves. And so they're anticipating, so and that is clearly, nobody would claim that there is any symbol or any causality required to do that, this is the kind of thing. So that would be your model for...

• 42:23Speaker 01

  In fact, your perception of the world is not the world as it is, it's the world as as it's going to be, because there is, you know, about 100 millisecond delay between what you see and how your brain interprets what it is. So in fact, your brain predicts 100 milliseconds in the future, your estimate of the world that you are consciously aware of is, was predicted from your perception from a tenth of a second ago. And that takes place everywhere, so you know, one of the current main theoretical concepts in systems neuroscience or computational neuroscience is this idea of predictive coding.

• 43:08Speaker 01

  Where basically, everything in the brain is trying to predict everything else in the brain, and, you know, predict future action, you know, future state of other parts of the brain and, and, and things like that. Now, this doesn't mean that people who are, you know, pushing for this kind of theory know how the brain works, because between a general concept of this type and sort of reduction to a practical algorithm, if you want, that you could sort of implement on a machine, there's a huge distance that has not been, has not been bridged yet. So that, that's kind of a big program, I think, for science for the next years.

• 43:45Speaker 02

  I was curious earlier, and it's a question I neglected to ask, but that was about face detection and the existence of, you know, a face system. It seems to me that seeing your mother's face a lot doesn't really equip you to distinguish between faces.

• 44:04Speaker 01

  That's right.

• 44:05Speaker 02

  And that's what the face system does. I mean, it's not, it, it's not only recognize this is a face, it, it is actually specialized in identifying faces, that is. And, and that, that ability to learn very quickly, a distinctive face, that seems to be, I mean, you know, it's the same problem. Again, I don't, I don't quite see how that gets done.

• 44:35Speaker 01

  I don't know. You know, historically also in computer vision, face detection preceded reliable face recognition by a decade or two. So we had reliable face detectors in the early 2000s, and reliable face recognition didn't pop up until 15 years later, roughly, with, with deep learning methods, actually. And it works surprisingly well. So you don't need a big neural net to be able to recognize, you know.

• 45:03Speaker 02

  Identify faces.

• 45:05Speaker 01

  Identify faces and, and have a system that recognizes more faces than any human can, with, you know, the same level of accuracy. You know, maybe not with the same sort of robustness to, you know, different changes imposed and facial hair and things like that, but, but in terms of number of different people being able to put a name on, they're incredibly, incredibly good, incredibly reliable. So it may not be that hard of a task as we thought, perhaps.

• 45:36Alex Kantrowitz?

  I kind of thought that this was going to be a discussion more of how we could transpose the human, learn from the human brain and turn that into AI. But it seems like we're actually talking a little bit about how we can learn about the human brain from the construction of AI.

• 45:54Speaker 02

  I think that this is, I think that this is actually happening. But there is an interesting question to me as a psychologist. You know, it's a version of the Turing test. So what would it take to, for the computer to fool you that it's, that it's human? And, and I was thinking that

• 46:21Speaker 02

  one characteristic of it is mistakes that strike us as absurd. That is, and they seem to violate some basic constraints. I mentioned earlier, an object now being in two places. And there is, I think, a finite set of absurd mistakes, or am I wrong? And would it be, would it be part of, I mean, would it be of any interest to have a system that makes the same mistakes, that, that avoids mistakes that people consider absurd? And that makes mistakes of the kind that people tolerate? I mean, is there such a category of absurd mistakes in, in your field?

• 47:07Speaker 01

  Well, there's different kinds of mistakes, right? I mean, there are computer vision systems that make stupid mistakes because, so in the past, when computer vision systems, you know, were like maybe 15 years ago, computer vision systems were okay at picking out, recognizing objects in images, but they were sometimes confused by the context. And, and people, they were not confused by the context, they just were not using the context at all. And so, you know, a system would make a mistake of recognizing a face that wasn't really a face. And from the context, you could tell it wasn't a face. Or would, you know, recognize an object that had, you know, the shape of, I don't know, a cow or something.

• 47:49Speaker 01

  You can tell from the context that, you know, it cannot possibly be a cow if you are, if you are human. So that was a big criticism towards computer vision system. They don't take context into account. And now the criticism you see is that they take, they take too much context into account. So there is this famous example of, you know, a modern vision system. I mean, it's a system of a few years ago. You train it to recognize, you know, things like cows and trains and airplanes and cars and stuff like that by showing you dozens of examples of each category. And then you show it a cow on the beach. And it, it, it can't recognize it as a cow because every example of cows it's seen were, were on, you know, green pasture.

• 48:32Speaker 01

  And so the system actually learns to use the context and it uses it too much. And the context was, you know, had a spurious correlation with the category. And the system doesn't have the common sense to say, well, that's a cow regardless of, you know, because a cow could be very well sitting on the beach.

• 48:51Speaker 01

  And, and so then how do, how do you fix that, you know?

• 48:55Speaker 02

  And how do you get that common sense? I mean, that seems.

• 48:58Speaker 01

  Get a common sense, exactly. So I think this is not going to be solved, in my opinion, by, you know, more tweaks on the architectures and more training data and things like this. I mean, it may be mitigated, but I think it's not going to be fixed by that. I think it's going to be fixed by systems that basically learn models of the world and then, you know, have the, the ability to tell that it's perfectly possible that, you know,

• 49:24Speaker 01

  a cow could be on the beach. That's the, I mean, that's the kind of compositional nature of, of the world that, you know, even combination of things you've never seen can very well happen. And and that is already happening. Not really, no. So there's a thing that, very interesting thing that is happening, which is, um, uh, uh, one of the hottest topics, as I was mentioning earlier, uh, in, uh, in AI today is, or in machine learning at least, is this idea of self-supervised learning, which I've been a big advocate of for, for many years. And it's the idea that you, you, you train a system, uh, not to solve a particular task, but to, um, you know, basically learn to represent the world in sort of a, you know, semi-genereic way.

• 50:08Speaker 01

  You train it by basically, uh, you, you train it to predict, essentially. Okay, so this works wonderful, wonderfully for natural language understanding system. The standard way nowadays, that has become standard over the last three years, of training a natural language understanding system is, uh, you take, uh, you take a, a sequence of words, a sentence or something longer than that, maybe a few hundred words or a thousand words. And you, you mask about ten or fifteen percent of those words. You, you just, you, you replace them by a blank marker. And you train some giant neural net to predict the words that are missing.

• 50:48Speaker 01

  Okay, and in the process of doing so, the, the network learns the, the, the nature of language, if you want. Um, so, you know, it, it learns that if you say, uh, the lion chases the blank in the savanna, the blank, you know, is likely to be something like a wildebeest or, or an antelope or whatever, right? Um, but if you say the cat chases the blank in the kitchen, that's probably a mouse. Uh, so a lot of semantics about the world, certainly the syntax, but also the semantics about the world, uh, ends up being represented by this network that just, is just trying to predict missing words, right? Now, of course, the system can never predict exactly which word is, is, is missing.

• 51:31Speaker 01

  Um, you know, it could be antelope or wildebeest or zebra or whatever. Uh, but it can easily represent, uh, a distribution over words, like a probability, a probability distribution of the words. Uh, because there is only a finite number of words in the, you know, in the dictionary, uh, in English, so you, you just have a number, which is a score for how likely it is for this particular word to appear. And that's how the systems handle the uncertainty in the prediction.

• 51:58Speaker 02

  Yeah, I mean, and and and the results are sort of amazing. You get texts that sounds like text in a particular style and it's coherent and it's grammatical.

• 52:10Speaker 01

  Right. But it makes stupid mistakes. It makes very stupid mistakes.

• 52:14Speaker 02

  It makes absurd mistakes.

• 52:15Speaker 01

  Absurd mistakes, yeah.

• 52:16Speaker 02

  Yeah. And and I was wondering, uh, when does it say that it makes absurd mistakes? You know, and it's a question that, you know, we talked about before, whether whether it whether it knows what it's talking about. And and the sense is that if you just merely predict words, uh, there is no content there, that that purely predictive system.

• 52:41Speaker 01

  Well, I don't know. It there is a little bit of content, but it's very superficial, right? So the understanding of the world about those systems is superficial. There is some understanding, uh, but it's very superficial. So, yeah, if you ask questions like, um, uh, you know, is the fifth leg of a of a dog longer than the other four? Um, you know, the thing will say yes or no, right? I mean, it won't tell you that dogs have, you know, have have four legs, right? And and, you know, and things of that type. They can't count, you know? I mean, there's a lot of things that they can do and it's because they don't have any common sense and it's because all of this learning is not grounded in an underlying reality. It's basically just from text.

• 53:25Speaker 01

  And and the amount of knowledge about the world that is encoded in all the texts in the world that those systems have been trained on, and it's billions of words, uh, is is is not present. Like, most of human knowledge is not represented in any text, uh, in existence. Uh, for example, you know, I, um, I I I take a I take an object, I take my phone and I put it on the table next to me and I push the table.

• 53:53Speaker 01

  You know, because of your physical intuition, that the object will move with the table. When I, when I push the table, the object that is on top of it will move with it, right? There, there is, there is nothing in any text, in any, any part of the world that explains this. And so a machine that is purely trained to predict missing words will not learn about this, uh, which are sort of basic, really basic facts about the world. So I'm, I'm one of those people who believes that, uh, truly intelligent systems, uh, will need to acquire knowledge, uh, will need to be grounded in some reality. It could be a simulated reality, it could be a, a virtual world. Um, but it has to be an environment that has its own logic and its own constraints and its own physics.

• 54:37Speaker 02

  So you think that in principle, if you took those transformer and, and you put them together with, if you put them to work on videos.

• 54:49Speaker 01

  Right.

• 54:50Speaker 02

  If you, if you put language and videos together, is that going to be a qualitative advance? And, and is that beginning to happen?

• 54:58Speaker 01

  Well, uh, yeah, so yes and no. Uh, so I think, you know, you should start with, uh, just, just video. Like, can you, can you have, can you build a neural net that will, or some, you know, learning system? That will watch video all day and basically learn basic concepts about, about the world just by watching video. The world is two dimensional, there are objects in front of others. Uh, there are objects that are animated, animate. There are objects, you know, uh, whose trajectory is completely predictable in animate objects. There are objects whose trajectory is not completely predictable, like, you know, the leaves on the tree. But, you know, qualitatively you can, so there's some level of representation where you can predict what, uh, what those things do. Uh, and then objects that are very difficult to predict, there are animate objects, uh, humans and things like that, right?

• 55:42Speaker 01

  Uh, or chaotic systems and whatever. Um, and so, uh, you know.

• 55:49Speaker 01

  A lot of research goes into this, essentially. None of them work at the moment. There's a lot of systems that attempt to do video prediction and basically attempt to learn representations of the world that are in an abstract way can can learn to predict what's going to happen in the video in the long term. They all work within a few frames of a video, like a fraction of a second. But then the predictions go, you know, go really bad and the representations that are learned by those systems are not very good. We can we can test them by using the representation as input to, let's say, an object classification system, for example, and measure, you know, how many samples does it take for the system to learn the concept of elephant, right? Does it does it still need 3,000 samples, a training example?

• 56:32Speaker 01

  Or would it would it work OK with three or four? And those things, the answer is the training from video, they don't work very well. There is one thing that is starting to work, and that gives us a lot of hope. So a particular form of self-supervised running, where you you you can simulate the video if you want. So you take you take an image and you you transform this image in some way. You you change the colors a little bit. You you change the scale, the orientation. You you translate it a little bit. You do you do some, you know, manipulation to it. It's called data augmentation. And then you train a neural net or two neural nets, rather, to basically map those two images to the same representation, the same vector.

• 57:16Speaker 01

  Typically, it's a vector, right? A list of numbers, typically a couple thousand. And so that's easy enough. You have two images that basically represent the same content, two views of the same object, for example. And you train the system to tell to tell you, well, the learn a representation that tells you the content of the image and not the not the details. The difficulty is how do you make sure that when you show two different objects, it will produce different representations? And the avoiding this this problem, this problem is called the collapse problem. And the big question is how you avoid this. So there's a lot of work now, really interesting work on what's called contrastive methods and also non-contrastive methods to to do this.

• 57:59Speaker 01

  And I'm I'm really excited about this area of research because I think it's the germ. I think it's our best shot as to You know, a path towards learning, you know, getting machines to learn abstract representations, you know, and predictive models, where those predictive models will not take place in the space of pixels, but will take place in the space of, or whatever it is that your, your input is, but in the space of abstract representations.

• 58:28Speaker 02

  But you think that the architectures that exist now will eventually, that it's not going to take a different architecture?

• 58:36Speaker 01

  Well, I think, I think the architecture is not the crucial question here. I think the architectural components are already here. What I think is not here is the, the whole learning paradigm. So let me, let me take an example, right? We can, we can build a giant neural net where you, you feed it a few frames of a video and you train it to predict the next frame or the next few frames. You can do this with LeSQUARE, right? So just measure the, the, some of the square, the differences between the values of the predicted pixels and the pixels that actually occur in the video. And when you do this, the predictions you get from your neural net are blurry. You get very blurry images as the prediction.

• 59:20Speaker 01

  And the more you, you let the system predict far in the future, the more blurry the predictions are. And the reason for this is that you ask the system to make one prediction and the system cannot predict a priori, if it's a video of us talking, it cannot predict if I'm going to move my hands this way or I'm going to move my head to the left or to the right. And so the only thing you can do is predict some sort of average of all the possible things that can happen. And that would be a very blurry version of us.

• 59:45Speaker 01

  And so that's not a good, a good model of the world. Now, how do you represent the uncertainty in the prediction? So in the case of text, it's easy because it's discrete. You can just have a distribution over which word are possible. But, but we don't have a good way of representing distributions over images, for example. That's why those techniques don't work currently for video prediction. And, and whatever representations they learn actually are not, not very good.

• 1:00:13Speaker 02

  That's.

• 1:00:13Speaker 01

  But, yeah.

• 1:00:15Speaker 02

  And that's because of the sheer complexity. Is it a quantitative problem? Or is it a point where quantity becomes quality?

• 1:00:25Speaker 01

  No, no, it's not a question of, you know, we don't have enough data or computers are not powerful enough or neural nets are not big enough. It's a question of principle. It's a question of like, what is the right objective function? What is the right way of representing uncertainty? And then technical questions like, how do you prevent this collapse? I was talking about, which I didn't explain very well, but which is a more technical issue. But it has to do with basically representing uncertainty in the prediction. The fact that you have an initial segment of a video, there are many, many ways that are plausible to continue it. And the machine has to basically represent all of those or a good chunk of those possibilities.

• 1:01:07Speaker 02

  And it would have to be predictive, that it couldn't work backwards?

• 1:01:12Speaker 01

  It could work both ways. It doesn't even have to be.

• 1:01:16Speaker 02

  The way that I think the mind works.

• 1:01:18Speaker 01

  Oh, yeah, sure.

• 1:01:19Speaker 02

  Really, you get prediction very short term, but by and large, a lot of it is simply making sense after the fact. You don't need to predict everything.

• 1:01:31Speaker 01

  Yeah, I mean, if you're a police inspector, you get to a scene and you have to basically figure out how it got there, right?

• 1:01:37Speaker 02

  Yes.

• 1:01:37Speaker 01

  Whether it's a sequence of events that, you know, led to this. So, yeah, I mean, I think, you know, I use the example of prediction of a future event, but it could very well be, you know, prediction of things you do not currently perceive, right? So you do not currently perceive the back of my head,

• 1:01:58Speaker 01

  but you have a pretty good idea what it looks like. And if I show it to you, then you can correct your internal model. You're not very surprised, but you know, maybe if I had a ponytail, you'd be surprised, slightly more surprised. So, you know, it's not just prediction in time. It can be prediction in space, you know, predicting a piece of an image from another piece, predicting a piece of a preset that is currently occluded or not, you know, cannot be obtained. And retrodiction, you know, predicting the past. Which you have not observed from the present and maybe in evolution. So yeah, I'm not insisting that it has to be a forward prediction.

• 1:02:43Speaker 01

  Forward prediction is useful for planning. So if we're talking about a form of reasoning that would be, you know, planning a sequence of actions to arrive at a particular result, then what you need is a forward prediction that will predict the state of the world as, you know, when we take an action.

• 1:03:04Speaker 02

  Well, I mean, you know, the amount of progress that is occurring in your field is just amazing.

• 1:03:11Speaker 01

  I have a high time following you.

• 1:03:13Speaker 02

  I'm very envious.

• 1:03:16Speaker 02

  I'm very envious because now a few years ago, I think I heard you say about a research program that we have the cherry, but we don't have the cake. But that's, you recognize that as.

• 1:03:33Speaker 01

  Oh, absolutely. Yeah, this has become a bit of a running joke in the, in the, in the community now. I, I use the analogy of the cake to say that, to, to basically sort of make very concrete the, the fact that most of what we learn as, as humans and animals and in, in the future that machine will learn is learned in this kind of self-supervised manner, basically by watching the world go by and by, you know, taking an action once in a while. But in a, in a, you know, non-task specific way, learning how the world works, self-supervised learning, that's the bulk of the cake. So if intelligence is a cake, the bulk of the cake is this type of learning. That's why we learn everything.

• 1:04:12Speaker 01

  And then there is a thin layer of, of, you know, well, there, there, well, there is supervised learning, right? You're taught at school or your parents, you know, teach you something or you, you read, you read a book or whatever, or you, you know, you show a, a young child a picture book and you say, that's an elephant and maybe say elephant. And with three examples, maybe he's figured out what an elephant is toddler, what an elephant is. So that's supervised learning. And then there is reinforcement learning where you learn a new skill by trial and error. And, you know, you, you, you get rewarded by your own success or, or, or not. And, and that was the cherry on the cake. The, the supervised learning was the, the icing, if you want.

• 1:04:54Speaker 01

  And this was sort of a metaphor to, to tell people like, you know, we. We're currently focusing on the icing and the cherry, but we haven't figured out how to bake the cake yet. And so I use this joke saying that, you know, this was the the dark matter of intelligence, because, uh, you know, it's kind of like physicists, right? They tell you dark matter exists and and it's, you know, most of the mass in the universe is dark matter and they have no idea what it is. It's very embarrassing. So we're in the same situation. Again, physicists, it's even worse because there's dark matter and dark energy and the combination of the two represents something like ninety-five percent of the mass in the universe. So that's really, really embarrassing and we have no idea what it is.

• 1:05:39Speaker 01

  Um, so we're in the same situation, which means, you know, we have a lot of work to do.

• 1:05:43Speaker 02

  Yeah.

• 1:05:43Speaker 01

  For understanding this.

• 1:05:44Speaker 02

  I mean, is the progress in your field exponential?

• 1:05:49Alex Kantrowitz?

  Yeah, but before you answer, I'm just watching the clock. You guys can go as long as you want, but I'm just saying that whenever you're ready.

• 1:05:55Speaker 02

  Oh, yeah. Oh.

• 1:05:56Alex Kantrowitz?

  Yeah.

• 1:05:56Speaker 01

  Yeah.

• 1:05:56Speaker 02

  I'm passing the hot stop. Just to answer this question.

• 1:06:00Alex Kantrowitz?

  Okay, sounds good.

• 1:06:01Speaker 01

  I think, um, I think it's increasing because the, there is more and more people joining the party and, and more and more, you know, governments and, and, and companies investing money in it. So you see, uh, you see a growth, you see a growth in applications. You don't necessarily a growth, uh, an exponential growth in sort of new concepts, but there are new concepts that are coming up at a surprisingly, uh, uh, high speed. And so I guess one big question is, uh, so, you know, one question is, are we going to have another AI winter like, like they were in the past? And my answer to this is probably no, or at least not to the same extent that we had in the past, because it's a big industry now behind,

• 1:06:42Speaker 01

  uh, behind AI. It's useful for a lot of things, you know, you take, you take deep learning out of, uh, you know, Google and Meta and a few other companies and they, they crumble. I mean, they're completely built around it now. So, um, uh, and, and certainly there's a lot of other areas also. So, so I think it's not, it's not going to collapse as, as it used to. The question is, are we going to make this next step towards, you know, machine common sense, uh, self-supervised learning, et cetera, before people funding all of this get tired? And that I don't know the answer. I'm just hoping that, you know, we make progress fast, you know?

• 1:07:18Speaker 02

  Fascinating question.

• 1:07:20Speaker 01

  Thank you, Jan. Well, thank you. Thank you, Danny. It's always a pleasure chatting with you.

• 1:07:27Alex Kantrowitz?

  And thanks to both of you. It's amazing to hear you talk about this. I feel like I was just in like an ultra graduate level course. We should have more psychology and AI together. Because this is, I think, like, it's just amazing to hear you guys bounce these concepts around. And in terms of energy, Jan, clearly, there's energy in the field. It's amazing hearing you. You know, no drop from the five years we've known each other, maybe even more. So, thank you, Danny. Thank you, Jan. So great to having you here.

• 1:07:58Alex Kantrowitz?

  Yeah, let's do it again.

• 1:07:59Speaker 01

  It's a great pleasure.

• 1:08:00Alex Kantrowitz?

  Yes, I'd love to do it again.

• 1:08:02Speaker 01

  Thank you.

• 1:08:02Alex Kantrowitz?

  Okay, all right. Thank you. Take care, Danny. Thank you, Nate Guatney, for doing the editing. Red Circle for hosting, and all of you, the listeners. We'll be back next Wednesday for another episode of the Big Technology Podcast. Until then, we'll see you.

• 1:08:28Speaker 03

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com. Not authorized by any candidate or candidate's committee.