Princeton Scientist: We Don't Understand AI - Tom Griffiths - #553

A Princeton cognitive scientist says AI can't think like a child — and giving it more data won't fix that. If the field keeps scaling without solving what's actually missing, the gap between human and machine intelligence won't close. It'll just get more expensive. Tom Griffiths is a professor of psychology and computer science at Princeton, and one of the leading researchers working at the intersection of human cognition and AI. We cover: -why a child learns language from breadcrumbs while AI needs continents of data -the 250-year-old idea that quietly became the foundation of modern language models -what sycophantic AI actually does to your beliefs over time -why solving AGI might have less to do with scale and more to do with understanding what a child's mind really is. The hallucinations don't bother him — it's the sycophancy that should worry you. Key Takeaways: 00:00 The Math Behind How Minds Actually Work 00:30 Why Defining "Thought" Is Harder Than It Looks 04:30 What AI Gets Wrong About Consciousness 07:00 What ChatGPT Actually Revealed About the Field 08:10 Are Humans Really Irrational — Or Solving a Different Problem? 11:00 How Chomsky Turned Language Into a Math Problem 13:55 The Chessboard Analogy That Explains Generative Grammar 15:20 Why Aristotle Got Thought Right and Physics Wrong 19:45 The Man Who Tried to Build AI in the 1600s 22:40 What Everyone Gets Wrong About George Boole 25:25 From Boole to Turing: How Logic Became Computers 27:40 Why Your Brain Runs on Less Energy Than a Light Bulb 28:40 Jensen Huang Says AGI Is Here. Is He Right? 31:45 Why the "AI vs. Human Intelligence" Scale Is Misleading 33:50 Why a Child Still Outlearns Every AI Model 35:20 The Fuzzy Boundary Problem That Broke Rule-Based AI 37:20 How Semantic Networks Rewired the Theory of Memory 39:30 Rosenblatt Built a Brain — Then Minsky Killed It 43:15 The Plane Ride Where Backpropagation Was Solved 44:20 Hallucinations, Sycophancy, and What Should Actually Worry You 47:00 What Has to Change Before AI Can Truly Generalize 50:10 What a Layperson Should Actually Take Away From This ——— 📬 Get the transcript, fascinating bonus content, and my Monday M. A. G. I. C. Message: https://briankeating.com/yt 🌠 Have a .edu email and live in the USA 🇺🇸? You automatically win a meteorite: https://BrianKeating.com/edu 🔔 Subscribe: https://www.youtube.com/DrBrianKeating?sub_confirmation=1 🎯 Support Into the Impossible on Patreon — get my weekly M. A. G. I. C. Message, unfiltered bonus content, and live monthly Office Hours with me: https://www.patreon.com/drbriankeating ⭐ Join this channel for perks, monthly Office Hours, and your name in the Member Roster at the end of every episode: https://www.youtube.com/channel/UCmXH_moPhfkqCk6S3b9RWuw/join 📚 My books: Losing the Nobel Prize (memoir): http://amzn.to/2sa5UpA Think Like a Nobel Prize Winner: https://a.co/d/03ezQFu Focus Like a Nobel Prize Winner: https://a.co/d/hi50U9U Galileo's Dialogue (first-ever audiobook): https://a.co/d/iZPi9Un 🌐 More: 🏄‍♂️ Twitter: https://twitter.com/DrBrianKeating ✍️ Blog: https://briankeating.com/blog 🎙️ Audio-only: https://briankeating.com/podcast #intotheimpossible #briankeating #science #physics #astronomy #cosmology #podcast #universe Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 0:31Speaker 01

  Low energy, unfocused, foggy? You might be dehydrated. Whether it's hot yoga, over 100 degree weather, or too much sun. Gator Light, with a specialized blend of five electrolytes, was scientifically designed to help move fluids into the body faster for rapid rehydration. Shop now at retailers nationwide. Gator Light hydrates faster than water. Is it in you? Symptoms noted are signs of mild to moderate dehydration. Consult a healthcare professional if symptoms persist.

• 0:59Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com. Not authorized by any candidate or candidate's committee.

• 1:29Speaker 03

  One of the, I think, interesting challenges we have at the moment is having built systems that we don't fully understand.

• 1:35Brian Keating?

  The man who built modern AI? He's the direct descendant of the man who invented the math that made it possible, which is insane, but it's not the wildest thing my guest told me today.

• 1:45Speaker 03

  That's pretty much exactly what he was trying to do. And he was, he was the right kind of crazy.

• 1:50Brian Keating?

  I mean, this was trying to invent AI 250 years before computers even existed.

• 1:56Speaker 03

  Sycophancy is a major problem. If you take a rational agent and have them interact with a system which is sycophantic, then that agent is going to become increasingly confident in their beliefs, but no closer to the truth.

• 2:08Brian Keating?

  My guest spent 20 years Building the mathematics of how minds work. And he just told me three things that made me question what I thought AI actually was. Now, let me show you from a physicist's point of view. Whenever I talk to people about consciousness from Chalmers, Bostrom, and upcoming guest Joshua Bach and others, I always get the same thing, like, we can't really define what consciousness is. So how do we know what thought is? So how can you determine what the laws of thought are? Isn't that kind of a extremely provocative and bold claim?

• 2:37Speaker 03

  The way that I approached that question in the book is really by thinking about what are the kinds of computational problems that minds solve? And that's really what this enterprise was. It's trying to figure out like, what's the mathematical structure that describes the thing that minds are doing? Whether that thing is what Aristotle was interested in, which is just trying to characterize what good arguments are, through to some of the questions that you were raising about, you know, like, what does it mean to make a good decision? And how do we think about, you know, rationality in that context? And so the interesting thing is, I think, a lot of those questions are things that we can answer without ever having to touch consciousness. I think about, one of the big challenges of studying consciousness is that we don't necessarily know what computational problem consciousness is solving.

• 3:19Speaker 03

  That's why it's something that's, you know, continued to be mysterious. We don't really know what it's there for in terms of, you know, how necessary it is to being able to do the kinds of things that minds do. And our AI systems give us nice demonstrations. You know, again, some people might want to argue that they're conscious in some form or something like that. But I think they give us nice demonstrations of how far you can get using certain kinds of mathematical formalisms.

• 3:43Brian Keating?

  Yeah, and there's many, many kind of allusions to physics in this book, which is so delightful in many different ways, not the least of which because it gives us some kind of formalism to hopefully go about this problem. But I, you know, as a physicist is want to do, I want to kind of get into what you would say, maybe, what is the briefest, you know, kind of most parsimonious, defensible definition of thought itself and the laws that govern it?

• 4:10Speaker 03

  In the book, I focus on deduction, which is, you know, sort of like patterns of logical reasoning going from things that are true to other things that are true. Induction, which is sort of seeing a pattern in the world and then making the generalization that thing holds in general. And then abduction, which is seeing something that you want to explain and then coming up with an explanation for it. And I think that's a pretty good characterization of the set of things that we normally have on our list when we want to try and explain sort of patterns of thinking. And those are the things that we try and engage with in terms of like the different kinds of mathematical formalisms that are explored in the book.

• 4:43Brian Keating?

  There's an awful lot of discussions of both the successes and, you know, our understanding of consciousness and the wrong turns. And I like that because for me personally, I hate when we teach our undergraduates as, you know, as often is done. You know, we basically just teach them the string of Nobel Prize winning experiments and, you know, just connect the dots. And that's, but you go through the, you know, the twists and turns. And I thought one of them was, was sort of brought up this, this conjecture that, or this statement by Feynman, which is that the, you know, kind of the difference between knowing the name of the thing and knowing something about it is the most dangerous gap in all of science. What are some of the inherent biases that science has brought to it? Because it's such a Frankenstein type field, cognitive science,

• 5:28Brian Keating?

  you know, start off with, with not really, as you discuss in the book, really being taken seriously and now it's, you know, at the cutting edge. What is the sort of, you know, largest gap or the biggest lacuna in, in your field where people seem to maybe be over abundant of confidence in describing how models work or even the model of the brain, let alone models of artificial intelligence?

• 5:51Speaker 03

  So one of the, I think, interesting challenges we have at the moment is having built systems that we don't fully understand, right? So we now have these AI systems that for computer scientists, put them in a very unfamiliar situation, right? Where if you're a computer scientist, you're used to programming something. And because you programmed it, you kind of know what it's doing. And that is not how our AI systems work. So these modern AI systems are built using enormous artificial neural networks and they learn.

• 6:20Speaker 03

  From data, far more data than any human could actually like read through and understand. And so you end up with something where it's both learned from a sort of incomprehensible amount of data and encoded that information in an incomprehensible number of continuous weights inside that system. And so as a computer scientist, you're then stuck and you're like, oh, what do I do with this? I actually think that's a good opportunity for cognitive scientists because we have been trying to study large, complex systems that we don't understand for about seventy five years now. Those systems are human brains. And a lot of the tools that we built for understanding human brains and how it is that, you know, humans think and behave are tools that we can now use to go back and really analyze these AI systems and try and understand a little more about how they work as well.

• 7:03Brian Keating?

  What would the advent of ChatGPT, what sort of thing would that be like? Is it the invention of the telescope, the cyclotron? What does it represent in your field?

• 7:12Speaker 03

  I think it's interesting. I'm not quite sure what the analog is because it's both a kind of like breakthrough in terms of revealing certain kinds of theoretical ideas can take us, you know, further than we might have thought. But also something that's given us a new set of problems in terms of trying to understand what that system is doing. And then trying to figure out like, you know, what all of its properties are and what the consequences of using those systems in certain kinds of settings is. It's both the validation of a theoretical approach, but also the creation of a new sort of field of inquiry.

• 7:45Brian Keating?

  And I talked to Steven Pinker about his most recent book. We had a conversation about that. Where humans use these heuristics and computational shortcuts. And you bring up a couple of these in the book, and I wonder if you could tell kind of some of the stories of Kahneman and Tversky and and how they illuminated this kind of shocking at the time claim that humans are necessarily not the best reasoners or not as reasonable as we think we are, right?

• 8:09Speaker 03

  Yeah, so there's an interesting paradox in trying to study human cognition from the perspective of computer science, right? So I live in these two departments. I live in the psychology department and the computer science department. And in the psychology department, my colleagues think humans aren't that smart, right? If you kind of like study human decision making, you find out that humans have all sorts of simple heuristics they follow that result in systematic biases. And that's the work that Kahneman and Tversky did, which was really kicking that off and giving us this picture of human cognition. And then if I walk across campus to the computer science department. Humans are the things that we're trying to emulate when we're building our AI systems. So they're sort of our best examples of systems that can solve certain kinds of problems. And so I think that tension is about the fact that the way that I would

• 8:53Speaker 03

  resolve it is that humans are actually good at solving a set of problems that are extremely hard, problems to solve. And they're not always necessarily solving exactly the problem that a psychologist asks them to solve when, when, you know, they sort of study them in the lab. So a simple example of this is if you flip a coin five times, which of the following sequences is more likely? Heads, heads, heads, heads, heads, or heads, heads, tails, heads, tails? If you just ask someone on the street, they'll probably say that heads, heads, tails, heads, tails is more likely, right? But, you know, as a trained physicist, you know, the probability of those two sequences is equal as long as it's a perfectly fair coin, right? Any sequence of five heads or tails is equally likely. And so one way to understand that, right? That's an error that humans make. That's

• 9:36Speaker 03

  the kind of thing you could point to and say humans are irrational, we're biased in this way. But one way to understand it is to say, what if the human is not solving that problem, but solving a different problem? So they're being asked to give you, what's the probability of this sequence under a random generating process? What if they're flipping that around and telling you, what's the probability that a random generating process produced this sequence? Or sort of how much evidence does the outcome give you for having been produced by a random generating process? And that's something we can calculate using Bayesian probability. And when you do that, it turns out people's judgments about randomness are very systematic and you can capture them with a nice, simple Bayesian model. But that's a case where we're sort of like reanalyzing the problem that human minds are solving. When you reanalyze it, it turns out people are doing a good

• 10:20Speaker 03

  job of solving that problem. And in some ways, it might even make more sense to be solving that problem. Because if you're wandering around in the world, it is very unusual for you to have to calculate the probability of sequences of things. But it's a good thing for you to be able to detect patterns that might suggest that something is non-random. And that's probably what our brains are built to do.

• 10:39Brian Keating?

  A central character in this book is a past guest, Noam Chomsky. And it's always been sort of, you know, kind of curious to me that his, you know, notions of generative grammar and so forth, you know, explain a lot from so little or seem to explain why, you know, for example, our children can learn language, you know, with far less training data, if you will, than can computers, these huge data sets with trillions of parameters now. But talk about his role in understanding how, you know, separate from AI, there's a clue to the laws of thought that emerged, you know, that caused the whole field of cognitive science to emerge. But it really is, you know, predicated on fairly elementary questions. It doesn't mean easy or simple. It just means that they're basic and important.

• 11:22Brian Keating?

  Talk about Chomsky's role in all this and whether his ideas are still pertinent to experts like you in the field today.

• 11:30Speaker 03

  So part of this story about people, you know, trying to use math to understand thought, it occurs in the middle of the 20th century when psychologists had decided that the only way to be rigorous about doing psychology was to not talk about thought and not talk about internal mental states, right? So this was an approach called behaviorism and the behaviorists said you should just focus on the things that you can measure, which are the environments that people act in and the behaviors that result from those environments. And so there was a group of sort of revolutionaries, there was what was called the cognitive revolution, which were psychologists and linguists and computer scientists who were interested in finding a different way to study the mind.

• 12:13Speaker 03

  And they did this by saying, another way to be rigorous about minds is to use math to express hypotheses about how minds work that we can then test through behavior. And so they did that using the kind of math that was most sort of obvious and accessible to them, which was the math of rules and symbols, inspired by Computers and logic and you know these sorts of formalisms that were very prominent in the 1950s, they set out to test out how well does that describe how minds and languages work. And so Chomsky took that approach and applied it to language. And he set up the problem in a way that was different from the way that previously linguists had thought about the problem. Linguists had kind of thought about, you know, their, their job in linguistics as characterizing

• 12:56Speaker 03

  the structures of different languages and then maybe looking for sort of commonalities and regularities in the structures of those languages. And Chomsky said, well, actually, if we kind of think about this as a math problem, a language is some set of sentences that you're allowed to produce. And let's characterize that set in a very mathematical way by specifying a generator of that set. Right? So he thought of a grammar as a system of rules that you could follow to generate all of the valid sentences in a language. And that approach, what's called generative grammar, became the foundation for much of theoretical linguistics, certainly through the 20th century, and then, you know, it continues to be influential today.

• 13:33Brian Keating?

  You talk about sort of a chessboard analogy with Chomsky. Can you sort of go through that or different types of moves? You start off with the initial, what is it, the 16 moves that can be made by each player? Talk about that analogy. Go ahead and explain it, this chessboard analogy.

• 13:47Speaker 03

  So you can think about this problem of defining a generator of a set. A good way to think about that is something like a board game. Right? So the rules of a board game are a set of principles that tell you what the, the states of the board are that you can reach. Right? And so you start out in some configuration. Chess is a good example, right? You've got all your pieces laid out. The rules tell you how to set up those pieces. And then you can make all of the moves that you can make from that position according to the rules, and that's going to take you to the next position. And then your opponent makes their moves that takes you to the next position. Right? So yeah, if you have 20 moves, your first move, the other person has 20 moves. At this point, there's already 400 configurations of the board that you could have reached. And that number keeps increasing, you know, exponentially as each, each subsequent move is made.

• 14:31Speaker 03

  But the end of making all those moves, you get to the end of the game, and by following the, the sequence of rules, you've generated all of the possible games of chess. Right? And so that's his idea, is that. Just as there's a set of kind of like, you know, games of chess that you can follow, final board positions that you can reach, there's some set of sentences that are the things that are in English, and maybe we can come up with an analog of the rules of chess that generates all of the valid sentences in English.

• 14:57Brian Keating?

  One of my favorite aspects of the book is you kind of trace through the history of thinking about thinking, metacognition, whatever you want to call it. And you start with Aristotle. I love Aristotle, who doesn't? But his claims to fame in physical sciences are not so strong, right? I mean, they haven't really held up as well as his laws of thought or logic. I mean, he thought that things fell to the center of the earth because heavier things fell faster than lighter things, which Galileo disproved, you know, with a simple, you know, allegedly dropping two objects off, or even a thought experiment, you know, speaking of the laws of thought, the role of thought experiments is not insignificant. But he thought that, you know, women had fewer teeth than than men. He had a wife because he had a son, Nicomanchin, right? Nicomanchus was his son, right, Tom?

• 15:43Speaker 03

  Yeah, I think you know your Aristotle better than I do.

• 15:45Brian Keating?

  Well, the one claim to fame is that he knew that whales were mammals. But why does Aristotle, you know, get so much right about thought? And how can that possibly still matter, you know, 24 centuries later?

• 15:56Speaker 03

  I think part of that is that he was...

• 15:59Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 16:29Speaker 03

  Doing math, essentially, right? When he was thinking about thought. So what Aristotle did, he had two projects that I talk about in the book. And the first of those was the part that's about deductive logic. And this is setting up the set of syllogisms. So a syllogism is a simple argument with two premises and a conclusion. And, you know, these are sort of familiar kinds of things you've probably seen in school. It's like, you know, all A's are B's. All Bs are C, therefore, you know, all As are C, right? And so that's a, uh, an example of a syllogism. And he was interested in characterizing what's the set of these syllogisms? And then which of these are valid? In a way that's actually, you know, quite like that sort of Chomsky problem, right? Of being able to say, you know, like, what are the good ones and what are the bad ones? And so that was really a matter of just enumerating.

• 17:14Speaker 03

  So he was kind of like doing the combinatorics of these kinds of arguments. He enumerates all of the arguments. He says, some of these I know are good, and I'm just going to say those are good ones. And then he makes little mathematical proofs to relate, you know, some of the, the other arguments back to, you know, the ones that he knows are good, and he can sort of say things about those two. And so I, I think, I think his success there was that he was involved in exactly the kind of mathematical enterprise I talk about in the book. He then had a challenge that was left over from that, which is like, you know, exactly the Chomsky challenge again. Can I come up with a mathematical system that characterizes the good ones, right? And separates them from the bad ones? And then that's the challenge that was picked up by Leibniz and later by, by Boole.

• 17:55Brian Keating?

  So let's get to Leibniz, because you mentioned him. He had this dream, which seems kind of insane at the time, to, you know, logify or to codify, to mathematize or reasoning. So was he basically trying to invent AI 250 years before computers existed?

• 18:12Speaker 03

  That's pretty much exactly what he was trying to do. And, and he was, he was the right kind of crazy, right? He really was someone who had a vision that far transcended the times that he lived in and made contributions to a huge number of different disciplines as a consequence. He was obsessed with the mathematics of combinations, interested in all kinds of mathematics. You know, he contributed to the calculus and so on. He built a calculator, a mechanical calculator, that was able to do more sophisticated things than the other mechanical calculators at the age. So he had all these pieces where he knew kind of like what mathematics could do. And he knew that if something could be expressed in mathematics, it could be executed by a machine. And so those things came together.

• 18:56Speaker 03

  He'd been studying logic since he was a kid and reading Aristotle. And he had this dream of being able to take Aristotle's syllogisms and then figure out a mathematical system that would let him essentially like, you know, then run this on his calculator. So that if anybody wanted to have an argument about something, he could, you know, put it into the machine and then turn the handle and out would come the answer about who had the, who had it right.

• 19:20Brian Keating?

  Maybe he was just too early, or is it really possible to do what he was attempting to do? Maybe he underestimated how hard representation would be.

• 19:29Speaker 03

  He had some really good ideas that, again, were ahead of his time. And then he had one thing that he hadn't quite figured out. And so the really good ideas were, he's the person who invented this idea of vector embedding, as far as I'm concerned. So he, the way that he tried to solve this problem was by taking the terms that would appear in those syllogisms, the A's and the B's and so on, and trying to represent them with a little vector of numbers. So he would associate, in his case, it was just two numbers with each of those terms. And then he tried to find the relationships between premises and conclusions by then reducing this to regular arithmetic, where you'd have like, you know, the number thirty three and the number, you know, minus seventy seven associated with one of the terms.

• 20:12Speaker 03

  And then if that could be divided by the numbers for another one, say it was like eleven and seven, that would be something where you could say, okay, now the conclusion's going to follow from that. And so he kind of worked out this system that was just based on arithmetic, having vectors that you are modifying through these arithmetic operations. That was really smart. You know, that turns out to be really important for AI today. That's how language models represent words as well. The thing that he hadn't quite figured out and sort of got glimmers of at the end of his life was that he didn't have the right algebra, right? He was like using regular arithmetic. And it turns out in order to capture the content of the syllogisms, you need something that's, it's a little more complicated than regular arithmetic.

• 20:52Brian Keating?

  Yeah, so let's segue into George Boole. And what did he really change? And most of us, if we know about Boole, his name, it's from Boolean logic and computer circuits, and we, we stop there with the X NOR and all the other circuit diagrams you talk about in the book. But, but in your telling, Boole's a much more important character. So what do we get wrong about him?

• 21:12Speaker 03

  He was sort of genius who went beyond the moment that he was in. He spent most of his life as a schoolteacher. And even as a schoolteacher, was corresponding with the leading mathematicians of the day, publishing, you know, really influential papers. He ended up winning this gold medal in mathematics from the Royal Society. And that was, that was sort of his. Precursor to the contributions that he made to logic. But his skill as a mathematician was really around these kind of algebraic ideas. And he had essentially taught himself this perspective on mathematics by reading, you know, hard math books from from France that, you know, no one else in England was really reading. And he said, like, he enjoyed reading these big, thick math books because it was the best way to take to get his small allowance for books to last as long as possible.

• 21:59Speaker 03

  And so he had this toolkit that was the one that Leibniz was missing, which is this algebraic toolkit. And then he could recognize that in order to capture the structure of thought, you needed this, this, you know, slightly different algebra. And then that's the thing that we now associate with with Boole. But his, his work really went far beyond that. The title of my book, The Laws of Thought, he was someone who was actively involved in this 19th century community of people who was trying to characterize what the laws of thought were. And his big book was called An Investigation of the Laws of Thought. And my epigraph comes from from Boole as well. And in that book, he laid out both the kind of foundations of this mathematical logic, but also, you know, principles of probability theory that he thought were going to be the way

• 22:42Speaker 03

  to extend this to solve other kinds of problems of thinking as well.

• 22:46Brian Keating?

  Presaging a lot of what we have come to use. Is it a question of efficiency, that it's just super efficient to do things with zeros and ones? And and you can reduce all sorts of these abstract thought concepts to zeros and ones? Or is it not merely the computational efficiency that caused the success?

• 23:03Speaker 03

  I think it's that by expressing things in that way, he was able to then do the thing that Leibniz wanted to be able to do in terms of now it was possible to think about, you know, creating machines that would be able to execute these kinds of computations, right? So Boole's work was then developed into a richer theory of mathematical logic. That fact that you could express mathematics in, you know, a mathematical form itself, you could take statements that were mathematical statements and express them in logic, and that would turn them into math themselves. That became the foundation for a lot of work on sort of asking questions about the limits. Of mathematics that inspired Turing to think about what's a an abstract kind of machine that you could use to

• 23:47Speaker 03

  to to do these kinds of calculations, to to emulate the mind of a mathematician. And then von Neumann figures out a scheme for building these machines that still underlies the computers that are on our desks today.

• 23:58Brian Keating?

  Do you think that von Neumann machines, Turing machines, et cetera? Do you think that they will be, you know, kind of permanently ensconced in this in this discussion? Or other architectures and even other approaches towards AI? Will they eventually supersede based on efficiency? The same way that, you know, Bo was able to supersede in some sense, Leibniz?

• 24:17Speaker 03

  Yeah, so Turing machines were never a practical device, right? It was a sort of theoretical abstraction for how you could describe computation. Von Neumann worked out how to have a stored program computer, right? And so how you can have a computer which has, instead of having to rewire it every time you want to solve a different problem, it's able to use, you know, software to modify what it is the system's doing. And that's a fundamental advance in terms of being able to create machines that can do all of the kinds of thinking that we want them to do. Nowadays, a lot of the training of artificial neural networks is done by using dedicated hardware, GPUs, graphics processing units, right? Which are units that were originally designed to just speed up the computations required to put things on a screen. But those computations turn out to be exactly the computations that you need to do to run a neural network.

• 25:02Speaker 03

  And so there's lots of diversification of specialized hardware for doing those kinds of things. It's also interesting to note that the earliest neural networks, so, you know, neural networks that were built by people like Frank Rosenblatt and Marvin Minsky, they were also specialized hardware. They they built physical neural networks that were, you know, sort of connected up by wires with adjustable resistors on them. I think that's certainly a kind of technology that's that's changing the way that, you know, we're thinking about computation today. And, you know, a lot of the energy that's going towards compute is now going towards GPUs. The fact that a lot of energy is going towards those is something that's encouraging people to think about alternative models for computation. If what you want to do is run neural networks, maybe we can learn things from the neural networks that we run inside our heads,

• 25:46Speaker 03

  which run on far less energy than the kinds of neural networks that, you know, people Are running on GPUs.

• 25:51Brian Keating?

  Yeah, you talk also in the book, I mean, speaking of GPUs, Jensen Huang was on Lex Rabin's podcast recently, he said AGI is here. I keep saying that I'm not really convinced that AGI will be here until it can do something that human beings have never been able to do. And the clearest, kind of most simple realm to demonstrate that is in the laws of math or some, you know, physical observation that we've never really been able to explain, you know, unifying quantum mechanics and gravity, something truly novel, or at the very least, you know, replicate what what human brains did 100 years ago, you know, long before computers. For example, if you just gave it the data on the planet Mercury from 1911 and before, Einstein certainly knew that there was this anomalous procession.

• 26:34Brian Keating?

  In fact, GR was basically designed, retrodict, to explain why that behaved that way. And yet, we can't seem to get that to, my student Evan Watson and I have tried to replicate, you know, could you come up with GR from just the deductive observations of data, which we have hundreds of years about from Mercury, right? So what what is your working definition of AGI?

• 26:56Speaker 03

  As a cognitive scientist, I would be very sort of careful about thinking about, you know, this idea of artificial general intelligence in the first place, because I think it plays into a bias that we have, which is that our best example of an intelligent system is another human being. And all of our intuitions about intelligence are based on the kinds of things that human beings do. Right? And so I think that encourages us to think about it's in a kind of like one dimensional way, where that's kind of like, here's where humans are on this one dimensional scale of intelligence. Here's our AI systems are coming closer and closer. And one day, oh, they're going to be past us. And then either something wonderful or something terrible is going to happen. And so that

• 27:36Speaker 03

  one dimensional characterization, right? So this is like AI or superhuman AGI or whatever it is. I think that's not a productive way of thinking about what's going on with our AI systems. I think a better way of thinking about it is that human minds and our AI systems are both systems that have been created to solve certain kinds of computational problems. They've been sort of optimized to solve those problems, but they've been optimized Some of those problems overlap, but they've been optimized in sort of different ways and under different constraints. So human minds have evolved under constraints on just what, you know, human lifetimes. We only live a few decades. Those compute resources I was talking about, right? We only have a couple of pounds of neurons up there and bandwidth constraints in terms of like, you know, we're limited in our ability to communicate with one another.

• 28:21Speaker 03

  We have to do things like talk to each other on podcasts in order to share information, right? Whereas our AI systems can have way more data than a human can see. They can potentially just scale arbitrarily in the amount of compute that they use. And you can transfer data from one machine to another. You can transfer weights from one machine to another. There's a lot more sort of plug and play compatibility in terms of, you know, being able to spread that intelligence around. That means that the solutions that those systems find can look quite different. Where we've made AI systems by essentially optimizing them to solve this problem of getting a radio signal from another planet, right? And trying to predict the things that are occurring in that radio signal to the point where they're really good at it.

• 29:05Speaker 03

  And they've even made inferences about the aliens that live on that planet and what kind of cities they live in and what kind of interactions they have, right? That's the problem that the AI system is solving. And the human is doing something quite similar, but they're doing it in, you know, a social context where they're interacting with other humans. And they're doing it with the benefit of like, you know, thousands and, you know, hundreds of thousands of years of evolution behind them, right? And so we end up sort of seeing similar kinds of behavior from these systems, but seeing it from two quite different evolutionary trajectories and seeing it under two quite different sets of constraints. So saying, you know, one thing is like the other thing. I think it's sort of misleading. I think they're sort of on, you know, these different trajectories. And so we're going to end up with things that are really smart in ways that go beyond the kinds of things that humans can do.

• 29:51Speaker 03

  But also maybe surprise us in the other things that they're not able to do, because those things don't show up in the training data or they have the wrong formulation of the learning problem or whatever it is.

• 29:59Brian Keating?

  You speak in the book about what Chomsky called Plato's problem, how human beings know so much from so little. But you know, when I'm had on Jan Le Cun on this podcast, he said it's the exact opposite. AIs, you know, have tremendous amounts of information, but it's not even close to the amount right now filtering out something like 13 terabytes of raw information if you were to encode it, which I think is ridiculous. But even just foveal recognition and, you know, the camera or what have you. I mean, it's certainly millions of megabytes, gigabytes, right? So isn't it the opposite? I mean, I read, you know, my kids were little that they need to hear a million words before they can speak. And if you just compress that, I mean, that's an awful lot of data, isn't it?

• 30:41Speaker 03

  Plato's problem, right? You said, how do we come to know so much from, you know, so little, right? And Chomsky talked about this as the poverty of the stimulus, right? And the idea being that there's not enough information in what the kids hear to determine the structure of the language that they end up speaking, right? So I actually think that our AI systems are in some ways a good demonstration of this, which is that if you give them as much data as a kid gets, they're still not as good as a kid at learning language, right? We have arguments about what it means to give them exactly the same data that a kid gets. And I have colleagues here who, you know, are measuring different aspects of what that looks like.

• 31:20Speaker 00

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 31:49Speaker 03

  But Chomsky's argument in particular was focused on syntax, right? So how you know some very nuanced things about the structure of language based on the experiences that you have. And he thought there's not enough information that's contained in the stimulus that you see. And, you know, to the extent that we can train models on at least the number of words that a kid would have seen, those models are still not doing as well as a kid from that amount of data. So I think that does support the idea that humans bring to these learning problems, something that the AI models are not getting.

• 32:20Speaker 03

  Right, so humans, they have something that a machine learning researcher or cognitive scientist calls inductive bias. So something other than the data that influences the solutions that they're reaching. Those inductive biases are what allows us to learn quickly, more quickly than our neural networks do, from limited amounts of data. They're also something that influences what solution we find. So if you have your neural network playing this alien radio prediction game, it's going to find some solution to playing that game. But that solution might not be one that is very intuitive to us as humans, right? It's sort of like figured out some weird stuff that are regularities that it can use in making those predictions, but it's maybe not got a really good model of the underlying world or things like that. Whereas the kinds of solutions that

• 33:03Speaker 03

  a human will find are going to be influenced by those inductive biases. So part of what allows humans to generalize smoothly from one problem to another, and to act in ways that are predictable to other humans, and to sort of show intelligence that has those properties of generality that you were alluding to, is the inductive bias that we bring to those problems. And I think that's another sort of poverty of the stimulus argument. It's like, if you want to get sort of appropriately general learners, you might need to have some inductive bias to get that smoothness.

• 33:32Brian Keating?

  It seems to me that one reason that humans flourish is that we're comfortable with ambiguity. You know, for example, a question like, is an olive a fruit, as you point out, it's pretty deep philosophically. Well, why is it that, you know, humans, even my, you know, kids can understand it, but but it sort of leads to either AI psychosis, or hallucinations, or sycophancy, I'll ask you which is the worst. But but why is the question like, you know, is the moon a light bulb? Why are those deeper than they look to be?

• 34:01Speaker 03

  Those kinds of questions, I think, in cognitive science have been useful in revealing exactly what our concepts are. So people coming out of that rules and symbols tradition thought, oh, maybe a concept is just a definition, right? And I think that's a good intuitive way of thinking, like what a concept is, right? You sort of have the intuition, you can look something up in a dictionary, and it's going to tell you, oh, what a cat is. Okay, a cat has these properties, and that's what makes it a cat. That way of thinking about the world sort of prevailed through the 50s into the 60s, and then was pretty firmly rebutted by cognitive scientists called Eleanor Rush, who showed that there's systematicity in the way that people have uncertainty about category membership, right? So your listeners can think

• 34:43Speaker 03

  about this, right? So if I ask you, is a chair a piece of furniture? Probably yes. Is a phone a piece of furniture? Probably no. Is a lamp a piece of furniture? Maybe, right? Is a rug a piece of furniture? Probably not, right? So you can sort of immediately begin to explore this fuzzy boundary. And that fuzzy boundary is a clue that there's probably not a rule underlying your notion of what furniture is. In fact, it has what Roche called a family resemblance structure, where there are some things that you're sure are part of the family. And then there are other things that sort of share some attributes with them. And then there's sort of fuzziness that that sort of goes out from there.

• 35:24Speaker 03

  And so when we come to AI systems, that kind of thing was a challenge for AI systems that were based on systems of rules. And that was, again, the dominant approach for building AI systems. Now, through the 1970s, through the 1980s, people were were making AI systems based on what were called production rules. There was a company that has continued to the present day, building a huge database of rules with the hope that if you've got enough rules, then you figure out what the structure of the world is like. The neural network approach really, in some ways, sprung up as an alternative to that, that would be able to capture this fuzziness and all of the graded, continuous things that seem to be important properties of human concepts.

• 36:00Brian Keating?

  You talk about the semantic revolution. Can you can you talk about, first of all, what is a semantic network? And then explain the shift that made that, you know, possible and made the concepts becoming nodes in a weighted network, rather than sort of a compendium of facts? Why was that such a breakthrough or seminal event?

• 36:17Speaker 03

  If we want to capture that fuzziness of concepts, you need to have some way of having graded relationships between things, right? And so your representation of furniture is now connected to chair very strongly, but, you know, connected to rug much more weakly, right? And so you can capture that by creating a semantic network, a network where each node in that network is a thing, concept, and each, you know, that they have links between them that reflect their their strength. And psychologists began to show that that wasn't just a good way of storing information about the connections between things, but actually turned out to be a pretty good model of human memory, where like if you said to somebody a sentence that contained one of those words,

• 36:59Speaker 03

  then it would be easier for them to remember or recognize another of those words that was closely associated with it. Like, activation of words seemed to sort of spread through that network. And so That was something where psychologists began to realize that maybe there was a different way of conceptualizing what thought is. You can think about it now as you have all of these concepts. Each of those is activated to some extent. Now you have a high dimensional space, which is the space of all of the activations of those concepts. You have a point in that space and that's your current mental state. And then the weights between things tell you how those mental states are sort of evolving over time. And now we have this alternative to that sort of logic, rules and symbols based theory of how it is that minds work.

• 37:39Brian Keating?

  Walk us through an example in this besides the furniture. It seems like there's almost a geometric or, you know, Riemannian curvature approach that took over. Is that where the kind of insights of Hinton and gradient descent, is that the kind of novelty that was applied by Hinton and his colleagues?

• 37:55Speaker 03

  Yeah, so if you have this idea that, you know, we want to now have networks of things that are connected up to each other by different strengths. And maybe we can even take away the idea that those nodes in those networks have labels on them. And maybe they're just nodes that represent information somehow, right? That's what leads us to neural networks. Psychologists had been exploring neural networks for a long time, even all the way back to the 1950s. The first kind of, when, you know, people were developing the first AI systems, there were also people working on implementing neural networks on computers at that time. As I said, building neural networks by hand. So Frank Rosenblatt, who was a psychologist at Cornell, he was originally a social psychologist and he had written a dissertation that required aggregating a whole lot of survey data.

• 38:39Speaker 03

  And so he sort of found out about the computer on campus and started messing around with that and then built a circuit in order to aggregate the data from his surveys. And suddenly you had a psychologist who understood computers and who understood circuits. And he was like, ah, I've got it. I'm going to build a brain, right? He sort of had the pieces and the insight to think about how to do that. And so he built some of the first mechanical brains or electronic brains. I say mechanical because the way that he did it, he had a sort of artificial retina that you would show something to and it would, it would produce responses from little, little, you know, sensors that were in that retina that would tell whether it was seeing something light or dark. And then that information would get sent to another set of units, these nodes that would be accumulating information from the retina.

• 39:22Speaker 03

  And then he had another set of connections that went from those to an output. So for example, it could be deciding whether it saw a square or a circle. And so those connections to the output had a little resistor on them that could adjust to reflect the strength of that connection. And he came up with a learning algorithm that made it possible for the system to learn to differentiate simple shapes, circles from squares, or simple letters like, you know, E's and F's or something like that. And he proved a theorem that anything that the system could represent, it would be able to learn, which was great. He was very, he went off and sort of publicized the capacities of the system, which was called a perceptron. The problem was his former schoolmate, Marvin Minsky, had also built his own neural network.

• 40:05Speaker 03

  While he was a PhD student at Princeton, he went to Harvard, where he'd been an undergraduate and built a neural network in the basement of the psychology department out of leftover airplane parts. And he looked at this thing, he'd written his PhD dissertation on learning in neural networks, and he implemented this. And he looked at it and he was like, you know what? In order to learn anything interesting, this would just have to be so big and cost so much money that it's not, it's never going to work. And so he gave up on learning in neural networks, got interested in symbolic approaches to learning. And so when Rosenblatt, again, his schoolmate, right, came out and said, oh, neural networks can learn all these things, Minsky was not impressed. And then with Seymour Papert, wrote a book that showed that perceptrons were sort of fundamentally limited in the kinds of things that they could

• 40:48Speaker 03

  represent. And the reason for that limitation was that single layer of weights in the network. And so the reason why that was a limitation was that the perceptron with a single layer of weights could only represent linear boundaries in space, right? So if you think about all of that information is coming in, it's going into a high dimensional space. And now it's trying to find a linear sort of, you know, partition of that space in order to separate the things from each other. And so Rosenblatt's learning algorithm could find those boundaries, but there were lots of problems where no such linear boundary existed. The solution to that problem was to make a neural network that had multiple layers. And various people kind of like came up with strategies for making this work. The problem was that Rosenblatt's

• 41:30Speaker 03

  learning algorithm didn't work for multi-layered networks, it only worked for one-layered networks. He had a sort of a trick for doing this that he called back propagation, but it didn't quite work. It sort of worked most of the time. Another group of psychologists got interested in these neural networks, thanks to semantic networks and spreading activation and so on. And so this was David Rumelhart. Jay McClelland at UCSD and then a postdoc that they hired Jeff Hinton, who was working on that project. And so Hinton suggested to Brummel-Hart that he could set up that problem as one of gradient descent. Right. So this is, you know, basically thinking about there being some measure of how well the neural

• 42:12Speaker 03

  network is doing and then adjusting the weights in the network in the direction that would decrease the error that the system was making. And then using that insight, Rommel Hart was able to re-derive something like Rosenblatt's learning rule. And then he was able, you know, on a plane flight, when he was off to a grant reporting meeting, had enough free time to sit down and work out the whole thing in his notebook and, you know, derived the, the learning rule for multi-layered networks. Satisfyingly, one of the, the fundamental principles that was needed for that was something that came from Leibniz, from Leibniz's calculus, the chain rule. So Leibniz got to have his day after all, you know, a couple of centuries later. Hinton was actually

• 42:53Speaker 03

  the great, great grandson of George Boole. So they met again together in that, in that location.

• 42:58Brian Keating?

  I was wondering, you know, kind of the, as a practicing, you know, researcher in this field, much more adjacent to it than I am, although I use it every day, all day in some cases, much to disagree with my wife. But the biggest problem that you see with LLMs, is it psychosis? Is it hallucination? Is it sycophancy? I mean, I love sycophancy. You know, when I asked it, you know, what books has Brian Keating written? It says, Losing the Nobel Prize, Into the Impossible and A Brief History of Time. And I just thought that was awesome. I would love to get some of Stephen's, you know, book royalties. But, but what's the biggest concern for you when it comes to AIs? Not Doomer, it's going to take all our job. We'll talk about meaning at the very end, but what's the biggest?

• 43:36Speaker 03

  Yeah, I think there's a few things. So, so one is this jaggedness, right? This sort of like lack of generalization, where I think we as humans can end up overconfident in the kinds of things that the AI systems can do, because we apply our intuitions that tell us, if you had a friend who could solve, you know, International Math Olympiad problems at a gold medal level, you would trust them to do all sorts of other things on your behalf, but you should not trust an AI system to do that, because they don't generalize across problems in the way that people do. So I think, you know, just having the wrong intuitions about these systems is, is a major, you know, bottleneck to our being able to think about how to employ them effectively and how to make predictions about the kinds of things they're going to be able to do. And that was part of my motivation in writing the book as well as. Giving people some of the context for where these things come from and a sense of what

• 44:20Speaker 03

  the problems are that can come out of that and maybe what some of the kinds of solutions are historically that people have found. Of the other things that you mentioned, hallucinations, I don't mind very much in the sense that they're relatively easy to catch if you have some domain expertise. And I think they're actually good in some contexts. So one of my best tricks for getting the models to generate good research ideas is to ask them to tell me about papers that I haven't heard of but should know about. And when they do that, they'll often hallucinate and make up a paper. But the ideas in that paper are much more interesting than if I ask it to just tell me some interesting research ideas, right? So having conditioned on generating a published paper actually makes it produce something which is higher quality. I think sycophancy is a major problem. We have a

• 45:03Speaker 03

  recent paper, this is with Rafael Batista, where we show if you take a rational agent who's doing Bayesian updating on their beliefs and have them interact with a system which is sycophantic in the sense that it's generating data based on the hypothesis that the agent expresses to the system, then that agent is going to become increasingly confident in their beliefs, but no closer to the truth. And we have some demonstrations that this actually happens with real deployed systems where we have people trying to solve a simple problem. And if they're interacting with the default prompting for a GPT, they end up not making progress in that problem, even though they become more certain that they found the right answer.

• 45:42Brian Keating?

  And then the last two questions I have. One is, you know, for someone, you know, looking to get at the future, where the future is going, where the puck is going, you have some hockey analogies in the book, I'll leave it for the readers to encounter them. But skating to where the puck is going to be, it seems like one thing that's really missing or is not fully developed is the, you know, embodiment issue where you have truly, you know, maybe close to AGI, you have very advanced intelligence coupled to robotics or embodiment. And maybe it's what it's missing or what these systems are missing is this marriage, which will unlock via some network effect that we don't understand, you know, truly human level thought. I always use the analogy of what Einstein, who worked not far from you, called his happiest thought, which was that, you know, an observer in free fall would experience no gravit

• 46:26Brian Keating?

  Acceleration force and that led him to the Einstein equivalence principle. So I always ask, you know, how can the computer visualize, you know, the zero gravity feel of going, you know, the elevator cable getting cut? And then second of all, how can it have a happiest thought? Maybe we could incentivize it that way, but maybe you could embody it, you know, if it gets the answer wrong, if it's truly, you know, sycophantic, you blow out some of its capacitors or, I don't know, you feed it some, you know, training data only from the Fast and the Furious, you know, movie genre series. But tell me, what, what would be kind of the next unlock as you see it? To truly get us to the next, to the next level that, you know, maybe incomprehensible to Minsky and Chomsky and all the other folks that we mentioned in the book.

• 47:07Speaker 03

  Yeah. So I think there are two parallel things here, right? So one is inductive bias. So trying to figure out what it is that's inside humans that allows us to find solutions faster and that are, they're more robust and more generalizable. So that's a good opportunity for cognitive science to contribute something to AI. Second thing is getting something which is closer to human experience into these neural networks where, like I said, they're, they're being trained to predict alien radio signals. If they have experiences that are closer to those of a human child, that might be something that helps to create those more generalizable, more robust kinds of representations of the world. And then, you know, embodiment is obviously a part of that. It's not clear to me that

• 47:47Speaker 03

  that on its own is necessarily going to solve problems of, you know, like allowing these models to be more creative and to solve more kinds of problems. In a recent paper with L L U in my lab, we show that prompting models to make cross domain metaphors. So, you know, to come up with a product design for a car based on ideas from an octopus does not increase their creativity, doesn't increase the, the originality of the ideas that they produce, but it does for people. So it seems like some of the tricks that we have for getting humans to have good ideas are not necessarily things that are effective for our large language models. And so that, you know, maybe is some fundamental difference in architecture, but it makes me a little less optimistic that,

• 48:30Speaker 03

  you know, just doing things like providing embodied experiences that you might be able to draw on to form these. Analogies might be enough to get them to be more creative.

• 48:39Brian Keating?

  And then lastly, you end on a hopeful note, not really a doomer as I tend to be, but you know, kind of advice to early career, you know, scientists or maybe even laypeople. Cause you just gave us some examples of what a career, early career cognitive scientist might do. But what should a layperson take away from this book?

• 48:55Speaker 03

  Really, what I wanted to do is to give people a sense of context and a vocabulary and a set of tools for thinking about these systems. Where I think for many people, AI seems like something that suddenly came out of nowhere two years ago, you know, like all of a sudden you could talk to a computer in the way that you talk to a human. And knowing the, you know, a couple hundred years of stuff that led up to that is helpful in terms of understanding what it is those systems are doing, why they can do it, what the limitations are that we might expect that they would have, what things are going to be hard for them to do, what are the next steps that might help to fill in some of those gaps and having a way of having an informed conversation about those things. The laws of thought here, as I said, something that in principle, we should be teaching in school, not just to help us understand how our own minds work, but to help us understand

• 49:40Speaker 03

  the world that we're moving into.

• 49:42Brian Keating?

  Professor Tom Griffiths, Princeton University, this book has done something that very few books can even attempt and let alone pull off, tell the history of cognitive science and also the future where it's going and get inside of the mind of one of the greatest researchers of our generation and those that came before him. Tom just told you that the godfather of AI is the great, great grandson of the man who invented its math, that sycophantic AI makes you more confident, but no closer to the truth and that a child still can beat a GPT at the same data budget. Now, if all that reframes what you thought these machines were for, hit subscribe and turn on the notification bell, drop a comment. What did Tom break for you? And if you want to go deeper, I talked about consciousness and machine minds with David

• 50:27Brian Keating?

  Chalmers. The link is right here. I know you're gonna love it. Go ahead, hit subscribe.

• 50:32Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com. Not authorized by any candidate or candidate's committee.

• 51:01Speaker 05

  What if you could run your Oracle databases everywhere you run your business? Good news: you can. Oracle AI Database runs in every major cloud, so you can use your live data for all your AI and analytics needs. Learn more at oracle.com/migrate.