Judea Pearl: Causal Reasoning, Counterfactuals, Bayesian Networks, and the Path to AGI

Judea Pearl is a professor at UCLA and a winner of the Turing Award, that’s generally recognized as the Nobel Prize of computing. He is one of the seminal figures in the field of artificial intelligence, computer science, and statistics. He has developed and championed probabilistic approaches to AI, including Bayesian Networks and profound ideas in causality in general. These ideas are important not just for AI, but to our understanding and practice of science. But in the field of AI, the idea of causality, cause and effect, to many, lies at the core of what is currently missing and what must be developed in order to build truly intelligent systems. For this reason, and many others, his work is worth returning to often. This conversation is part of the Artificial Intelligence podcast. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter https://twitter.com/lexfridman, LinkedIn https://www.linkedin.com/in/lexfridman/, Facebook https://www.facebook.com/lexfridman, Medium https://medium.com/@lexfridman, or YouTube https://www.youtube.com/lexfridman where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts https://podcasts.apple.com/us/podcast/artificial-intelligence/id1434243584 or support it on Patreon https://www.patreon.com/lexfridman. This episode is presented by Cash App. Download it (App Store https://apps.apple.com/us/app/cash-app/id711923939, Google Play https://play.google.com/store/apps/details?id=com.squareup.cash&hl=en_US), use code “LexPodcast”. Here’s the outline of the episode. On some podcast players you should be able to click the timestamp to jump to that time. 00:00 – Introduction 03:18 – Descartes and analytic geometry 06:25 – Good way to teach math 07:10 – From math to engineering 09:14 – Does God play dice? 10:47 – Free will 11:59 – Probability 22:21 – Machine learning 23:13 – Causal Networks 27:48 – Intelligent systems that reason with causation 29:29 – Do(x) operator 36:57 – Counterfactuals 44:12 – Reasoning by Metaphor 51:15 – Machine learning and causal reasoning 53:28 – Temporal aspect of causation 56:21 – Machine learning (continued) 59:15 – Human-level artificial intelligence 1:04:08 – Consciousness 1:04:31 – Concerns about AGI 1:09:53 – Religion and robotics 1:12:07 – Daniel Pearl 1:19:09 – Advice for students 1:21:00 – Legacy

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Lex Fridman?

  The following is a conversation with Judea Pearl, professor at UCLA and a winner of the Turing Award, that's generally recognized as the Nobel Prize of Computing. He's one of the seminal figures in the field of artificial intelligence, computer science, and statistics. He has developed and championed probabilistic approaches to AI, including Bayesian networks, and profound ideas in causality in general. These ideas are important, not just to AI, but to our understanding and practice of science. But in the field of AI, the idea of causality, cause and effect, to many, lie at the core of what is currently missing and what must be developed in order to build truly intelligent

• 0:44Lex Fridman?

  systems. For this reason, and many others, his work is worth returning to often. I recommend his most recent book, called Book of Why, that presents key ideas from a lifetime of work in a way that is accessible to the general public. This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, give it five stars on Apple Podcasts, support it on Patreon, or simply connect with me on Twitter, at Lex Fridman, spelled F-R-I-D-M-A-N. If you leave a review on Apple Podcasts especially, but also Castbox or comment on YouTube, consider mentioning topics, people, ideas, questions, quotes, and science, tech, and philosophy that you find interesting, and I'll read them on this podcast.

• 1:29Lex Fridman?

  I won't call out names, but I love comments with kindness and thoughtfulness in them, so I thought I'd share them with you. Someone on YouTube highlighted a quote from the conversation with Noam Chomsky, where he said that "the significance of your life is something you create." I like this line as well. "On most days, the existentialist approach to life is one I find liberating and fulfilling." I recently started doing ads at the end of the introduction. I'll do one or two minutes after introducing the episode, and never any ads in the middle that break the flow of the conversation. I hope that works for you and doesn't hurt the listening experience.

• 2:08Lex Fridman?

  This show is presented by Cash App, the number one finance app in the App Store. I personally use Cash App to send money to friends, but you can also use it to buy, sell, and deposit Bitcoin in just seconds. Cash App also has a new investing feature. You can buy fractions of a stock, say $1 worth, no matter what the stock price is. Brokerage services are provided by Cash App Investing, a subsidiary of Square, a member SIPC. I'm excited to be working with Cash App to support one of my favorite organizations called First, best known for their first robotics and Lego competitions. They educate and inspire hundreds of thousands of students in over 110 countries and have a perfect rating on Charity Navigator,

• 2:51Lex Fridman?

  which means the donated money is used to the maximum effectiveness. When you get Cash App from the App Store or Google Play and use code LEXPODCAST, you'll get $10 and Cash App will also donate $10 to First, which again is an organization that I've personally seen inspire girls and boys to dream of engineering a better world. And now here's my conversation with Judea Pearl.

• 3:35Lex Fridman?

  You mentioned in an interview that "Science is not a collection of facts by a constant human struggle with the mysteries of nature." What was the first mystery that you can recall that hooked you, that captivated your curiosity?

• 3:48Speaker 01

  Oh, the first mystery, that's a good one. Yeah, I remember that. I had a fever for three days. And when I learned about Descartes' analytic geometry, and I found out that you can do all the construction geometry using algebra. And I couldn't get over it. I simply couldn't get out of bed.

• 4:15Lex Fridman?

  So what kind of world does analytic geometry unlock?

• 4:20Speaker 01

  Well, it connects algebra with geometry.

• 4:24Speaker 01

  Okay, so Descartes had the idea that geometrical construction and geometrical theorems and assumptions can be articulated in the language of algebra, which means that all the proof that we did in high school trying to prove that the three bisectors meet at one point and that, okay, all this can be proven by just shuffling around notation. Yeah, that was a traumatic experience.

• 5:00Lex Fridman?

  Traumatic experience.

• 5:01Speaker 01

  For me it was, I'm telling you.

• 5:03Lex Fridman?

  So it's the connection between the different mathematical disciplines that they all

• 5:07Speaker 01

  Not to be into the languages.

• 5:10Lex Fridman?

  Languages.

• 5:11Speaker 01

  Yeah.

• 5:11Lex Fridman?

  So which mathematic discipline is most beautiful? Is geometry it for you?

• 5:16Speaker 01

  Both are beautiful. They have almost the same power.

• 5:19Lex Fridman?

  But there's a visual element to geometry being of

• 5:22Speaker 01

  Visual, it's more transparent. But once you get over to algebra, then a linear equation is a straight line. This translation is easily absorbed. And to pass a tangent to a circle, you know, you have the basic theorems and you can do it with algebra. So but the transition from one to another was really, I thought that Descartes was the greatest mathematician of all times.

• 5:52Lex Fridman?

  So you have been at the, if you think of engineering and mathematics as a spectrum.

• 6:00Speaker 01

  Yes.

• 6:01Lex Fridman?

  You have been, you have walked casually along this spectrum throughout your, throughout your life. You know, a little bit of engineering and then, you know, you have done a little bit of mathematics here and there.

• 6:15Speaker 01

  Not a little bit. I mean, we got a very solid background in mathematics because our teachers were geniuses. Our teachers came from Germany in the 1930s, running away from Hitler. They left their careers in Heidelberg and Berlin and came to teach high school in Israel. And we were the beneficiary of that experiment. So I, and they taught us math the good way.

• 6:42Lex Fridman?

  What's the good way to teach math?

• 6:44Speaker 01

  Chronologically.

• 6:46Lex Fridman?

  The people.

• 6:47Speaker 01

  The people behind the theorems, yeah. Their cousins and their nieces and their faces. And how they jumped from the bathtub when they scream eureka and ran naked in town.

• 7:03Lex Fridman?

  So you're almost educated as a historian of math.

• 7:06Speaker 01

  No, we just got a glimpse of that history together with the theorem. So every exercise in math was connected with a person.

• 7:17Speaker 01

  And the time of the person.

• 7:20Speaker 01

  The period.

• 7:20Lex Fridman?

  The period also mathematically.

• 7:22Speaker 01

  Mathematically speaking, yes, not the politics.

• 7:26Lex Fridman?

  So and then in in university, you have you have gone on to do engineering.

• 7:33Speaker 01

  Yeah, I get a BS in engineering in Technion. Right, and then I moved here for graduate work. And I got to I did engineering in addition to physics in Rutgers. And it would combine very nicely with my thesis, which I did in RCA Laboratories in Superconductivity.

• 7:57Lex Fridman?

  And then somehow thought to switch to almost computer science, software, even even not switch, but long to become to get into software engineering a little bit.

• 8:10Speaker 01

  Yes.

• 8:11Lex Fridman?

  Programming, if you can call it that in the 70s. So there's all these disciplines.

• 8:15Speaker 01

  Yeah.

• 8:15Lex Fridman?

  If you were to pick a favorite, what in terms of engineering and mathematics, which path do you think has more beauty? Which path has more power?

• 8:26Speaker 01

  It's hard to choose, no. I enjoy doing physics and even have a vortex named on my name. So I have investment in immortality.

• 8:40Lex Fridman?

  So what is a vortex?

• 8:42Speaker 01

  Vortex is in superconductivity.

• 8:44Lex Fridman?

  In the superconductivity.

• 8:45Speaker 01

  You have permanent current swirling around. One way or the other, you can have a store one or zero for computer. That's what we worked on in the 1916 RCA. And I discovered a few nice phenomena with the vortices.

• 9:01Lex Fridman?

  So there's a pearl vortex.

• 9:04Speaker 01

  Pearl vortex, right? You can Google it, right? I didn't know about it, but the physicist picked up on my thesis, on my PhD thesis, and it becomes popular. Thin-film superconductors became important for high-temperature superconductors. So they call it pearl vortex without my knowledge. I discovered it only about 15 years ago.

• 9:32Lex Fridman?

  You have footprints in all of the sciences. So let's talk about the universe a little bit. Is the universe at the lowest level deterministic or stochastic? In your amateur philosophy view, put another way, does God play dice?

• 9:46Speaker 01

  Well, we know it is stochastic, right?

• 9:50Lex Fridman?

  Today, today, we think it is stochastic.

• 9:52Speaker 01

  Yes. We think because we have the Heisenberg uncertainty principle, and we have some experiments to confirm that.

• 10:03Lex Fridman?

  All we have is experiments to confirm it. We don't understand why.

• 10:07Speaker 01

  Why is all...

• 10:08Lex Fridman?

  You wrote a book about why.

• 10:12Speaker 01

  Yeah, it's a puzzle. It's a puzzle that you have the dice-flipping machine, or god,

• 10:22Speaker 01

  and the result of the flipping propagated with a speed faster than the speed of light. We can't explain it, okay? So, but it only governs microscopic phenomena.

• 10:38Lex Fridman?

  So you don't think of quantum mechanics as useful for understanding the nature of reality?

• 10:45Speaker 01

  No, diversionary.

• 10:47Lex Fridman?

  So in your thinking, the world might as well be deterministic?

• 10:53Speaker 01

  The world is deterministic. And as far as the neuron firing is concerned, it's deterministic to first approximation.

• 11:04Lex Fridman?

  What about free will?

• 11:06Speaker 01

  Free will is also a nice exercise. Free will is an illusion that we AI people are going to solve.

• 11:16Lex Fridman?

  So what do you think, once we solve it, that solution will look like? Once we put it in a page?

• 11:21Speaker 01

  The solution will look like, first of all, it will look like a machine. A machine that acts as though it has free will. It communicates with other machines as though they have free will. And you wouldn't be able to tell the difference between a machine that does and a machine that doesn't have free will.

• 11:42Lex Fridman?

  So the illusion, it propagates the illusion of free will amongst the other machines?

• 11:46Speaker 01

  And faking it is having it. Okay, that's what Turing test is all about.

• 11:52Lex Fridman?

  Yeah.

• 11:52Speaker 01

  Faking intelligence is intelligent because it's not easy to fake. It's very hard to fake. And you can only fake if you have it.

• 12:05Lex Fridman?

  Yeah.

• 12:06Speaker 01

  So.

• 12:06Lex Fridman?

  That's such a beautiful statement.

• 12:10Lex Fridman?

  That's, yeah, you could, yeah.

• 12:13Speaker 01

  Yeah.

• 12:13Lex Fridman?

  You can't fake it if you don't have it. Yeah. So let's begin at the beginning with probability, both philosophically and mathematically. What does it mean to say the probability of something happening is 50%? What is probability?

• 12:34Speaker 01

  It's a degree of uncertainty that an agent has about the world.

• 12:39Lex Fridman?

  You're still expressing some knowledge in that statement.

• 12:42Speaker 01

  Of course. If the probability is 90%, it's absolutely a different kind of knowledge than if it is 10%.

• 12:49Lex Fridman?

  But it's still not solid knowledge? It's.

• 12:53Speaker 01

  It is solid knowledge, by. If you tell me that 90% assurance smoking will give you lung cancer in five years versus 10%, it's a piece of useful knowledge.

• 13:09Lex Fridman?

  So the statistical view of the universe.

• 13:13Speaker 01

  Yeah.

• 13:13Lex Fridman?

  Why is it useful? So we're swimming in complete uncertainty. Most of everything around us.

• 13:19Speaker 01

  It allows you to predict things with a certain probability. And computing those probabilities are very useful. That's the whole idea of prediction. And you need prediction to be able to survive. If you cannot predict the future, then you're just crossing the street. It will be extremely fearful.

• 13:43Lex Fridman?

  And so you've done a lot of work in causation. And so let's, let's think about correlation.

• 13:49Speaker 01

  I started with probability.

• 13:51Lex Fridman?

  You started with probability. You've invented the Bayesian networks.

• 13:56Speaker 01

  Yeah.

• 13:56Lex Fridman?

  And so, you know, we'll, we'll, we'll dance back and forth between these levels of uncertainty. But what is correlation? What is it? So probability of something happening is something. But then there's a bunch of things happening. And sometimes they happen together, sometimes not. They're independent or not. So how do you think about correlation of things?

• 14:21Speaker 01

  Correlation occurs when two things vary together over a very long time is one way of measuring it. Or when you have a bunch of variables that are very coincidentally,

• 14:33Speaker 01

  then we call we have a correlation here. And usually, when we think about correlation, we really think causally. Things cannot be correlated unless there is a reason for them to vary together. Why should they vary together? If they don't see each other, why should they vary together?

• 14:52Lex Fridman?

  So underlying it somewhere is causation.

• 14:55Speaker 01

  Yes. Hidden in our intuition, there is a notion of causation because we cannot grasp any other logic except causation.

• 15:06Lex Fridman?

  And how does conditional probability differ from causation?

• 15:13Lex Fridman?

  So what is conditional probability?

• 15:15Speaker 01

  Conditional probability, how things vary when one of them stays the same. Now, staying the same means that I have chosen to look only at those incidents where the guy has the same value as previous one. It's my choice as an experimenter. So things that are not correlated before could become correlated. Like, for instance, if I have two coins, which are uncorrelated, and I choose only those flippings experiments in which a bell rings, and a bell rings when at least one of them is a tail.

• 15:58Speaker 01

  Then suddenly, I see correlation between the two coins. Because I only look at the cases where the bell rang. You see, it's my design, with my ignorance essentially, with my audacity to ignore certain incidents, I suddenly create a correlation where it doesn't exist physically.

• 16:25Lex Fridman?

  Right, so that's, you just outlined one of the flaws of observing the world and trying to infer something fundamental about the world from looking at the correlation.

• 16:34Speaker 01

  I don't look at it as a flaw, the world works like that. But the flaws comes if we try to impose causal logic on correlation, it doesn't work too well.

• 16:52Lex Fridman?

  I mean, but that's exactly what we do, that's what, that has been the majority of science.

• 16:57Speaker 01

  Majority of naive science. Statisticians know it, statisticians know it. If you condition on a third variable, then you can destroy or create correlations among two other variables.

• 17:12Lex Fridman?

  Right.

• 17:13Speaker 01

  They know it, it's in the data.

• 17:15Lex Fridman?

  Right.

• 17:15Speaker 01

  Nothing is surprising, that's why they all dismiss the Simpson paradox. Ah, we know it, they don't know anything about it.

• 17:24Lex Fridman?

  Well, there's disciplines like psychology where all the variables are hard to account for. And so oftentimes, there's a leap between correlation to causation. You're imposing a leap.

• 17:38Speaker 01

  Who is trying to get causation from correlation?

• 17:43Lex Fridman?

  Not, you're not proving causation, but you're sort of discussing it, implying, sort of hypothesizing with all ability.

• 17:52Speaker 01

  Which discipline you have in mind, I'll tell you if they are obsolete, or if they are outdated, or they are about to get outdated.

• 18:01Lex Fridman?

  Yes, yes.

• 18:02Speaker 01

  Tell me which one you have in mind.

• 18:04Lex Fridman?

  Oh, psychology, you know.

• 18:05Speaker 01

  Psychology, what, is it SEM, structural equation?

• 18:08Lex Fridman?

  No, no, I was thinking of a psychology studying. For example, we work with human behavior in semi-autonomous vehicles, how people behave, and you have to conduct these studies of people driving cars.

• 18:21Speaker 01

  Everything starts with a question, what is the research question?

• 18:25Lex Fridman?

  What is the research question? The research question, do people fall asleep when the car is driving itself?

• 18:36Speaker 01

  Do they fall asleep or do they tend to fall asleep more frequently?

• 18:40Lex Fridman?

  More frequently.

• 18:41Speaker 01

  Than with a car not driving?

• 18:43Lex Fridman?

  Nah, it's not driving itself.

• 18:44Speaker 01

  That's a good question, okay?

• 18:46Lex Fridman?

  And so you measure, you put people in the car, because it's real world. You can't conduct an experiment where you control everything.

• 18:53Speaker 01

  Why can't you con-

• 18:55Lex Fridman?

  You could.

• 18:55Speaker 01

  Turn the autom- automatic module on and off?

• 19:02Lex Fridman?

  Because it's on road public, I mean, there's, you have, it's, there's aspects to it that's unethical. Because it's testing on public roads. So you can only use vehicle, you, they have to, the people, the drivers themselves have to make that choice themselves. And so they regulate that. And so you just observe when they drive it autonomously and when they don't. And then-

• 19:28Speaker 01

  But maybe they turn it off when they were very tired.

• 19:30Lex Fridman?

  Yeah, that's kind of thing. But you, you don't know those variables.

• 19:34Speaker 01

  So that you have now uncontrolled experiment.

• 19:36Lex Fridman?

  Uncontrolled experiment.

• 19:37Speaker 01

  We, we call it observational study.

• 19:39Lex Fridman?

  Yeah.

• 19:40Speaker 01

  And we form the correlation detected, we have to infer causal relationship. Whether it was the automatic piece that caused them to fall asleep or, okay. So that is an issue that is about a hundred and twenty years old.

• 20:01Lex Fridman?

  Yeah.

• 20:02Speaker 01

  I should only go a hundred years old, okay? And, um, oh, maybe it's not, actually, I should say it's two thousand years old. Because we have this experiment by Daniel about the Babylonian king that wanted the exile, the people from Israel that were taken in exile to Babylon to serve the king. He wanted to serve them king's food, which was meat. And Daniel, as a good Jew, couldn't eat a non-cozzy food. So he asked them to eat vegetarian food.

• 20:44Speaker 01

  But the king overseer says, I'm sorry, but if the king sees that your performance falls below that of other kids, you know, he's going to kill me. Daniel said, let's make an experiment. Let's take four of us from Jerusalem, okay? Give us vegetarian food. Let's take the other guys to eat the king's food. In about a week's time, we'll test our performance. And you know the answer. Of course, he did the experiment. And they were so much better than the others. And the kings nominated them to superposition in his king. So it was a first experiment, yes.

• 21:27Speaker 01

  So there was a very simple, it's also the same research question. We want to know if vegetarian food assist or obstruct your mental ability. And, okay, so that was the question is very old one. Even Democritus said, if I could discover one cause of things, I would rather discover one cause than be a king of Persia. They, task of discovering causes was in the mind of ancient people from many, many years ago.

• 22:10Speaker 01

  But the mathematics of doing that was only developed in the 1920s. So science has left us orphaned. Okay, science has not provided us with the mathematics to capture the idea of X causes Y and Y does not cause X. Because all the question of physics are symmetrical, algebraic. The equality sign goes both ways.

• 22:39Lex Fridman?

  Okay, let's look at machine learning. Machine learning today, if you look at deep neural networks, you can think of it as a kind of conditional probability estimators.

• 22:51Speaker 01

  Correct, beautiful.

• 22:52Lex Fridman?

  So.

• 22:53Speaker 01

  Where did you say that?

• 22:56Lex Fridman?

  What?

• 22:56Speaker 01

  Conditional probability estimators. None of the machine learning people clevered you? Attack you?

• 23:04Lex Fridman?

  Listen, most people, and this is why this today's conversation, I think, is interesting is most people would agree with you. There's certain aspects that are just effective today. But we're going to hit a wall and there's a lot of ideas. I think you're very right that we're gonna have to return to about causality. And that it would be, let's try to explore it. Let's, let's, let's even take a step back. You've invented Bayesian networks.

• 23:34Lex Fridman?

  That look awfully a lot like they express something like causation, but they don't. Not necessarily. So how do we turn Bayesian networks into expressing causation? How do we build causal networks? This A causes B, B causes C. How do we start to infer that kind of thing?

• 23:56Speaker 01

  We start asking ourselves question. What are the factors that would determine the value of X? X could be blood pressure, death, hungry, hunger.

• 24:11Lex Fridman?

  But these are hypotheses that we propose.

• 24:13Speaker 01

  Hypotheses, everything which has to do with causality comes from a theory.

• 24:20Speaker 01

  The difference is only what kind, how you interrogate the theory that you have in your mind.

• 24:28Lex Fridman?

  So it still needs the human expert to propose.

• 24:31Speaker 01

  Right, you need the human expert to specify the initial model. Initial model could be very qualitative. Just who listens to whom? By whom listen to, I mean, one variable listen to the other. So I say, okay, the tide is listening to the moon. And not to the rooster crow. And so forth. This is our understanding of the world in which we live. Scientific understanding of reality. We have to start there. Because if we don't know how to handle

• 25:14Speaker 01

  cause and effect relationship, when we do have a model, and we certainly do not know how to handle it when we don't have a model. So let's start first. In AI slogan is representation first, discovery second. But if I give you all the information that you need, can you do anything useful with it? That is the first, representation. How do you represent it? I give you all the knowledge in the world, how do you represent it? When you represent it, I ask you, can you infer X or Y or Z? Can you answer certain queries? Is it complex? Is it polynomial?

• 25:56Speaker 01

  All the computer science exercises we do, once you give me a representation for my knowledge. Then you can ask me, now I understand how to represent things, how do I discover them? It's a secondary thing.

• 26:12Lex Fridman?

  So first of all, I should echo the statement that mathematics and the current much of the machine learning world has not considered causation, that A causes B. Just in anything, that seems like a

• 26:29Lex Fridman?

  that seems like a non-obvious thing, that you think we would have really acknowledged it, but we haven't. So we have to put that on the table. So, knowledge. How hard is it to create a knowledge from which to work?

• 26:45Speaker 01

  In certain area, it's easy, because we have only four or five major variables. And an epidemiologist or an economist can put them down.

• 27:00Speaker 01

  What, minimum wage, unemployment, policy, X, Y, Z, and start collecting data and quantify the parameters that were left unquantified with the initial knowledge. That's the

• 27:21Speaker 01

  Routine work that you find in experimental psychology.

• 27:27Lex Fridman?

  Yes.

• 27:27Speaker 01

  In economics, everywhere, in the health science. That's a routine thing. But I should emphasize, you should start with a research question. What do you want to estimate? Once you have that, you have to have a language of expressing what you want to estimate. You think it's easy? No.

• 27:50Lex Fridman?

  So we can talk about two things, I think. One is how the science of causation is very useful for answering certain questions. And then the other is, how do we create intelligent systems that need to reason with causation? So if my research question is, how do I pick up this water bottle from the table? All the knowledge that is required to be able to do that. How do we construct that knowledge base? Does it, do we return back to the problem that we didn't solve in the eighties with expert systems? Do we have to solve that problem of automated construction of knowledge?

• 28:35Speaker 01

  Hmm. You're talking about the task of eliciting knowledge from an expert.

• 28:44Lex Fridman?

  Task of eliciting knowledge from an expert, or the self-discovery of more knowledge, more and more knowledge. So automating the building of knowledge as much as possible.

• 28:56Speaker 01

  It's a different game in the causal domain. Because it's essentially the same thing. You have to start with some knowledge and you're trying to enrich it. But you don't enrich it by asking for more rules. You enrich it by asking for the data, to look at the data and quantifying and ask queries that you couldn't answer when you started. You couldn't because the question is...

• 29:28Speaker 01

  Quite complex and it's not within the capability of ordinary cognition, of ordinary person, ordinary expert even, to answer.

• 29:40Lex Fridman?

  So what kind of questions do you think we can start to answer?

• 29:44Speaker 01

  Even simple, suppose, yeah, I start with easy one.

• 29:48Lex Fridman?

  Let's do it.

• 29:49Speaker 01

  Okay, what's the effect of a drug on recovery? What did aspirin that caused my headache to be cured? Or what did the television program or the good news I received? This is already, you see, it's a difficult question because it's find the cause from effect. The easy one is find the effect from cause.

• 30:14Lex Fridman?

  That's right. So first you construct a model saying that this is an important research question. This is an important question. Then you do.

• 30:21Speaker 01

  I didn't construct a model yet. I just said it's important question.

• 30:24Lex Fridman?

  Important question.

• 30:25Speaker 01

  And the first exercise is express it mathematically. What do you want to like? If I tell you what will be the effect of taking this drug? You have to say that in mathematics. How do you say that?

• 30:40Lex Fridman?

  Yes.

• 30:40Speaker 01

  Can you write down the question, not the answer? I want to find the effect of a drug on my headache.

• 30:49Lex Fridman?

  Right.

• 30:50Speaker 01

  Write down, write it down.

• 30:52Lex Fridman?

  That's where the do calculus comes in.

• 30:53Speaker 01

  Yes, do operator, what do you do operator?

• 30:55Lex Fridman?

  Do operator, yeah.

• 30:56Speaker 01

  Yeah.

• 30:57Lex Fridman?

  This is nice. It's the difference in association and intervention. Very beautifully sort of constructed.

• 31:03Speaker 01

  Yeah, so we have a do operator. So the do calculus connected on the do operator itself connects the operation of doing to something that we can see.

• 31:15Lex Fridman?

  Right. So as opposed to the purely observing, you're making the choice to change a variable.

• 31:23Speaker 01

  That's what it expresses. And then the way that we interpret it, the mechanism by which we take your query and we translate into something that we can work with is by giving it semantics, saying that you have a model of the world and you cut off all the incoming arrow into X. And you're looking now in the modified mutilated model, you ask for the probability of Y. That is interpretation of doing X. Because by doing things, you've liberated them from all influences that acted upon them earlier. And you subject them to the tyranny of your muscles.

• 32:07Lex Fridman?

  So you remove all the questions about causality by doing them.

• 32:13Speaker 01

  So there's one level of questions. Answer questions about what will happen if you do things.

• 32:19Lex Fridman?

  If you do, if you drink the coffee, if you take the aspirin. So how do we get the, how do we get the doing data?

• 32:28Speaker 01

  Now, the question is, if we cannot run experiments, right, then we have to rely on observational study.

• 32:38Lex Fridman?

  So first, we could, sorry to interrupt, we could run an experiment. Where we do something, where we drink the coffee and don't. And this, the do operator allows you to sort of be systematic about expressing.

• 32:49Speaker 01

  To imagine how the experiment will look like, even though we cannot physically and technologically conduct it. I'll give you an example. What is the effect of blood pressure on mortality? I cannot go down into your vein and change your blood pressure. But I can ask the question. Which means I can, if I have a model of your body, I can imagine the effect of your, how the blood pressure change will affect your mortality. How? I go into the model and I conduct this surgery about the blood pressure, even though physically I can do, I cannot do it.

• 33:34Lex Fridman?

  Let me ask the quantum mechanics question. Does the doing change the observation? Meaning, the surgery of changing the blood pressure is, I mean.

• 33:46Speaker 01

  No, the surgery is called very delicate.

• 33:52Lex Fridman?

  It's very delicate, infinitely delicate.

• 33:55Speaker 01

  Incisive and delicate, which mean, do means, do X means, I'm gonna touch only X.

• 34:03Lex Fridman?

  Only X.

• 34:04Speaker 01

  Directly into X.

• 34:07Speaker 01

  So that means that I change only things which depends on X by virtue of X changing. But I don't depend things which are not depend on X. Like I wouldn't change your sex or your age, I just change your blood pressure.

• 34:24Lex Fridman?

  So in the case of blood pressure, it may be difficult or impossible to construct such an experiment.

• 34:30Speaker 01

  No, physically, yes, but hypothetically, no.

• 34:33Lex Fridman?

  Hypothetically, no.

• 34:34Speaker 01

  If we have a model, that is what the model is for. So you conduct surgeries on a model, you take it apart, put it back. That's the idea of a model. The idea of thinking counterfactively, imagining, and that's the idea of creativity.

• 34:52Lex Fridman?

  So by constructing that model, you can start to infer if the higher the blood pressure leads to mortality, which increases or decreases by.

• 35:04Speaker 01

  I construct a model, I can still cannot answer it. I have to see if I have enough information in the model that would allow me to find out the effects of intervention from a non-interventional study. From an hands-off study.

• 35:21Lex Fridman?

  So what's needed to make that?

• 35:23Speaker 01

  You need to have assumptions about who affects whom. If the if the graph had a certain property, the answer is yes, you can get it from observational study. If the graph is too meshy, bushy, bushy, the answer is no, you cannot. Then you need to find either different kind of observation that you haven't considered or one experiment.

• 35:52Lex Fridman?

  So basically, does that that puts a lot of pressure on you to encode wisdom into that graph?

• 35:59Speaker 01

  Correct. But you don't have to encode more than what you know. God forbid, if you put, like economists are doing this. They call identifying assumptions. They put assumptions, even if they don't prevail in the world. They put assumptions so they can identify things.

• 36:16Lex Fridman?

  But the problem is, yes, beautifully put. But the problem is you don't know what you don't know. So.

• 36:23Speaker 01

  You know what you don't know. Because if you don't know, you say it's possible. It's possible that X affect the traffic tomorrow. And it's possible. You put down an arrow which says it's possible. Every arrow in the graph says it's possible.

• 36:41Lex Fridman?

  So there's not a significant cost to adding arrows that.

• 36:45Speaker 01

  The more arrow you add.

• 36:47Lex Fridman?

  The better.

• 36:48Speaker 01

  The less likely you are to identify things from purely observational data.

• 36:55Speaker 01

  So if the whole world is bushy.

• 36:58Speaker 01

  And everybody affect everybody else. The answer is, you can answer it ahead of time. I cannot answer my query from observational data. I have to go to experiments.

• 37:14Lex Fridman?

  So you talk about machine learning is essentially learning by association or reasoning by association. And this due calculus is allowing for intervention. I like that word.

• 37:25Speaker 01

  Intervention.

• 37:26Lex Fridman?

  Action. So you also talk about counterfactuals.

• 37:29Speaker 01

  Yeah.

• 37:30Lex Fridman?

  And trying to sort of understand the difference in counterfactuals and intervention.

• 37:37Lex Fridman?

  What's the, first off, what is counterfactuals? And why are they useful? Why are they especially useful as opposed to just reasoning what, what effect actions have?

• 37:52Speaker 01

  But counterfactual contains what we normally call explanations.

• 37:57Lex Fridman?

  Can you give an example?

• 37:58Speaker 01

  If I tell you that acting one way affects something else. I didn't explain anything yet. But if I, if I ask you, was it the aspirin that cured my headache? I'm asking for explanation, what cured my headache? And putting a finger on aspirin.

• 38:20Speaker 01

  Provide explanation. It was aspirin. It was responsible for your headache going away. If you didn't take the aspirin, you will still have a headache.

• 38:33Lex Fridman?

  So by saying, if I didn't take aspirin, I would have a headache. You're thereby saying that aspirin is the thing that removes the headache.

• 38:43Speaker 01

  Yes, but you have to have another important information. I took the aspirin and my headache is gone. It's very important information. Now I'm reasoning backward and I said, what is the aspirin?

• 38:57Lex Fridman?

  Yeah. By considering what would have happened if everything else is the same, but I didn't take aspirin.

• 39:04Speaker 01

  That's right. So you know that things took place, you know? Joe killed Schmo. And Schmo would be alive had John not used his gun. Okay, so that is the counterfactual. It had a conflict here or clash between observed fact,

• 39:28Speaker 01

  that he did shoot, okay? And the hypothetical predicate, which says, had he not shot? You have a clash, a logical clash. They cannot exist together. That's a counterfactual. And that is the source of our explanation of our idea of responsibility, regret and free will.

• 39:52Lex Fridman?

  Yes, it certainly seems, that's the highest level of reasoning, right?

• 39:57Speaker 01

  Yes, and physicists do it all the time.

• 39:59Lex Fridman?

  Who does it all the time?

• 40:00Speaker 01

  Physicists.

• 40:00Lex Fridman?

  Physicists.

• 40:02Speaker 01

  In every equation of physics. Let's say you have a Hooke's law. And you put one kilogram on the spring and the spring is one meter. And you say, had this weight been two kilogram, the spring would have been twice as long.

• 40:19Speaker 01

  It's no problem for physicists to say that. Except that mathematics is only is in a form of equation. Equating the weight, proportionality constant, and the length of the string. So you don't have the asymmetry in the equation of physics, although every physicist thinks counterfactual. Ask the high school kids. Had the weight been three kilograms, what would be the length of the spring? They can answer it immediately. Because they do the counterfactual processing in their mind, and then they put it into equation, algebraic equation, and they solve it. But a robot cannot do that.

• 41:04Lex Fridman?

  How do you make a robot learn these relationships?

• 41:08Speaker 01

  And why you would learn? Suppose you tell him, can you do it? So before you go learning, you have to ask yourself, suppose I give him all the information. Can the robot perform the task that I ask him to perform? Can he reasonably say, no, it wasn't the aspirin. It was the good news you received on the phone.

• 41:32Lex Fridman?

  Right, because, well, unless the robot had a model, a causal model of the world.

• 41:41Speaker 01

  Right, right.

• 41:41Lex Fridman?

  I'm sorry, I have to linger on this.

• 41:43Speaker 01

  But now we have to linger on, we have to say, how do we do it?

• 41:46Lex Fridman?

  How do we build it?

• 41:47Speaker 01

  Yes.

• 41:47Lex Fridman?

  How do we build a causal model without a team of human experts running around?

• 41:54Speaker 01

  Why don't you go to learning right away? You're too much involved with learning.

• 41:58Lex Fridman?

  Because I like babies, babies learn fast. I'm trying to figure out how they do it.

• 42:02Speaker 01

  Good. That's another question. How do the babies come out with the counterfactual model of the world? And babies do that. They know how to play with, in the crib. They know which balls hits another one. And they learn it by playful manipulation.

• 42:23Speaker 01

  Of the world.

• 42:24Lex Fridman?

  Yes.

• 42:24Speaker 01

  A simple world involve only toys and balls and chimes.

• 42:31Speaker 01

  But it's a, if you think about it, a complex world.

• 42:34Lex Fridman?

  We take for granted.

• 42:35Speaker 01

  Yes.

• 42:36Lex Fridman?

  Uh, how complex.

• 42:37Speaker 01

  And kids do it by playful manipulation plus parent guidance, pure wisdom.

• 42:45Lex Fridman?

  Yeah.

• 42:46Speaker 01

  And hearsay. They meet each other and they say, you shouldn't have taken my toy.

• 42:56Lex Fridman?

  Right. But and they, these multiple sources of information, they're able to integrate.

• 43:02Speaker 01

  Yeah.

• 43:03Lex Fridman?

  So the challenge is about how to integrate, how to form these causal relationship from different sources of data.

• 43:11Speaker 01

  Correct.

• 43:12Lex Fridman?

  So how, how, how much is information is it to play? How much causal information is required to be able to play in the crib with different objects?

• 43:24Speaker 01

  I, I don't know. I haven't experimented with the crib.

• 43:28Lex Fridman?

  Okay, not a crib.

• 43:30Speaker 01

  I don't know. It's a very interesting.

• 43:31Lex Fridman?

  Manipulating physical objects on this very, opening the pages of a book, all the tasks, the physical manipulation tasks. Do you have a sense? Because my sense is the world is extremely complicated.

• 43:45Speaker 01

  Extremely complicated. I agree. And I don't know how to organize it because I've been spoiled by easy problems such as cancer and death.

• 43:57Lex Fridman?

  First, we have to start trying to.

• 43:59Speaker 01

  Easy, there is in a sense that you have only 20 variables. And they are just variables and not mechanics. Okay, it's easy. You just put them on a graph and they, they, they speak to you.

• 44:15Lex Fridman?

  Yeah. And you, you're providing a methodology for letting them speak. Yeah.

• 44:20Speaker 01

  I'm working only in the abstract. The abstract is knowledge in, knowledge out, data in between.

• 44:29Lex Fridman?

  Now, can we take a leap to trying to learn in this very, when it's not 20 variables, but 20 million variables, trying to learn causation in this world? Not learn, but somehow construct models? I mean, it seems like you would only have to be able to learn, because constructing it manually would be too difficult. Do you have ideas of?

• 44:55Speaker 01

  I think it's a matter of combining simple models from many, many sources, from many, many disciplines, and many metaphors. Metaphors are the basics of human intelligence, the basis.

• 45:09Lex Fridman?

  Yeah, so how do you think about a metaphor? In terms of its use in human intelligence?

• 45:13Speaker 01

  Metaphors is an expert system. An expert, it's mapping problem with which you are not familiar to a problem with which you are familiar. Like, I'll give you a good example. The Greek believed that the sky is an opaque shell. It's not really a infinite space. It's an opaque shell, and the stars are holes poked in the shells through which you see the eternal light. It was a metaphor. Why?

• 45:54Speaker 01

  Because they understand how you poke holes in shells. They are not familiar with infinite space. And we are walking on a shell of a turtle. And if you get too close to the edge, you're going to fall down to Hades or wherever. Yeah? That's a metaphor. It's not true. But this kind of metaphor enabled Aristoteneus to measure the radius of the earth. Because he said, "Come on. If we are walking on a turtle shell, then a ray of light coming to this angle will be different.

• 46:37Speaker 01

  This place will be a different angle than coming to this place. I know the distance, I'll measure the two angles. And then I have the radius of the shell of the turtle." And he did, and he found his... Measurement very close to the measurements we have today. It's through the, what, six thousand and seven hundred, seven hundred kilometers. That's something that would not occur to Babylonian astronomer. Even though the Babylonian experiments were the machine learning people of the time.

• 47:22Speaker 01

  They fit curves and they could predict the eclipse of the moon much more accurately than the Greek. Because they fit curve. Okay. That's a different metaphor. Something that you're familiar with, a game, a turtle shell. Okay. What does it mean if you are familiar? Familiar means that answers to certain questions are explicit. You don't have to derive them.

• 47:53Lex Fridman?

  And they were made explicit because somewhere in the past, you've constructed a model of that.

• 48:00Speaker 01

  You're familiar with, so the child is familiar with billiard balls.

• 48:05Lex Fridman?

  Yes.

• 48:05Speaker 01

  So the child could predict that if you let loose of one ball, the other one will bounce off. These are, you obtain that by familiarity. Familiarity is answering questions and you store the answer explicitly. You don't have to derive them. So this is the idea of a metaphor. All our life, all our intelligence is built around metaphors. Mapping from the unfamiliar to the familiar. But the marriage between the two is a tough thing, which I, which we haven't yet been able to algorithmatize.

• 48:42Lex Fridman?

  So you think of that process of, of using metaphor to leap from one place to another, we can call it reasoning? Is it a kind of reasoning?

• 48:53Speaker 01

  It is a reasoning by metaphor, metaphorical reasoning.

• 48:55Lex Fridman?

  Reasoning by metaphor. Do you think of that as learning?

• 49:01Lex Fridman?

  So, learning is a popular terminology today in a narrow sense.

• 49:05Speaker 01

  It is, it is, it is definitely a form.

• 49:07Lex Fridman?

  So you may not, okay, right.

• 49:09Speaker 01

  It's one of the most important learning. Taking something which theoretically is derivable and store it in accessible format. I'll give you an example, chess, okay?

• 49:25Speaker 01

  Finding, winning, winning starting move in chess is hard. But, uh, it, it, there is an answer. Either there is a winning move for white, or there isn't, or there is a draw. Okay, so it is, the, the answer to that is available through the rule of the game. But we don't know the answer. So what does a chess master have that we don't have? He has stored explicitly an evaluation of certain complex pattern of the board. We don't have it, ordinary people like me, I don't know about you. I'm not a chess master.

• 50:10Speaker 01

  So for me, I have to derive.

• 50:12Lex Fridman?

  Yes.

• 50:13Speaker 01

  Things that for him is explicit. He has seen it before, or he has seen the pattern before, or similar pattern, or similar for. Yeah, and he generalized and said, "Don't move, it's a dangerous move."

• 50:30Lex Fridman?

  Uh, it's just that, not in the game of chess, but in the game of billiard balls, we humans are able to initially derive very effectively and then reason by metaphor very effectively. And we make it look so easy that it makes one wonder, how hard is it to build it in a machine? So, in, in your sense, uh, how far away are we to be able to construct?

• 50:58Speaker 01

  I don't know, I'm not a futurist. All I can tell you is that we are making tremendous progress in the causal reasoning domain. Something that I even...

• 51:13Speaker 01

  Dare to call it revolution, the code of revolution. Because what we have achieved in the past three decades is something that dwarfed everything that was derived in the entire history.

• 51:32Lex Fridman?

  So there's an excitement about current machine learning methodologies. And there's really important, good work you're doing in causal inference. Where do the... What is the future? Where do these worlds collide? And what does that look like?

• 51:52Speaker 01

  First, they're going to work without collisions. It's going to work in harmony.

• 51:58Lex Fridman?

  Harmony, it's not collision.

• 51:59Speaker 01

  The human is going to to jumpstart the exercise by providing qualitative, non-committing models of how the universe works. Universe, I say we have. How in reality, the domain of discourse works. The machine is going to take over from that point of view and derive whatever the calculus says can be derived. Namely, quantitative answer to our questions. These are complex questions. I'll give you some example of complex questions that would boggle your mind if you think about it.

• 52:44Speaker 01

  You take result of studies in diverse population under diverse condition. And you infer the cause effect of a new population, which doesn't even resemble any of the ones studied. And you do that by do calculus. You do that by generalizing from one study to another. See, what's common with it too? What is different? Let's ignore the differences and pull out the commonality. And you do it over maybe 100 hospitals around the world. From that, you can get really mileage from big data.

• 53:30Speaker 01

  It's not only you have many samples. You have many sources of data.

• 53:36Lex Fridman?

  So that's a really powerful thing, I think, for, especially for medical applications, I mean, cure cancer, right? That's how, from data, you can cure cancer. So we're talking about causation, which is the temporal, temporal relationships between things.

• 53:52Speaker 01

  Not only temporal. It was structural and temporal. Temporal enough, temporal precedence by itself cannot replace causation.

• 54:02Lex Fridman?

  Is temporal precedence the arrow of time in physics?

• 54:08Speaker 01

  It's important, necessary.

• 54:09Lex Fridman?

  It's important.

• 54:10Speaker 01

  But it's efficient, yes.

• 54:11Lex Fridman?

  Is it?

• 54:12Speaker 01

  Yes. I never see the cause propagate backward.

• 54:16Lex Fridman?

  But if we call, if we use the word cause, but there's relationships that are timeless. I suppose that's still forward in the arrow of time. But the, are there relationships, logical relationships, that fit into the structure?

• 54:34Speaker 01

  Sure. The whole, do calculus is logical relationship.

• 54:39Lex Fridman?

  That doesn't require temporal. It has just the condition that it, it's, you're not traveling back in time.

• 54:46Speaker 01

  Yes. Correct.

• 54:48Lex Fridman?

  So it's really a generalization of, a powerful generalization of what-

• 54:57Speaker 01

  Of Boolean logic.

• 54:58Lex Fridman?

  Yeah, Boolean logic.

• 54:59Speaker 01

  Yes.

• 55:00Lex Fridman?

  That is sort of simply put, it allows us to, you know, reason, reason about the order of events, the source, the-

• 55:12Speaker 01

  Not about, which means we're not deriving the order of events. We are given cause-effect relationship, okay? They ought to be-

• 55:21Speaker 01

  Obeying the time president's relationship. We are given that. And now that we ask questions about other causal relationship that could be derived from the initial ones, but were not given to us explicitly.

• 55:37Lex Fridman?

  Yeah.

• 55:38Speaker 01

  Like the case of the firing squad I gave you in the first chapter. And I ask, what if rifleman A declined to shoot? Would the prisoner still be dead? To decline to shoot, it means that he disobey order. And the rule of the games were that he is an obedient marksman. That's how you start, that's the initial order. But now you ask question about breaking the rules. What if he decided not to pull the trigger? He just became a pacifist. And you can, you and I can answer that.

• 56:23Speaker 01

  The other rifleman would have killed him, okay? I want a machine to do that. Is it so hard to ask a machine to do that? It's such a simple task.

• 56:34Lex Fridman?

  No.

• 56:35Speaker 01

  But they have to have a calculus for that.

• 56:36Lex Fridman?

  Yes.

• 56:37Speaker 01

  Yes.

• 56:38Lex Fridman?

  But the curiosity, the natural curiosity for me is that, yes, you're absolutely correct and important. And it's hard to believe that we haven't done this seriously, extensively, already a long time ago. So this is really important work. But I also want to know, you know, this, maybe you can philosophize about how hard is it to learn?

• 57:00Speaker 01

  Okay, let's assume we're learning. We want to learn it, okay?

• 57:03Lex Fridman?

  Want to learn.

• 57:03Speaker 01

  So what do we do? We put a learning machine that watches execution trials in many countries and many locations, okay? All the machine can learn is to see shut or not shut, dead, not dead, court issued an order or didn't. Okay, that's the fact. From the fact, you don't know who listens to whom. You don't know that they... Condemned person, listen to the bullets. And the bullets are listening to the captain, okay? All we hear is one command, two shots, dead, okay? A triple of variables, yes, no, yes, no.

• 57:46Speaker 01

  Okay, from that you can learn who listens to whom. And you can answer the question, no.

• 57:51Lex Fridman?

  Definitively no, but don't you think you can start proposing ideas for humans to review?

• 57:59Speaker 01

  You want machine to learn, right? You want a robot. So a robot is watching trials like that.

• 58:06Lex Fridman?

  Yep.

• 58:06Speaker 01

  Two hundred trials. And then he has to answer the question, what if rifleman A refrain from shooting?

• 58:14Lex Fridman?

  Yeah.

• 58:16Lex Fridman?

  So how to do that?

• 58:19Speaker 01

  That's exactly my point. That looking at the facts don't give you the strings behind the facts.

• 58:24Lex Fridman?

  Absolutely. But do you think of machine learning as is currently defined? As only something that looks at the facts and tries to do?

• 58:35Speaker 01

  Right now, they only look at the facts, yeah.

• 58:36Lex Fridman?

  So is there a way to modify?

• 58:39Speaker 01

  Yeah.

• 58:39Lex Fridman?

  In your sense?

• 58:40Speaker 01

  Playful manipulation.

• 58:42Lex Fridman?

  Playful manipulation?

• 58:43Speaker 01

  Yes.

• 58:43Lex Fridman?

  Doing the interventionist kind of thing?

• 58:46Speaker 01

  Yes.

• 58:46Lex Fridman?

  Intervention.

• 58:47Speaker 01

  But it could be at random. For instance, the rifleman is sick that day. Or he just vomits or whatever. So machine can observe this unexpected event, which introduce noise. The noise still have to be random to be able to relate it to randomized experiment. And then you have observational studies from which to infer the strings behind the facts. It's doable to a certain extent. But now that we are expert in what you can do, once you have a model, we can reason back and say, what do you kind of data you need to build a model?

• 59:30Lex Fridman?

  Got it. So I know you're not a futurist, but are you excited? Have you, when you look back at your life, long for the idea of creating a human level intelligence system?

• 59:43Speaker 01

  Yeah, I'm driven by that. All my life, I'm driven just by one thing.

• 59:51Speaker 01

  But I go slowly. I go from what I know to the next step incrementally.

• 59:56Lex Fridman?

  So without imagining what the end goal looks like, do you imagine what?

• 1:00:01Speaker 01

  And the end goal is gonna be a machine that can answer sophisticated questions, counterfactuals of regret, compassion, responsibility, and free will.

• 1:00:16Lex Fridman?

  So what is a good test? Is a Turing test a reasonable test?

• 1:00:22Speaker 01

  A test of free will doesn't exist yet.

• 1:00:25Lex Fridman?

  How would you test free will?

• 1:00:28Speaker 01

  So far, we know only one thing.

• 1:00:32Speaker 01

  If robots can communicate with reward and punishment among themselves,

• 1:00:41Speaker 01

  hitting each other on the wrist and say, "You shouldn't have done that."

• 1:00:47Speaker 01

  Playing better soccer because they can do that.

• 1:00:51Lex Fridman?

  What do you mean because they can do that?

• 1:00:53Speaker 01

  Because they can communicate among themselves.

• 1:00:55Lex Fridman?

  Because of the communication, they can do this.

• 1:00:57Speaker 01

  Because they communicate, like us, reward and punishment. Yes, you didn't pass the ball the right time and so forth. Therefore, you're gonna sit on the bench for the next two. If they start communicating like that, the question is, will they play better soccer? As opposed to what? As opposed to what they do now? Without this ability to reason about reward and punishment, responsibility.

• 1:01:23Lex Fridman?

  And counterfactuals?

• 1:01:26Speaker 01

  So far, I can only think about communication.

• 1:01:29Lex Fridman?

  Communication is, and not necessarily natural language, but just communication.

• 1:01:33Speaker 01

  Yeah, just communication. And that's important to have a quick and effective means of communicating knowledge. If the coach tells you, "You should have passed the ball," ping. He conveys so much knowledge to you, as opposed to what? Go down and change your software. That's the alternative. But the coach doesn't know your software. So how can a coach tell you, "You should have passed the ball"? But that, our language is very effective. "You should have passed the ball." You know your software, you tweak the right module, okay? And next time, you don't do it.

• 1:02:09Lex Fridman?

  Now, that's for playing soccer, where the rules are well-defined.

• 1:02:12Speaker 01

  No, no, no, well, they're not well-defined. When you should pass the ball.

• 1:02:16Lex Fridman?

  It's not well-defined.

• 1:02:17Speaker 01

  No, it's a, it's very soft, very noisy.

• 1:02:21Lex Fridman?

  Yeah.

• 1:02:22Speaker 01

  You have to do it under pressure.

• 1:02:24Lex Fridman?

  It's art. But in terms of aligning values between computers and humans, do you think this cause and effect type of thinking is important to align the values? Values, morals, ethics, under which the machines make decisions? Is the cause-effect where the two can come together?

• 1:02:49Speaker 01

  Cause-effect is a necessary component to build an ethical machine. Because the machine has to empathize, to understand what's good for you, to build a model of you as a recipient, which should be very much... What is compassion? They imagine that you suffer pain as much as me.

• 1:03:13Lex Fridman?

  As much as me.

• 1:03:14Speaker 01

  I do have already a model of myself, right? So it's very easy for me to map you to mine. I don't have to rebuild a model. It's much easier to say, "Oh, you're like me. Okay, therefore, I would not hate you."

• 1:03:27Lex Fridman?

  And the machine has to imagine, has to try to fake to be human, essentially, so you can imagine that you're, that you're like me, right?

• 1:03:36Speaker 01

  Moreover, who is me? That's the first, that's consciousness. They have a model of yourself. Where do you get this model? You look at yourself as if you are a part of the environment. If you build a model of yourself versus the environment, then you can say, "I need to have a model of myself. I have abilities, I have desires and stuff all, okay? I have a blueprint of myself, though. Not a full detail, because I cannot get the whole thing problem, but I have a blueprint. So on that level of a blueprint, I can modify things. I can look at myself in the mirror and say, "Hmm, if I change this, tweak this model, I'm gonna perform differently." That is what we mean by free will.

• 1:04:23Lex Fridman?

  And consciousness. What do you think is consciousness? Is it simply self-awareness, so including yourself into the model of the world?

• 1:04:32Speaker 01

  That's right.

• 1:04:34Speaker 01

  Some people tell me, no, this is only part of consciousness. And then they start telling me what they really mean by consciousness, and I lose them.

• 1:04:40Lex Fridman?

  Yeah.

• 1:04:42Speaker 01

  For me, consciousness is having a blueprint of your software.

• 1:04:48Lex Fridman?

  Do you have concerns about the future of AI? All the different trajectories of all of our research?

• 1:04:57Speaker 01

  Yes.

• 1:04:58Lex Fridman?

  Where's your hope? Where the movement heads? Where are your concerns?

• 1:05:01Speaker 01

  I'm concerned because I know we are building a new species that has the capability of exceeding our exceeding us

• 1:05:14Speaker 01

  exceeding our capabilities and can breed itself and take over the world, absolutely. It's a new species that is uncontrolled. We don't know the degree to which we control it. We don't even understand what it means to be able to control this new species. So I'm concerned. I don't have anything to add to that because it's such a gray area, that unknown. It never happened in history.

• 1:05:45Lex Fridman?

  Yeah.

• 1:05:47Speaker 01

  The only

• 1:05:49Speaker 01

  the only time it happened in history was evolution with a human being. And it wasn't very successful, was it?

• 1:05:58Speaker 01

  Some people say it was a great success.

• 1:06:00Lex Fridman?

  For us, it was, but a few people along the way, a few creatures along the way would not agree. So so it's just because it's such a gray area, there's nothing else to say?

• 1:06:12Speaker 01

  We have a sample of one.

• 1:06:14Lex Fridman?

  Sample of one.

• 1:06:15Speaker 01

  It's us.

• 1:06:17Lex Fridman?

  But some people would look at you and say, yeah, but we were looking to you to help us make sure that sample two works out okay.

• 1:06:30Speaker 01

  Actually, we have more than a sample of one. We have theories.

• 1:06:34Lex Fridman?

  Yeah.

• 1:06:34Speaker 01

  And that's a good, we don't need to be statisticians. So sample of one doesn't mean poverty of knowledge. It's not. Sample of one plus theory, conjectural theory of what could happen.

• 1:06:49Lex Fridman?

  Yeah.

• 1:06:50Speaker 01

  That we do have. But I really feel helpless in contributing to this argument because I know so little and my imagination is limited. And I know how much I don't know. And I, but I'm concerned.

• 1:07:12Lex Fridman?

  You were born and raised in Israel.

• 1:07:15Speaker 01

  Born and raised in Israel, yes.

• 1:07:16Lex Fridman?

  And later served in Israel military, Defense Forces.

• 1:07:22Speaker 01

  In the, in the Israel Defense Force.

• 1:07:25Lex Fridman?

  Yeah.

• 1:07:25Speaker 01

  Yeah.

• 1:07:27Lex Fridman?

  What did you learn from that experience?

• 1:07:31Speaker 01

  From this experience?

• 1:07:33Lex Fridman?

  There's a kibbutz in there as well.

• 1:07:35Speaker 01

  Yes, because I was in the nachal, which is a combination of agricultural work and military service. We were supposed, I was really idealist. I wanted to be a member of the kibbutz throughout my life and to live a communal life. And so I prepared myself for that. Slowly, slowly, I had a greater challenge.

• 1:08:08Lex Fridman?

  So that's a, that's a far world away, both in time.

• 1:08:12Speaker 01

  What I learned from that, what I can add, it was a miracle. It was a miracle that I served in the 1950s. I don't know how we survived.

• 1:08:28Speaker 01

  The country was under austerity. It tripled its population from 600,000 to 1,000,800 when I finished college. No one went hungry. Austerity, yes. When you wanted to buy, to make an omelet in a restaurant, you had to bring your own egg.

• 1:08:55Speaker 01

  And they imprisoned people from bringing food from the farming here, from the villages, to the city. But no one went hungry. And I always add to it.

• 1:09:13Speaker 01

  Higher education did not suffer any budget cut. They still invested in me, in my wife, in our generation, to get the best education that they could.

• 1:09:27Speaker 01

  So I'm really grateful for the opportunity. And I'm trying to pay back now. It's a miracle that we survived the War of 1948. They were so close to a second genocide. It was all in plan. But we survived it by miracle. And then the second miracle that not many people talk about, the next phase, how no one went hungry and the country managed to triple its population. You know what I mean, to triple the population? Imagine the United States going from, what, 350 million to 1,000,800?

• 1:10:08Lex Fridman?

  Yeah.

• 1:10:10Speaker 01

  Unbelievable.

• 1:10:11Lex Fridman?

  This is a really tense part of the world. It's a complicated part of the world, Israel and all around. Religion is, is at the core of that complexity. One of the components.

• 1:10:26Speaker 01

  Religion is a strong motivating force for many, many people in the Middle East, yes.

• 1:10:33Lex Fridman?

  In your view, looking back, is religion good for society?

• 1:10:40Speaker 01

  That's a good question for robotics, you know.

• 1:10:43Lex Fridman?

  There's echoes of that question.

• 1:10:45Speaker 01

  Equip robot with religious beliefs. Suppose we find out, or we agree that religion is good to you to keep you in line.

• 1:10:54Lex Fridman?

  Yeah.

• 1:10:55Speaker 01

  Should we give a robot the metaphor of a god?

• 1:11:00Lex Fridman?

  Yeah.

• 1:11:00Speaker 01

  As a matter of fact, the robot will get it without us also. Why? Because the robot will reason by metaphor. And what is the most primitive metaphor? A child grows with mother's smile, father teaching, father image, and mother image. That's god. So whether you want it or not, the robot will... Well, assuming that the robot is gonna have a mother and a father. It may only have a programmer, which doesn't supply warmth and discipline.

• 1:11:38Lex Fridman?

  Yeah.

• 1:11:38Speaker 01

  But discipline it does. So the robot will have this model of the trainer. And everything that happens in the world, cosmology and so on, is going to be mapped into the programmer. That's god.

• 1:11:53Lex Fridman?

  Yeah. The thing that represents the origin of everything for that robot.

• 1:12:01Speaker 01

  It's the most primitive relationship.

• 1:12:03Lex Fridman?

  So it's gonna arrive there by metaphor. And so the question is if, over all that metaphor, has served us well as humans?

• 1:12:13Speaker 01

  I really don't know. I think it did. But as long as you keep in mind, it's only metaphor.

• 1:12:22Lex Fridman?

  So if you think we can, can we talk about your son?

• 1:12:28Speaker 01

  Yes, yes.

• 1:12:30Lex Fridman?

  Can you tell his story?

• 1:12:32Speaker 01

  A story? Well...

• 1:12:34Lex Fridman?

  Daniel?

• 1:12:35Speaker 01

  The story is known. He was abducted in Pakistan by Al Qaeda-driven sect. And...

• 1:12:47Speaker 01

  Under various pretenses. I don't even pay attention to what the pretense was. Originally, they wanted to have the United States

• 1:13:00Speaker 01

  deliver some promised airplanes. It was all made up and all these demands were bogus. I don't know really, but eventually he was executed in front of a camera.

• 1:13:21Lex Fridman?

  At the core of that is hate and intolerance.

• 1:13:24Speaker 01

  At the core, yes, absolutely, yes. We don't really appreciate the depth of the hate at which billions of peoples are educated.

• 1:13:43Speaker 01

  We don't understand it. I just listen, listen to what they teach you in Mogadishu.

• 1:13:56Speaker 01

  When the water stopped in the tap,

• 1:14:03Speaker 01

  we knew exactly who did it, the Jews.

• 1:14:06Lex Fridman?

  The Jews.

• 1:14:08Speaker 01

  We didn't know how, but we knew who did it. We don't appreciate what it means to us. The depth is unbelievable.

• 1:14:17Lex Fridman?

  Do you think all of us are capable of evil?

• 1:14:24Lex Fridman?

  And the education, the indoctrination is really what creates evil.

• 1:14:28Speaker 01

  Absolutely, we are capable of evil. If you are indoctrinated sufficiently long and in depth, we are capable of ISIS, we are capable of Nazism. Yes, we are. But the question is whether we, after we have gone through some Western education and we learn that everything is really relative, that there's no absolute God, there's only a belief in God, whether we are capable now of being transformed under certain circumstances to become brutal.

• 1:15:07Lex Fridman?

  Yeah.

• 1:15:08Speaker 01

  That is a, I'm worried about it because some people say, yes, given the right circumstances, given the economical, bad economical crisis, you are capable of doing it too. That worries me. I want to believe it, I'm not capable.

• 1:15:29Lex Fridman?

  This is seven years after Daniel's death. He wrote an article at the Wall Street Journal titled, Daniel Pearl and the Normalization of Evil.

• 1:15:37Speaker 01

  Yes.

• 1:15:37Lex Fridman?

  What was your message back then? And how did it change today over the years?

• 1:15:44Speaker 01

  I lost.

• 1:15:48Lex Fridman?

  What was the message?

• 1:15:49Speaker 01

  The message was that we are not treating terrorism as a taboo. We are treating it as a bargaining device that is accepted. People have grievance and they go and bomb restaurants. Okay, it's normal. Look, you're even not surprised when I tell you that.

• 1:16:15Lex Fridman?

  Yeah.

• 1:16:17Speaker 01

  Twenty years ago, you say, what? For grievance, you go and blow a restaurant? Today, it's becoming normalized. The banalization of evil. And we have created that to ourselves by normalizing, by making it part of political life. It's a political debate. Every terrorist yesterday becomes a freedom fighter today. And tomorrow, it becomes a terrorist again. It's switchable.

• 1:16:56Lex Fridman?

  All right, and so we should call out evil when there's evil?

• 1:17:00Speaker 01

  If we don't want to be part of it.

• 1:17:04Lex Fridman?

  Become it.

• 1:17:06Speaker 01

  Yeah, if we want to separate good from evil. That's one of the first things that, what was it? In the Garden of Eden, remember the first thing that God told him was, hey, you want some knowledge? Here is a tree of good and evil.

• 1:17:26Lex Fridman?

  So this evil touched your life personally. Does your heart have anger, sadness, or is it hope?

• 1:17:37Speaker 01

  I see some beautiful people coming from Pakistan. I see beautiful people everywhere. But I see horrible propagation of evil in this country too. It shows you how populistic slogans can catch the mind of the best intellectuals.

• 1:18:05Lex Fridman?

  Today is Father's Day.

• 1:18:07Speaker 01

  I didn't know that.

• 1:18:08Lex Fridman?

  Yeah.

• 1:18:09Speaker 01

  I heard it.

• 1:18:10Lex Fridman?

  What's a fond memory you have of Daniel?

• 1:18:15Speaker 01

  Oh, many good memories. Immense. He was my mentor.

• 1:18:23Speaker 01

  He had a sense of balance that I didn't have.

• 1:18:31Lex Fridman?

  Yeah.

• 1:18:32Speaker 01

  He saw the beauty in every person.

• 1:18:36Speaker 01

  He was not as emotional as I am, more looking at things in perspective. He really liked every person. He really grew up with the idea that a fawn is a reason for curiosity, not for fear.

• 1:19:00Speaker 01

  One time we went in Berkeley and homeless came out from some dark alley and said, "Hey, man, can you spare a dime?" I retreated back, you know, two feet back. And Daniel just hugged him and said, "Here's a dime. Enjoy yourself. Maybe you want some money to take a bath or whatever." Where did he get it? Not from me.

• 1:19:27Lex Fridman?

  Do you have advice for young minds today dreaming about creating, as you have dreamed, creating intelligent systems? What is the best way to arrive at new breakthrough ideas and carry them through the fire of criticism and past conventional ideas?

• 1:19:46Speaker 01

  Ask your questions.

• 1:19:49Speaker 01

  Freely. Your questions are never dumb.

• 1:19:55Speaker 01

  And solve them your own way.

• 1:19:58Speaker 01

  And don't take no for an answer.

• 1:20:02Speaker 01

  Look, if they are really dumb, you will find out quickly by trying an arrow to see that they are not leading any place. But follow them and try to understand things your way. That is my advice. I don't know if it's gonna help anyone.

• 1:20:21Lex Fridman?

  No, that's brilliantly put.

• 1:20:22Speaker 01

  There is a lot of

• 1:20:27Speaker 01

  inertia in science, in academia. It is slowing down science.

• 1:20:35Lex Fridman?

  Yeah, those two words, "your way", that's a powerful thing. It's against inertia, potentially, against the flow.

• 1:20:44Speaker 01

  Against your professor.

• 1:20:46Lex Fridman?

  Against your professor.

• 1:20:47Speaker 01

  I wrote the Book of Why in order to democratize common sense.

• 1:20:56Speaker 01

  In order to instill rebellious spirits in students. So they wouldn't wait until the professor gets things right.

• 1:21:10Lex Fridman?

  So you wrote the manifesto of the rebellion against the professor?

• 1:21:15Speaker 01

  Against the professor, yes.

• 1:21:17Lex Fridman?

  So looking back at your life of research, what ideas do you hope ripple through the next many decades? What do you hope your legacy will be?

• 1:21:27Speaker 01

  I already have a tombstone carved.

• 1:21:37Lex Fridman?

  Oh, boy.

• 1:21:38Speaker 01

  The fundamental law of counterfactuals. That's what it's. It's a simple equation. What a counterfactual in terms of a model surgery? That's it, because everything follows from that.

• 1:21:57Speaker 01

  If you get that, all the rest, I can die in peace. And my student can derive all my knowledge by mathematical means.

• 1:22:09Lex Fridman?

  The rest follows.

• 1:22:10Speaker 01

  Yeah.

• 1:22:11Lex Fridman?

  Judea, thank you so much for talking today. I really appreciate it.

• 1:22:14Speaker 01

  Well, thank you for being so attentive and instigating.

• 1:22:21Lex Fridman?

  We did it.

• 1:22:22Speaker 01

  We did it.

• 1:22:23Lex Fridman?

  The coffee helped. Thanks for listening to this conversation with Judea Pearl. And thank you to our presenting sponsor Cash App. Download it, use code LEGSPODCAST, you'll get $10. And $10 will go to FIRST, a STEM education nonprofit that inspires hundreds of thousands of young minds to learn and to dream of engineering our future. If you enjoy this podcast, subscribe on YouTube, give it five stars on Apple Podcasts, support it on Patreon, or simply connect with me on Twitter. And now, let me leave you with some words of wisdom from Judea Pearl. You cannot answer a question that you cannot ask, and you cannot ask a question that you have no words for.

• 1:23:06Lex Fridman?

  Thank you for listening and hope to see you next time.