#148 – Charles Isbell and Michael Littman: Machine Learning and Education

Charles Isbell is the Dean of the College of Computing at Georgia Tech. Michael Littman is a computer scientist at Brown University. Please support this podcast by checking out our sponsors: – Athletic Greens: https://athleticgreens.com/lex and use code LEX to get 1 month of fish oil – Eight Sleep: https://www.eightsleep.com/lex and use code LEX to get special savings – MasterClass: https://masterclass.com/lex to get 2 for price of 1 – Cash App: https://cash.app/ and use code LexPodcast to get $10 EPISODE LINKS: Charles’s Twitter: https://twitter.com/isbellHFh Charles’s Website: https://www.cc.gatech.edu/~isbell/ Michael’s Twitter: https://twitter.com/mlittmancs Michael’s Website: https://www.littmania.com/ Michael’s YouTube: https://www.youtube.com/user/mlittman PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ YouTube Full Episodes: https://youtube.com/lexfridman YouTube Clips: https://youtube.com/lexclips SUPPORT & CONNECT: – Check out the sponsors above, it’s the best way to support this podcast – Support on Patreon: https://www.patreon.com/lexfridman – Twitter: https://twitter.com/lexfridman – Instagram: https://www.instagram.com/lexfridman – LinkedIn: https://www.linkedin.com/in/lexfridman – Facebook: https://www.facebook.com/LexFridmanPage – Medium: https://medium.com/@lexfridman OUTLINE: Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time. (00:00) – Introduction (07:51) – Is machine learning just statistics? (12:14) – NeurIPS vs ICML (14:30) – Data is more important than algorithm (20:14) – The role of hardship in education (28:57) – How Charles and Michael met (33:30) – Key to success: never be satisfied (36:47) – Bell Labs (48:15) – Teaching machine learning (58:25) – Westworld and Ex Machina (1:06:24) – Simulation (1:13:14) – The college experience in the times of COVID (1:41:52) – Advice for young people (1:48:44) – How to learn to program (2:00:07) – Friendship

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Lex Fridman?

  The following is a conversation with Charles Isbell and Michael Littman. Charles is the dean of the College of Computing at Georgia Tech, and Michael is a computer science professor at Brown University. I've spoken with each of them individually on this podcast, and since they are good friends in real life, we all thought it would be fun to have a conversation together. Quick mention of each sponsor, followed by some thoughts related to the episode. Thank you to Athletic Greens, the only one drink that I start every day with to cover all my nutritional bases. Eight Sleep, a mattress that cools itself and gives me yet another reason to enjoy sleep. MasterClass, online courses from some of the most amazing humans in history,

• 0:43Lex Fridman?

  and Cash App, the app I use to send money to friends. Please check out the sponsors in the description to get a discount and to support this podcast. As a side note, let me say that having two guests on the podcast is an experiment that I've been meaning to do for a while, in particular because down the road, I would like to occasionally be a kind of moderator for debates between people that may disagree in some interesting ways. If you have suggestions for who you would like to see debate on this podcast, let me know. As with all experiments of this kind, it is a learning process. Both the video and the audio might need improvement. I realize I think I should probably do three or more cameras next time as opposed to just two,

• 1:28Lex Fridman?

  and also try different ways to mount the microphone for the third person. Also, after recording this intro, I'm going to have to go figure out the thumbnail for the video version of the podcast, since I usually put the guest's head on the thumbnail, and now there's two heads and two names to try to fit into the thumbnail. It's a kind of bin packing problem, which in theoretical computer science happens to be an NP-hard problem.

• 2:02Lex Fridman?

  Whatever I come up with, if you have better ideas for the thumbnail, let me know as well. And in general, I always welcome ideas how this thing can be improved. If you enjoy it, subscribe on YouTube, review it with five stars on Apple Podcast, follow on Spotify, support on Patreon, or connect with me on Twitter at Lex Fridman. As usual, I'll do a few minutes of ads now and no ads in the middle. I try to make these interesting, but I do give you timestamps so you can go ahead and skip if you must. But please do check out the sponsors by clicking the links in the description. It's the best way to support this podcast. This show is sponsored by Athletic Greens, the all-in-one daily drink to support

• 2:43Lex Fridman?

  better health and peak performance. It replaced the multivitamin for me and went far beyond that with 75 vitamins and minerals. I do intermittent fasting of 16 to 24 hours every day and always break my fast with Athletic Greens. I can't say enough good things about these guys. It helps me not worry whether I'm getting all the nutrients I need. One of the many reasons I'm a fan is that they keep iterating on their formula. I love continuous improvement. Life is not about reaching perfection, it's about constantly striving for it and making sure each iteration is a positive delta. The other thing I've taken

• 3:22Lex Fridman?

  for a long time outside of Athletic Greens is fish oil. So I'm especially excited, even though I genetically don't seem to be capable of generating the sound of excitement with my voice. I'm excited now that they're selling fish oil and are offering listeners of this podcast free one month's supply of wild-caught omega-3 fish oil. When you go to athleticgreens.com/lex to claim this special offer, click athleticgreens.com/lex in the description to get the fish oil and the all-in-one supplement I rely on for the nutritional foundation of my physical and mental performance.

• 4:05Lex Fridman?

  This episode is also sponsored by Asleep and its Pod Pro mattress. It controls temperature with an app. It's packed with sensors. It can cool down to as low as 55 degrees on each side of the bed separately. It's been a game changer for me. I just enjoy sleep and power naps more. I feel like I fall asleep faster and get more restful sleep. Combination of cool bed and warm blanket is amazing. Now, if you love your current mattress, but still looking for temperature control, Asleep's new Pod Pro cover adds dynamic cooling and heating capabilities onto your current mattress.

• 4:45Lex Fridman?

  It can cool down to 55 degrees or heat up to 110 degrees and do so on each side of the bed separately. It's magic, really. Also, it can track a bunch of metrics like heart rate variability, but honestly, cooling alone is worth the money. Go to Asleep.com/Lex and when you buy stuff there during the holidays, you get special savings as listeners of this podcast. And you know the savings are special because I use the word special. Again, that's Asleep.com/Lex. This show is also sponsored by MasterClass.

• 5:25Lex Fridman?

  $180 a year for an all access pass to watch courses from literally the best people in the world on a bunch of different topics. Let me list some that I have watched and enjoyed. Chris Hatfield on space exploration. Neil deGrasse Tyson on scientific thinking and communication. I probably should get Neil on this podcast soon. Will Wright, creator of SimCity and Sims on game design. Carlos Santana on guitar. I'm working on Europa right now, actually. Garry Kasparov on chess. Daniel Negrono on poker. Neil Gaiman on storytelling. Martin Scorsese on filmmaking. Tony Hawk on skateboarding. And Jane Goodall on conservation. By the way, you can watch it on basically any device.

• 6:09Lex Fridman?

  Sign up at masterclass.com/Lex for the buy one, get one free membership for you and a friend. That's masterclass.com/Lex.

• 6:21Lex Fridman?

  This show is presented by Cash App, the number one finance app in the App Store. When you get it, use code LEXPODCAST. Cash App lets you send money to friends, buy Bitcoin, and invest in the stock market with as little as $1. In fact, just yesterday, I think, I tweeted that the Mars economy will run on cryptocurrency. I do believe that's true. It's kind of the obvious trajectory, but it's also fun to talk about. And I wonder what that cryptocurrency will be. Right now Bitcoin and Ethereum seem to be dominating the space, but who knows what 10, 20, 30, 50, 100 years from now looks like. Anyway, I hope to talk to a bunch of folks from the cryptocurrency space on this podcast soon,

• 7:05Lex Fridman?

  including, once again, the great, the powerful, Vitalik Buterin. So again, if you get Cash App from the App Store, Google Play, and use code LEXPODCAST, you get $10, and Cash App will also donate $10 to FIRST, an organization that is helping to advance robotics and STEM education for young folks around the world. And now here's my conversation with Charles Isbell and Michael Lidman.

• 7:51Lex Fridman?

  You'll probably disagree about this question, but what is your biggest, would you say, disagreement about either something profound and very important, or something completely not important at all?

• 8:04Charles Isbell?

  I don't think you have any disagreements at all.

• 8:05Speaker 02

  Ah, I'm not sure that's true.

• 8:08Charles Isbell?

  We walked into that one, didn't we?

• 8:10Speaker 02

  So, so one thing that you sometimes mention is that, and we did this one on air too, as it were, whether or not machine learning is computational statistics.

• 8:19Charles Isbell?

  It's not.

• 8:21Speaker 02

  But it is.

• 8:22Charles Isbell?

  Well, it's not. And in particular, and more importantly, it is not just computational statistics.

• 8:27Lex Fridman?

  So what's missing in the picture?

• 8:28Charles Isbell?

  All the rest of it.

• 8:31Speaker 02

  What's missing? That which is missing. Oh, yes, well, you can't be wrong now.

• 8:35Charles Isbell?

  Well, it's not just the statistics. He doesn't even believe this. We've had this conversation before. If it were just the statistics, then we would be happy with where we are. But it's not just the statistics.

• 8:44Speaker 02

  That's why it's computational statistics.

• 8:46Charles Isbell?

  Or if it were just the computational.

• 8:47Speaker 02

  I agree that machine learning is not just statistics.

• 8:49Charles Isbell?

  It is not just statistics.

• 8:50Speaker 02

  We can agree on that.

• 8:51Charles Isbell?

  Nor is it just computational statistics.

• 8:52Speaker 02

  It's computational statistics.

• 8:54Charles Isbell?

  It is computational.

• 8:54Lex Fridman?

  What is the computational and computational statistics? Does this take us into the realm of computing?

• 8:59Charles Isbell?

  It does, but I think perhaps the way I can get him to admit that he's wrong is that it's about rules.

• 9:08Speaker 02

  It's about rules.

• 9:08Charles Isbell?

  It's about symbols. It's about all these other things.

• 9:10Speaker 02

  But the statistics is not about rules? I'm gonna say statistics is about rules.

• 9:13Charles Isbell?

  But it's not just the statistics, right? It's not just a random variable that you choose and you have a probability.

• 9:17Speaker 02

  I think you have a narrow view of statistics.

• 9:19Charles Isbell?

  OK, well, then what would be the broad view of statistics that would still allow it to be statistics and not say history that would make computational statistics OK?

• 9:27Speaker 02

  Well, OK, so I had my first sort of research mentor, a guy named Tom Landauer, taught me to do some statistics, right? And I was annoyed all the time because the statistics would say that what I was doing was not statistically significant. And I was like, but, but, but. And basically, what he said to me is statistics is how you're gonna keep from lying to yourself, which I thought was really deep.

• 9:54Charles Isbell?

  It is a way to keep yourself honest in a particular way. I agree with that.

• 9:58Speaker 02

  Yeah, and so you're trying to find rules. I'm just gonna bring it back to rules.

• 10:03Lex Fridman?

  Wait, wait, wait. Could you possibly try to define rules?

• 10:08Speaker 02

  Even regular statisticians, non-computational statisticians, do spend some of their time evaluating rules, right? Applying statistics to try to understand, does this, you know, does this rule capture this? Does this not capture that?

• 10:20Lex Fridman?

  You mean like hypothesis testing kind of thing? Or like confidence intervals, like?

• 10:25Speaker 02

  More like hypothesis. Like, I feel like the word statistic literally means like a summary, like a number that summarizes other numbers. But I think the field of statistics actually applies that idea to things like rules, to understand whether or not a rule is valid.

• 10:40Charles Isbell?

  So software engineering statistics?

• 10:42Speaker 02

  No.

• 10:43Charles Isbell?

  Programming languages, statistics?

• 10:45Speaker 02

  No.

• 10:45Charles Isbell?

  Because I think there's a very, it's useful to think about a lot of what AI and machine learning is, or certainly should be, as software engineering, as programming languages. Just if to put it in language that you might understand, the hyperparameters beyond the problem itself.

• 11:00Speaker 02

  The hyperparameters is too many syllables for me to understand.

• 11:02Charles Isbell?

  The hyperparameters.

• 11:04Speaker 02

  That's better.

• 11:04Charles Isbell?

  That goes around it, right? It's the decisions you choose to make. It's the metrics you choose to use. It's the loss function.

• 11:10Speaker 02

  You want to say the practice of machine learning is different than the practice of statistics? Like, the things you have to worry about and how you worry about them are different, therefore, they're different?

• 11:19Charles Isbell?

  Right. At a very little, I mean, at the very least, it's that that much is true. It doesn't mean that statistics, computational or otherwise, aren't important. I think they are. I mean, I do a lot of that, for example, but I think it goes beyond that. I think that we could think about game theory in terms of statistics, but I don't think it's very as useful to do. I mean, the way I would think about it, or a way I would think about it, is this way. Chemistry is just physics. But I don't think it's as useful to think about chemistry as being just physics. It's useful to think about it as chemistry. The level of abstraction really matters here.

• 11:51Speaker 02

  So I think it is, there are contexts in which it is useful.

• 11:55Charles Isbell?

  Yes.

• 11:55Speaker 02

  Think of it that way, right?

• 11:56Charles Isbell?

  No, that is.

• 11:56Speaker 02

  And so finding that connection is actually helpful. And I think that's when I when I emphasize the computational statistics thing. I think I think I want to befriend statistics and not absorb them.

• 12:06Charles Isbell?

  Here's the here's the a way to think about it beyond what I just said, right? So what would you say, and I want you to think back to a conversation we had a very long time ago, what would you say is the difference between, say, the early 2000s, ICML and what we used to call NIPS, NERPs? Is there a difference, a lot of particularly on the machine learning that was done there?

• 12:25Lex Fridman?

  ICML is around that long?

• 12:27Charles Isbell?

  Oh, yeah.

• 12:28Lex Fridman?

  So ICL is the new conference, newish.

• 12:31Charles Isbell?

  Uh, yeah, I guess so.

• 12:32Lex Fridman?

  And ICML is around the 2000s?

• 12:34Charles Isbell?

  Oh, ICML predates that.

• 12:36Speaker 02

  I think my most cited ICML paper is from 94.

• 12:40Charles Isbell?

  Michael knows this better than me because, of course, he's significantly older than I am. But the point is.

• 12:43Lex Fridman?

  Yeah.

• 12:43Charles Isbell?

  What is the difference? What was the difference between ICML and NERPs in the late 90s, early 2000s?

• 12:49Speaker 02

  I don't know what everyone else's perspective would be, but I had a particular perspective at that time, which is I felt like ICML was more of a of a computer science place and that NIPS, NERPs was more of an engineering place, like the kind of math that happened at the two places. As a computer scientist, I felt more comfortable with the ICML math. And the NERPs people would say that that's because I'm dumb. And that's such an engineering thing to say, so.

• 13:16Charles Isbell?

  I agree with that part of it, but I do a little differently. We actually had a nice conversation with Tom Dietrich about this.

• 13:21Speaker 02

  On Twitter.

• 13:21Charles Isbell?

  On Twitter just a couple of days ago. I put it a little differently, which is that ICML was machine learning done by computer scientists. And NERPs was machine learning done by computer scientists trying to impress statisticians.

• 13:37Charles Isbell?

  Which was weird because it was the same people, at least by the time I started paying attention. But it just felt very, very different. And I think that that perspective of whether you're trying to impress the statisticians or you're trying to impress the programmers is actually very different and has real impact on what.

• 13:52Speaker 02

  Yeah, on the focus.

• 13:52Charles Isbell?

  What you choose to worry about and what kind of outcomes you come to. So I think it really matters. I think computer statistics is a means to an end. It is not an end in some sense. And I think that really matters here in the same way that I don't think computer science is just engineering or just science or just math or whatever.

• 14:07Speaker 02

  Okay, so I'd have to now agree that now we agree on everything.

• 14:10Charles Isbell?

  Yes, yes. The important thing here is that, you know, my opinions may have changed, but not the fact that I'm right, I think, is what we just came to.

• 14:19Speaker 02

  Right. And my opinions may have changed and not the fact that I'm wrong.

• 14:22Charles Isbell?

  That's right.

• 14:23Lex Fridman?

  You lost me. I'm not even.

• 14:24Charles Isbell?

  I think I lost myself there too. But anyway.

• 14:26Speaker 02

  But we're back.

• 14:27Charles Isbell?

  We're back.

• 14:28Speaker 02

  We, this happens to us sometimes. We're sorry.

• 14:30Lex Fridman?

  How does neural networks change this, just to even linger on this topic, change this idea of statistics? How big of a pie statistics is within the machine learning thing? Like, because it sounds like hyperparameters and also just the role of data. You know, this, people are starting to use this terminology of software 2.0, which is like the act of programming as a, as a, like you're a designer in the hyperparameter space of neural networks and you're also the collector and the organizer and the cleaner. Of the data and that's part of the programming.

• 15:11Lex Fridman?

  So how did, on the NeurIPS versus ICML topic, what's the role of neural networks and redefining the size and the role of machine learning?

• 15:21Charles Isbell?

  I can't, I can't wait to hear what Michael thinks about this, but I would add one. But you will. But that's true, I will, I'll force myself to. I think the, the, there's one thing I would add to your description, which is the kind of software engineering part of what does it mean to debug, for example? But this is a difference between the kind of computational physics view of machine learning and the computational view of machine learning, which is, I think, one is worried about the equation, as it were. And by the way, this is not a value judgment, I just think it's about perspective. But the kind of questions you would ask, you start asking yourself, well, what does it mean to program and develop and build the system, is a very computer sciency view of the problem. I mean, when, if you get on data science Twitter and econ Twitter, you actually hear

• 16:03Charles Isbell?

  this a lot with the, you know, the economist and the data scientist complaining about the machine learning people, well, it's, you know, it's just statistics and I don't know why they don't, don't see this, but they're not even asking the same questions. They're not thinking about it as a kind of programming problem. And I think that that really matters, just asking this question. I actually think it's a little different from programming and hyperparameter space and sort of collecting the data, but I do think that that immersion really matters. So I'll give you a quick, a quick example of the way I think about this. So I teach machine learning. Michael and I have co-taught a machine learning class, which has now reached, I don't know, 10,000 people at least, over the last several years, or somewhere there's abouts. And my machine learning assignments are of this form.

• 16:46Charles Isbell?

  So the super, the first one is something like, implement these five algorithms, you know, KNN and S, you know, SVMs and boosting and decision trees and neural networks, and maybe that's it, I can't remember. And when I say implement, I mean steal the code. I am completely uninterested. You get zero points for getting the thing to work. I don't want you spending your time worrying about getting the corner case right of, you know, what happens when you are trying to normalize distances and the points on the thing and so you divide by zero. I'm not interested in that, right? Steal the code. However, you're going to run those algorithms on two data sets. The data sets have to be interesting. What does it mean to be interesting? Well, data sets interesting if it reveals differences between algorithms, which presumably are all the same because they can represent whatever they can represent.

• 17:31Charles Isbell?

  And two data sets are interesting together if they show different differences, as it were. And you have to analyze them. You have to justify their interestingness and you have to analyze them a whole bunch of ways. But all I care about is the data in your analysis, not the programming. And I occasionally end up in these long discussions with students. Well, I don't really, I copy and paste the things that I've said the other 15,000 times it's come up, which is, they go, but the only way to learn, really understand, is to code them up, which is a very programmer, software engineering view of the world. If you don't program it, you don't understand it. Which is, by the way, I think is wrong in a very specific way. But it is a way that you come to understand because then you have to wrestle with the algorithm. But the thing about machine learning is it's not just sorting numbers, where in some sense the data doesn't matter.

• 18:13Charles Isbell?

  What matters is, well, does the algorithm work on these abstract things, one, the list, the other? In machine learning, the data matters. It matters more than almost anything. And not everything, but almost anything. And so, as a result, you have to live with the data and don't get distracted by the algorithm per se. And I think that that focus on the data and what it can tell you and what question it's actually answering for you, as opposed to the question you thought you were asking, is a key and important thing about machine learning and is a way that computationalists, as opposed to statisticians, bring a particular view about how to think about the process. The statisticians, by contrast, bring, I think I'd be willing to say, a better view about the kind of formal math that's behind it

• 18:56Charles Isbell?

  and what an actual number ultimately is saying about the data. And those are both important, but they're also different.

• 19:03Lex Fridman?

  I didn't really think of it this way, is to build intuition about the role of data, the different characteristics of data by having two data sets that are different and that reveal the differences in the differences. That's, that's a really fascinating, that's a really interesting educational approach.

• 19:19Speaker 02

  The students love it, but not right away.

• 19:22Charles Isbell?

  No, they love it at the end.

• 19:23Speaker 02

  They love it later.

• 19:23Charles Isbell?

  They love it at the end, not at the beginning.

• 19:27Speaker 02

  Not even, not even immediately after.

• 19:29Lex Fridman?

  I feel like there's a deep, profound lesson about education there.

• 19:32Speaker 02

  Yeah.

• 19:32Lex Fridman?

  That you can't listen to students about whether what you're doing is the right or the wrong thing.

• 19:40Charles Isbell?

  Well, as a wise, Michael Littman once said to me about children, which I think applies to teaching, is you have to give them what they need without bending to their will. And students are like that. You have to figure out what they need. You're a curator. Your whole job is to curate and to present. Because on their own, they're not going to necessarily know where to search. So you're providing pushes in some direction and learn space. And you have to give them what they need in a way that keeps them engaged enough so that they eventually discover what they want. And they get the tools they need to go and learn other things off them.

• 20:14Lex Fridman?

  What's your view? Let me put on my Russian hat, which believes that life is suffering.

• 20:20Speaker 02

  I like Russian hats, by the way. If you have one, I would like this.

• 20:22Lex Fridman?

  Those are ridiculous. Yes.

• 20:25Speaker 02

  But in a delightful way. But sure.

• 20:28Lex Fridman?

  What do you think is the role of, we talked about balance a little bit. What do you think is the role of hardship in education? Like, I think the biggest things I've learned, like what made me fall in love with math, for example, is by being bad at it until I got good at it. So like, like struggling with a problem, which increased the level of joy I felt when I finally figured it out. And it always felt with me, with teachers, especially modern discussions of education, how can we make education more fun, more engaging, more all those things?

• 21:09Lex Fridman?

  Or from my perspective, it's like, you're maybe missing the point that education, that life is suffering. Education is supposed to be hard, and that actually what increases the joy you feel when you actually learn something. Is that ridiculous? Do you like to see your students suffer?

• 21:29Speaker 02

  Okay, so this may be a point where we differ.

• 21:32Charles Isbell?

  I suspect not. Let's do go on.

• 21:34Speaker 02

  Well, what would your answer be?

• 21:35Charles Isbell?

  I want to hear you first.

• 21:36Speaker 02

  Okay, well, I was gonna not answer the question.

• 21:40Charles Isbell?

  So you don't want the students to know you're going to suffer?

• 21:42Speaker 02

  No, no, no, no, no. I was, I was gonna say that there's. I think there's a distinction that you can make in the kind of suffering, right? So I think you can be in a mode where you're you're suffering in a hopeless way versus you're suffering in a hopeful way, right? Where you're like, you can see that if you that you still have, you can still imagine getting to the end, right? And as long as people are in that mindset where they're struggling, but it's not a hopeless kind of struggling, that's that's productive. I think that's really helpful. But if struggling, like if you you break their will, if you leave them hopeless, no, that don't, sure, some people are gonna, whatever, lift themselves up by their bootstraps.

• 22:25Speaker 02

  But like mostly you give up and certainly it takes the joy out of it. And you're not going to spend a lot of time on something that brings you no joy. So it's it's it is a bit of a delicate balance, right? You have to thwart people in a way that they still believe that there's a way through.

• 22:41Charles Isbell?

  Right. So that's a that we strongly agree, actually. So I think, well, first off, struggling and suffering aren't the same thing, right? Spain poetic. Oh, no, I actually appreciate the poetry. And and I, one of the reasons I appreciate it is that they are often the same thing and often quite different, right? So you can struggle without suffering, you can certainly suffer, suffer, suffer pretty easily. You don't necessarily have to struggle to suffer. So I think that you want people to struggle, but that hope matters. You have to, they have to understand that they're going to get through it on the other side. And it's very easy to confuse the two. I actually think Brown University has a very, just philosophically, has a very different take on the relationship with their students, particularly undergrads from, say, a place like Georgia Tech, which is Which university's better?

• 23:26Charles Isbell?

  Well, I have my opinions on that.

• 23:28Speaker 02

  I mean, remember, Charles said, it doesn't matter what the facts are, I'm always right.

• 23:32Charles Isbell?

  The correct answer is that it doesn't matter, they're different. But

• 23:39Speaker 02

  He went to a school like the school where he is as an undergrad. I went to a school, specifically the same school, though it was, it changed a bit in the in the intervening years.

• 23:48Charles Isbell?

  Brown or Georgia Tech?

• 23:48Speaker 02

  No, I was talking about Georgia Tech. And I went

• 23:50Charles Isbell?

  Georgia Tech's changed.

• 23:50Speaker 02

  Yeah. And I went to an undergrad place that's a lot like the place where I work now. And so it does seem like we're more familiar with these models.

• 23:57Charles Isbell?

  There's a similarity between Brown and Yale?

• 24:00Speaker 02

  Yeah, I think that I think they're quite similar. Yeah.

• 24:03Charles Isbell?

  And Duke.

• 24:03Speaker 02

  Duke has some similarities, too, but it's got a little southern drawl.

• 24:07Charles Isbell?

  You've kind of worked your, you've sort of worked at universities that are like the places where you learn. And the same would be true for me.

• 24:16Lex Fridman?

  Are you uncomfortable venturing outside the box? Is that what you're saying?

• 24:21Charles Isbell?

  Not what I'm saying.

• 24:23Speaker 02

  Yeah, Charles is definitely. He only goes to places that have institute in the name, right?

• 24:27Charles Isbell?

  It has worked out that way. Well, academic places, anyway. Well, no, I was a visiting scientist at UPenn or visiting, visiting something at UPenn.

• 24:35Lex Fridman?

  Oh, wow. I just, I just understood your joke.

• 24:38Speaker 02

  Which one?

• 24:41Lex Fridman?

  Five minutes later.

• 24:42Speaker 02

  I like to set these sort of time bombs.

• 24:44Lex Fridman?

  The institute is in the, uh, that Charles only goes to places that have institute in the name. So I guess Georgia, I forget that Georgia Tech is Georgia Institute of Technology.

• 24:55Charles Isbell?

  The number of people who refer to it as Georgia Tech University is large and incredibly irritating. It's one of the few things that genuinely gets under my skin.

• 25:03Speaker 02

  But like, schools like Georgia Tech and MIT have as part of the ethos, like, there is, I want to say there's a, there's an abbreviation that someone taught me, like IHTFP, something like that. Like, there's a, there's a, there's an expression which is basically, I hate being here, which they say so proudly. And that is definitely not the ethos at Brown. Like, Brown is, there's a little more pampering and empowerment and stuff. And it's not like, we're gonna crush you and you're gonna love it. So, yeah, I think there's a, I think the ethos is are different.

• 25:33Lex Fridman?

  Mm-hmm. That's interesting, yeah.

• 25:34Charles Isbell?

  We had drown-proofing.

• 25:36Lex Fridman?

  What's that?

• 25:37Charles Isbell?

  In order to graduate from Georgia Tech, this is a true thing, feel free to look it up. If you.

• 25:42Speaker 02

  A lot of schools have this, by the way.

• 25:43Charles Isbell?

  No, actually, Georgia Tech was apparently the first.

• 25:45Speaker 02

  Brandeis has it.

• 25:46Charles Isbell?

  Had it.

• 25:47Lex Fridman?

  I feel like Georgia Tech was the first in a lot of things.

• 25:51Charles Isbell?

  It was, it was the first in a lot of things. Had the first master's degree. Stop that. First masters in computer science, actually.

• 25:58Speaker 02

  Right, online masters.

• 26:00Charles Isbell?

  Well, that too, but way back in the 60s. NSF grant, yeah, yeah. You're the first information and computer science master's degree in the country. But the Georgia Tech, it used to be the case that in order to graduate from Georgia Tech, you had to take a drown-proofing class, where effectively, they threw you water, tied you up. If you didn't drown, you got to graduate.

• 26:19Speaker 02

  Tied you up?

• 26:20Charles Isbell?

  I believe so.

• 26:21Speaker 02

  No.

• 26:22Charles Isbell?

  There were certainly versions of it, but, I mean, luckily, they ended it just before I had to graduate, because otherwise I would have never graduated. It wasn't gonna happen. I want to say 84, 83, someone around then, they, they ended it. But, yeah, you used to have to prove you could tread water for some ridiculous amount of time, or you couldn't.

• 26:38Speaker 02

  Two minutes.

• 26:38Charles Isbell?

  You couldn't graduate. No, it was more than two minutes. Okay, well, we'll look.

• 26:41Lex Fridman?

  And it was in a bathtub.

• 26:43Speaker 02

  Yeah, you could just say.

• 26:44Charles Isbell?

  It was in a pool, but it was a real thing. But that idea that, you know, push you.

• 26:48Speaker 02

  Fully clothed.

• 26:49Charles Isbell?

  Yeah, fully clothed.

• 26:49Speaker 02

  I don't think, I bet, I bet it was that and not tied up. Because like, who needs to learn how to swim when you're tied? Nobody, but who needs to learn when to swim when you're actually falling into the water dressed? That's a real thing.

• 26:59Lex Fridman?

  I think your facts are getting in the way with a good story.

• 27:02Speaker 02

  Oh, that's fair. That's fair. I didn't mean to. All right, so they tie you up.

• 27:05Charles Isbell?

  The narrative matters more. But whatever it was, you had to, it was called drown proofing for a reason. The point of the story, Michael, is.

• 27:12Lex Fridman?

  Struggle.

• 27:13Charles Isbell?

  It's, well, no, but that's good.

• 27:15Speaker 02

  That was, yeah, that's right.

• 27:17Charles Isbell?

  That's a part of what Georgia Tech has always been. And we struggle with that, by the way, about what we want to be.

• 27:22Speaker 02

  And they like it.

• 27:23Charles Isbell?

  As things go, but you, you sort of, how much can you be pushed without breaking? And you come out of the other end stronger, right? There, there's this saying we used to have when I was in undergrad. There was Georgia Tech, building tomorrow the night before. And it was just kind of, kind of idea that, you know, give me something impossible to do and I'll do it in a couple of days because that's what I just spent the last four or five or six.

• 27:48Speaker 02

  That ethos definitely stuck to you. Having now done a number of projects with you, you definitely will do it the night before.

• 27:55Charles Isbell?

  That's not entirely true. There's nothing wrong with waiting until the last minute. The secret is knowing when the last minute is.

• 28:00Lex Fridman?

  Right, that's brilliant, that's brilliantly put.

• 28:02Speaker 02

  Yeah, that, that's, that is a definite Charles statement that I am trying not to embrace.

• 28:09Charles Isbell?

  And I appreciate that because you helped move my last minute.

• 28:12Lex Fridman?

  That's a social construct, the way you converge together, what the definition of last minute is. And we, we figure that out all together. In fact, MIT, you know, I'm, I'm sure a lot of universities have this, but MIT has like MIT time that.

• 28:25Speaker 02

  Yeah.

• 28:25Lex Fridman?

  Everyone is always agreed together that there is such a concept and everyone just keeps showing up like 10 to 15 to 20, depending on the department, late to everything. So there's like a weird drift that happens. It's kind of fascinating.

• 28:39Charles Isbell?

  Yeah, we're five minutes.

• 28:40Lex Fridman?

  Five minutes?

• 28:41Charles Isbell?

  In fact, the classes will say, you know, well, this is no longer true, actually. But it used to be a class would start at eight, but actually it started at eight oh five. It ends at nine, actually it ends at eight fifty five. Everything's five minutes off and nobody expects anything to start until five minutes after the half hour, whatever it is. It still exists.

• 28:56Speaker 02

  It hurts my head.

• 28:57Lex Fridman?

  Well, let's rewind the clock back to the 50s and 60s when you guys met. I'm just kidding. But what, can you tell the story of how you met? So you've, like, the internet and the world kind of knows you as, as, as connected in some ways in terms of education, of teaching the world. That's like the public facing thing, but how did you as human beings and as collaborators meet?

• 29:25Charles Isbell?

  I think there's two stories. One is how we met and the other is how we got to know each other. I'm not gonna say fell in love. I'm gonna say that we came to understand that we.

• 29:35Speaker 02

  Had some common something. Yeah. It's funny because on the surface, I think we're, we're different in a lot of ways, but there's something.

• 29:43Charles Isbell?

  Yeah, I mean, now we completely.

• 29:44Speaker 02

  Just consonant.

• 29:46Charles Isbell?

  There you go.

• 29:46Speaker 02

  Afternoon.

• 29:48Charles Isbell?

  So I will tell the story of how we met and I'll let Michael tell the story of how we.

• 29:52Speaker 02

  Okay, all right.

• 29:53Charles Isbell?

  Okay, so here's how we met. I was already at, at that point it was AT&T Labs. There's a long, interesting story there. But anyway, I was there and Michael was coming to interview. He was a professor at Duke at the time, but decided for reasons that he wanted to be in New Jersey. And so that would mean Bell Labs slash AT&T Labs. And we were doing interviews. Interviews are very much like academic interviews. And so I had to be there. We all had to meet with him afterwards and so on, one on one. But it was obvious to me that he was going to be hired, like no matter what, because everyone loved him. They were just talking about all the great stuff he did. Oh, he did this great thing and you had just won something at AAAI, I think, or maybe you got 18 papers in AAAI that year.

• 30:33Speaker 02

  I got the best paper award at AAAI for the crossword stuff.

• 30:36Charles Isbell?

  Right, exactly. So that had all happened and everyone was going on and on and on about it. Actually, so Tinder was saying incredibly nice things about you.

• 30:41Speaker 02

  Really?

• 30:42Charles Isbell?

  Yes.

• 30:42Speaker 02

  He can be very grumpy.

• 30:44Charles Isbell?

  Yes.

• 30:44Speaker 02

  That's very, that's nice to hear.

• 30:45Charles Isbell?

  He was grumpily saying very nice things.

• 30:47Speaker 02

  Oh, that's, that makes sense.

• 30:48Charles Isbell?

  Yeah, it does make sense. So, you know, so he was gonna come. So why were we, why was I meeting him? I had something else I had to do. I can't remember what it was. Probably involved.

• 30:55Speaker 02

  So he remembers meeting me as inconveniencing his afternoon.

• 30:58Charles Isbell?

  So he came, so eventually came to my office. I was in the middle of trying to do something. I can't remember what. And he came and he sat down and for reasons that are purely accidental, despite what Michael thinks, my desk at the time was set up. In such a way that had sort of an L shape and the chair on the outside was always lower than the chair that I was in. And, you know, the kind of point was to-

• 31:17Speaker 02

  The only reason I think that it was on purpose is because you told me it was on purpose.

• 31:20Charles Isbell?

  I don't remember that. Anyway, the thing is that, you know, it kind of-

• 31:23Speaker 02

  His guest chair was really low so that he could, he could look down at everybody.

• 31:27Charles Isbell?

  The idea was just to simply create a nice environment that you were asking for a mortgage and I was gonna say no. That was the point. Very simple idea here. Anyway, so we sat there and we just talked for a little while and I think he got the impression that I didn't like him. It wasn't true.

• 31:39Speaker 02

  I strongly got that impression.

• 31:40Charles Isbell?

  The talk was really good and he was full of excitement.

• 31:42Speaker 02

  The talk, by the way, was terrible. And after, right after the talk, I said to my host, Michael Kearns, who ultimately was my boss.

• 31:47Lex Fridman?

  I'm a huge, I'm a friend and a huge fan of Michael, yeah.

• 31:50Speaker 02

  Yeah, he is a remarkable person. I, after my talk, I went into the-

• 31:55Charles Isbell?

  Irritating with basketball.

• 31:57Lex Fridman?

  I went- Racquetball, he's good at everything.

• 31:58Charles Isbell?

  No, basketball.

• 31:59Lex Fridman?

  No, but basketball, racquetball too.

• 32:00Charles Isbell?

  Squash.

• 32:01Lex Fridman?

  Squash, squash, squash, not racquetball.

• 32:02Charles Isbell?

  Yeah, squash, which is not. Racquetball, yes. Squash, no. And I hope you, you hear that, Michael.

• 32:08Lex Fridman?

  You mean in terms of-

• 32:10Speaker 02

  Of Michael Kearns.

• 32:11Lex Fridman?

  As a game, not his skill level, because I'm pretty sure he's-

• 32:13Speaker 02

  All right, there's some competitiveness there. But the point is that it was like the middle of the day. I had a full day of interviews. Like, I met with people, but then in the middle of the day, I gave a job talk. And then, and then there was gonna be more interviews. But I, I pulled Michael aside and I said, I think it's in both of our best interests if I just leave now. Because that was so bad that it'd just be embarrassing if I have to talk to any more people. Like, you look bad for having invited me. Like, it's just, let's just forget this ever happened. So I don't think the talk went well.

• 32:46Charles Isbell?

  That's one of the most Michael Littman set of sentences I think I've ever heard. He did great. Or at least, everyone knew he was great, so maybe it didn't matter. I was there. I remember the talk. And I remember him being very much the way I remember him now, on any given week. So it was good. And we met and we talked about stuff. He thinks I didn't like him, but-

• 33:02Speaker 02

  Because he was so grumpy.

• 33:03Lex Fridman?

  Must have been the chair thing.

• 33:05Speaker 02

  The chair thing and the low voice, I think. Like, he obviously-

• 33:08Lex Fridman?

  And that like, that like slight, like skeptical look.

• 33:11Speaker 02

  Yes.

• 33:12Lex Fridman?

  You're like-

• 33:13Speaker 02

  I have no idea what you're talking about.

• 33:14Charles Isbell?

  Well, I probably didn't have any idea what you were talking about. Anyway, I liked him.

• 33:19Speaker 02

  He asked me questions. I answered questions. I felt bad about myself. It was a normal day.

• 33:24Charles Isbell?

  It was a normal day. And then he left.

• 33:26Lex Fridman?

  And then he left, and that's how you met.

• 33:27Speaker 02

  Can we take a- And then I got hired and I was in the group.

• 33:30Lex Fridman?

  Can we take a slight tangent on this topic of, it sounds like, maybe you could speak to the bigger picture. It sounds like you're quite self-critical.

• 33:39Speaker 02

  Who, Charles?

• 33:40Lex Fridman?

  No, you.

• 33:41Speaker 02

  Oh, I think I can, I can do better. I can do better. I'll, I'll, try me again. I'll, I'll do better. I'll be so self-critical. I won't, I won't.

• 33:48Lex Fridman?

  Yeah, that, that was like a, like a three out of 10 response. So, uh, so let's, let's try to work it up to five and six. You know, I remember Marvin Minsky said, uh, on, on a video interview, something that the key to success in academic research is to hate everything you do.

• 34:08Lex Fridman?

  For some reason-

• 34:09Speaker 02

  I think I followed that because I hate everything he's done.

• 34:13Lex Fridman?

  Uh, that's a good line. That's a, that's a success.

• 34:17Speaker 02

  Maybe that's a keeper, but, um-

• 34:19Lex Fridman?

  But do, do, do you find that resonates with you at all in, in how you think about talks and so on?

• 34:24Charles Isbell?

  I would say it differently. It's not that.

• 34:26Speaker 02

  No, not really.

• 34:27Charles Isbell?

  That's such an MIT view of the world, though. So I remember, I, I remember talking about this when, uh, as a student, you know, you were basically told, uh, I will, I will clean it up for the purposes of the podcast. Um, uh, my work is crap, my work is crap, my work is crap, my work is crap. Then you like go to a conference or something. You're like, everybody else's work is crap. Everybody else's work is crap. And you feel better and better about it.

• 34:46Speaker 02

  Yeah.

• 34:46Charles Isbell?

  Relatively speaking, and then you sort of keep working on it. I, I don't hate my work.

• 34:51Speaker 02

  That resonates with me.

• 34:52Charles Isbell?

  Yes. I've never hated my work, but I have, I have been dissatisfied with it. And I think being dissatisfied, being okay with the fact that you've taken a positive step, the derivative's positive. Maybe even the second derivative's positive. That's important because that's a part of the, the hope, right? But you have to, but I haven't gotten there yet. If, if that's not there, that I haven't gotten there yet, then, you know, it's hard to, it's hard to move forward, I think. So I buy that, which is a little different from hating everything that you do.

• 35:21Speaker 02

  Yeah. I mean, there's, there's things that I've done that I like better than I like myself. So it's, it's separating me from the work, essentially. So I think I, I am very critical of myself. But sometimes the work I'm really excited about, and sometimes I think it, it's kind of good.

• 35:35Charles Isbell?

  Right away. So I, I found the work that I've liked, that I've done. Most of it, I liked it in retrospect more when I was far away from it in time.

• 35:45Speaker 02

  I have to be fairly excited about it to get done.

• 35:49Charles Isbell?

  No, excited at the time, but then happy with the result or what, but years later, or even I might go back, you know what, that actually turned out to matter.

• 35:55Speaker 02

  Yeah, yeah.

• 35:55Charles Isbell?

  That turned out to matter, or oh gosh, it turns out I've been thinking about that. It's actually influenced all the work that I've done since without realizing it.

• 36:02Speaker 02

  Boy, that guy was smart.

• 36:03Charles Isbell?

  Yeah, that guy had a future. Yeah, I, yeah.

• 36:08Speaker 02

  He's going places.

• 36:09Charles Isbell?

  I think there's, so yeah, so I think there's something to it. I think there's something to the idea you gotta, you know, hate what you do, but it's not quite hate. It's just being unsatisfied and different people motivate themselves differently. I don't happen to motivate myself with self-loathing. I happen to motivate myself with something else.

• 36:23Lex Fridman?

  So you're able to sit back and be proud of, in retrospect, of the work you've done?

• 36:29Charles Isbell?

  Well, and it's easier when you can connect it with other people because then you can be proud of them.

• 36:33Lex Fridman?

  Proud of the people, yeah.

• 36:34Charles Isbell?

  And then the question.

• 36:35Lex Fridman?

  And then you can still safely hate yourself.

• 36:37Charles Isbell?

  Yeah, that's right. It's win-win, Michael. Or at least win-lose, which is what you're looking for.

• 36:43Lex Fridman?

  Oh, wow, there's so many brilliant lines in this.

• 36:46Speaker 02

  There's levels.

• 36:48Lex Fridman?

  So how did you actually meet meat?

• 36:50Charles Isbell?

  Yeah, Mike.

• 36:51Speaker 02

  So my, the way I think about it is, because we didn't do much research together at H and T. But, but then we all got laid off, so, so that was.

• 37:01Lex Fridman?

  By the way, sorry to interrupt, but that was like one of the most magical places historically speaking.

• 37:07Charles Isbell?

  They did not appreciate what they had.

• 37:10Lex Fridman?

  And how do we, I feel like there's a profound lesson in there too. How do we get it, like what was, why was it so magical? Is this a coincidence of history or is there something special about?

• 37:20Speaker 02

  There were some really good managers and people who really believed in machine learning as this is going to be important. Let's get the, the people who are thinking about this in creative and, and insightful ways and put them in one place and stir.

• 37:34Charles Isbell?

  Yeah, but even beyond that, right? It's, it was. It was Bell Labs at its heyday and even when we were there, which I think was past its

• 37:42Speaker 02

  And to be clear, he's gotten to be at Bell Labs. I never got to be at Bell Labs. I joined after that.

• 37:46Charles Isbell?

  Yeah, I showed up in 91 as a grad student. So I was there for a long time, every summer.

• 37:52Speaker 02

  So twice I worked for companies that had just stopped being Bell Labs.

• 37:55Charles Isbell?

  Right.

• 37:56Speaker 02

  Bellcore and then AT&T Labs.

• 37:58Lex Fridman?

  So Bell Labs was several locations or for the for the researcher as a what like

• 38:02Speaker 02

  Definitely several.

• 38:03Lex Fridman?

  Jerseys involved somehow?

• 38:05Speaker 02

  They're all in Jersey.

• 38:06Charles Isbell?

  Yeah, they're all over the place.

• 38:07Speaker 02

  But they were in a couple of places in Jersey.

• 38:08Charles Isbell?

  Murray Hill was the Bell Labs place.

• 38:12Speaker 02

  So you you had you had an office at Murray Hill at one point in your career?

• 38:15Charles Isbell?

  Yeah, I played ultimate frisbee on the cricket pitch at Bell Labs at Murray Hill. And then it became AT&T Labs when it split off with during what we called a trivestiture.

• 38:25Lex Fridman?

  Are you better than Michael Kearns at ultimate frisbee?

• 38:28Speaker 02

  Yeah.

• 38:28Charles Isbell?

  Oh, yeah.

• 38:29Speaker 02

  Okay. But I think that was not boasting. I think that I think Charles plays a lot of ultimate. And I don't think Mike.

• 38:34Charles Isbell?

  No, I was. Yes, but but that wasn't the point. The point is, yes.

• 38:37Speaker 02

  Sorry.

• 38:37Charles Isbell?

  I'm finally better.

• 38:38Speaker 02

  Yes, sorry. Okay, I have played on a championship winning ultimate frisbee team or whatever ultimate team with Charles. So I know how good he is. He's really good.

• 38:47Charles Isbell?

  How good I was anyway when I was younger. But the thing is.

• 38:50Speaker 02

  I know how young he was when he was younger.

• 38:51Charles Isbell?

  That's true. That is true.

• 38:52Speaker 02

  So much younger than now.

• 38:53Lex Fridman?

  He's older now.

• 38:54Charles Isbell?

  Yeah, I'm older. Michael was a much was a much better basketball player than I was.

• 38:57Speaker 02

  Michael Kearns.

• 38:58Charles Isbell?

  Yes. No, not Michael. Let's be very clear.

• 39:01Speaker 02

  To be clear, I've not played basketball with you. So you don't know how terrible I am. But you have a probably pretty good guess.

• 39:06Charles Isbell?

  That you're not as good as Michael Kearns.

• 39:08Speaker 02

  He's tall and.

• 39:10Charles Isbell?

  And he cared about it. He's very athletic. He's very good.

• 39:12Speaker 02

  And probably competitive.

• 39:13Charles Isbell?

  I love hanging out with Michael. Anyway, but we were talking about something else, although I no longer remember what it was. What were we talking about?

• 39:18Lex Fridman?

  Oh, Bell Labs.

• 39:19Speaker 02

  Oh, Bell Labs.

• 39:19Charles Isbell?

  But also Labs. So this was kind of cool about what was magical about it. The first thing you have to know is that Bell Labs was an arm of the government, right? Because AT&T was an arm of the government. It was a monopoly. Uh, and you know, every month you paid a little thing on your phone bill, which turned out was a tax for like all the research that Bell Labs was doing. And you know, they invented transistors and the laser and whatever else is

• 39:42Speaker 02

  The Big Bang or whatever, the cosmic background radiation.

• 39:45Charles Isbell?

  Yeah, they did all that stuff. They had some amazing stuff with directional microphones. By the way, I got to go in this room where they, they had all these panels and everything. And we would talk and one another and he'd move some panels around. And then he'd have me step two steps to the left. And I couldn't hear a thing he was saying because nothing was bouncing off the walls. And then he would shut it all down and you could hear your heartbeat.

• 40:04Speaker 02

  Yeah.

• 40:05Charles Isbell?

  Deeply disturbing to hear your heartbeat. You can feel it. I mean, you can feel it now. There's so much all this sort of noise around. Anyway, Bell Labs is about pure research. It was a university, in some sense, the purest sense of a university, but without students. So it was all the faculty working with one another. And students would come in to learn. They would come in for three or four months, you know, during the summer and they would go away. But it was just this kind of wonderful experience. I could walk out my door. In fact, I would often have to walk out my door and deal with Rich Sutton and Michael Kearns yelling at each other about whatever it is they were yelling about. The proper way to prove something or another. And I could just do that and Dave McAllister and even and Peter Stone and, and all of these other people, including Sutender and then eventually Michael. And it was just a place where you could think thoughts.

• 40:49Charles Isbell?

  And it was okay because so long as once every 25 years or so, somebody invented a transistor, it paid for everything else. You could afford to take the risk. And then when that all went away, it became harder and harder and harder to justify it as far as the folks who were very far away were concerned. And there was such a fast turnaround among middle management on the AT&T side that you never had a chance to really build a relationship. At least people like us didn't have a chance to, to, to build a relationship. So when the DASRA happened, it was amazing, right?

• 41:19Speaker 02

  Yeah.

• 41:20Charles Isbell?

  Everybody left and I, I think everybody ended up at a, at a great place and made, made a huge, made a, continue to do really good work with, with machine learning. But it was a wonderful place and people will ask me, you know, what's the best job you've ever had? And as a professor, anyway, the answer that I would give is, well.

• 41:37Charles Isbell?

  Probably Bell Labs in some very real sense, and I will never have a job like that again because Bell Labs doesn't exist anymore. And you know, Microsoft Research is great and Google does good stuff and you can pick, you can tell if you want to, but Bell Labs was magical. It was around for, it was an important time and it represents a high watermark in, in basic research in the US.

• 41:57Lex Fridman?

  Is there something you could say about the physical proximity and the chance collisions? Like, we live in this time of the pandemic where everyone is maybe trying to see the silver lining and accepting the remote nature of things. Is, is there, one of the things that people like faculty that I talk to miss is the procrastination. Like the chance to, like, everything is about meetings that are supposed to be, there's not a chance to just, you know, talk about comic book or whatever, like go into discussion that's totally pointless.

• 42:31Charles Isbell?

  So it's funny you say this because that's how we met, met. It's exactly that. So I'll let Michael say that, but I'll just add one thing, which is just that, you know, research is a social process. And it helps to have random social interactions, even if they don't feel social at the time. That's how you get things done. One of the great things about the AI lab when I was there, I don't quite know what it looks like now, once they moved buildings, but we had entire walls that were whiteboards and people would just get up there and they would just write and people would walk up and you'd have arguments and you'd explain things to one another and you got so much out of the freedom to do that. You had to be okay with people challenging every frickin' word you said, which I would sometimes find deeply irritating, but most of the time it was, it was quite useful.

• 43:14Charles Isbell?

  But the sort of pointlessness and the interaction was, in some sense, the point, at least for me.

• 43:19Lex Fridman?

  Yeah, I mean, you, I think offline yesterday I mentioned Josh Tenenbaum and he's very much, he put, he's, man, he's such an inspiration in, in the childlike way that he pulls you in on any topic. It doesn't even have to be about machine learning or, or the brain. He'll just pull you into a closest, rightable surface, which is still, you can find whiteboards at MIT everywhere. And, and just like. Like basically cancel all meetings and talk for a couple hours about some some aimless thing and it it feels like the whole world, the time, space continuum kind of warps and that becomes the most important thing. And then it's just

• 43:58Charles Isbell?

  It's so true.

• 43:59Lex Fridman?

  It's it's it's definitely something worth missing in this in this world where everything's remote. There's some magic to the physical presence.

• 44:07Charles Isbell?

  Whenever I wonder myself whether MIT really is as great as I remember it, I just go talk to Josh.

• 44:12Lex Fridman?

  Yeah, you know, that's funny is there's a few people in this world that carry the the best of what particular institutions stand for, right? And this

• 44:21Charles Isbell?

  It's Josh. I mean, I don't I my guess is he's unaware of this.

• 44:25Lex Fridman?

  That's the point. That the masters are not aware of their mastery. So

• 44:32Charles Isbell?

  How did you meet?

• 44:34Lex Fridman?

  Yes, but but first a tangent. No. How did you meet me?

• 44:39Speaker 02

  So I'm not sure what you were thinking of, but I my when it started to dawn on me that maybe we had a longer term bond was after we all got laid off. And you had decided at that point that there that we were we're still paid. We were given an opportunity to like do job search and kind of make a transition. But it was clear that we were done. And I would go to my office to work and you would go to my office to keep me from working. That was that was my recollection of it. You had decided that there was no really no point in working for the company because the company our relationship with the company was was done.

• 45:14Charles Isbell?

  Yeah, but remember I felt that way beforehand. It wasn't about the company. It was about the set of people there doing really cool things. And it always always been that way. But we were working on something together.

• 45:22Speaker 02

  Oh, yeah, yeah, that's right. So at the very end, we all got laid off. But then our boss came to our boss's boss came to us because our boss was Michael Kearns and he had jumped ship brilliantly, like perfect timing, like things like right before the ship was about to sink. He was like, gotta go and and and landed perfectly because Michael Kearns.

• 45:42Charles Isbell?

  Because Michael Kearns.

• 45:43Speaker 02

  And the leaving the rest of us to go like, this is fine. And then it was clear that it wasn't fine. And we were all toast. So we had this sort of long period of time. But then our boss figured out, okay, wait, maybe we can save a couple of these people. If we can have them do something really useful. And the useful thing was, we were going to make a, basically, an automated assistant that could help you with your calendar. You could like, tell it things, and it would, it would respond appropriately, it would just kind of integrate across all sorts of your personal information. And so, me and Charles and Peter Stone were this, were set up as the crack team to actually solve this problem. Other people maybe were too

• 46:26Speaker 02

  theoretical that they thought, and, and, but we could actually get something done. So we sat down to get something done, and there wasn't time, and it wouldn't have saved us anyway. And so it all kind of went downhill. But, the interesting, I think, coda to that is that our boss's boss is a guy named Ron Brockman, and he, when he left AT&T, because we were all laid off, he went to DARPA, started up a program there that became Kalo, which is the program from which Siri sprung, which is a digital assistant that helps you with your calendar and a bunch of other things. It really, you know, in some ways, got its start with me and Charles and Peter trying to implement this vision

• 47:09Speaker 02

  that Ron Brockman had, that he ultimately got implemented through his role at DARPA. So, when I'm trying to feel less bad about having been laid off from what is possibly the greatest job of all time, I think about, well, we kind of helped birth Siri. That's something.

• 47:26Charles Isbell?

  And he did other things, too. But, the, we got to spend a lot of time in his office, and talk about.

• 47:32Speaker 02

  We got to spend a lot of time in my office, yeah. Yeah, yeah. And so, so then we went on our merry way. Everyone went to different places. Charles landed at Georgia Tech, which was what he always dreamed he would do. And so, that worked out well. I came up with a saying at the time, which is, luck favors the Charles. It's kind of like, luck favors the prepared. But Charles, like, like, he wished something, and then it would basically happen just the way he wanted. It was, it was inspirational to see things go that way.

• 48:03Charles Isbell?

  Things worked out.

• 48:03Speaker 02

  And we stayed in touch, and then... I think it really helped when you were working on, I mean, you'd kept me in the loop for things like threads and the work that you were doing at Georgia Tech. But then when they were starting their online master's program, he knew that I was really excited about MOOCs and online teaching. And he's like, I have a plan. And I'm like, tell me your plan. He's like, I can't tell you the plan yet because they were deep in negotiations between Georgia Tech and Udacity to make this happen and they didn't want it to leak. So Charles would kept teasing me about it, but wouldn't tell me what was actually going on. And eventually it was announced and he said, I would like you to teach the machine learning course with me. I'm like, that can't possibly work. But it was a great idea and it was, it was super fun. It was a lot of work to put together, but it was, it was really great.

• 48:48Lex Fridman?

  Was that the first time you thought about, first of all, was it the first time you got seriously into teaching?

• 48:54Speaker 02

  I mean, you know, I was a professor.

• 48:56Lex Fridman?

  This is already, this is already after you jumped to, so like, there's a little bit of jumping around in time.

• 49:03Speaker 02

  Yeah, sorry about that.

• 49:04Charles Isbell?

  There's a pretty big jump in time.

• 49:05Lex Fridman?

  So like the MOOCs thing.

• 49:07Speaker 02

  So Charles got to Georgia Tech and he, I mean, maybe Charles, maybe this is a Charles story.

• 49:10Charles Isbell?

  I got to Georgia Tech in 2002.

• 49:12Speaker 02

  He got to Georgia Tech in 2002. And, but then, and worked on things like revamping the curriculum, the undergraduate curriculum, so that it had some kind of semblance of modular structure because computer science was at the time, moving from a fairly narrow, specific set of topics to touching a lot of other parts of, of intellectual life. And the curriculum was supposed to reflect that. And so Charles played a big role in, in kind of redesigning that. And then.

• 49:40Charles Isbell?

  And for my, and for my, my labors, I ended up as associate dean.

• 49:45Speaker 02

  Right. He got to become associate dean of, in charge of educational stuff.

• 49:49Lex Fridman?

  This should be a valuable lesson. If you're good at something, they will give you responsibility to do more of that thing.

• 49:58Charles Isbell?

  Well.

• 49:58Lex Fridman?

  Until you.

• 49:59Speaker 02

  Don't show competence.

• 50:00Lex Fridman?

  Don't show competence if you.

• 50:01Charles Isbell?

  Well, you know what they say.

• 50:02Lex Fridman?

  Responsibility.

• 50:03Charles Isbell?

  Here's what they say.

• 50:04Lex Fridman?

  Yeah.

• 50:05Charles Isbell?

  The reward for good work is more work.

• 50:07Lex Fridman?

  Yeah.

• 50:08Charles Isbell?

  The reward for bad work is less work.

• 50:11Charles Isbell?

  Which, I don't know, depending about what you're trying to do that week, one of those is better than the other.

• 50:15Lex Fridman?

  Well, one of the problems with the word work, sorry to interrupt, is that it's seems to be an antonym in this particular language. We have the opposite of happiness. But it seems like they're they're like, that's one of, you know, we talked about balance. It's a it's always like work, life balance. It always rubbed me the wrong way as a terminology. I know it's just words.

• 50:38Speaker 02

  Right, the opposite of work is play, but ideally work is play.

• 50:42Charles Isbell?

  Oh, I can't tell you how much time I'd spend, certainly in Isabel Labs, except for a few very key moments. As a professor, I would do this too. I would just say, I cannot believe they're paying me to do this. Because it's fun. It's something that I would I would do for a hobby if I could anyway. So that's what it worked out.

• 51:00Lex Fridman?

  You sure you want to be saying that when this is being recorded?

• 51:03Charles Isbell?

  As a dean, that is not true at all. I need a raise. But but I think here with this, that even though a lot of time passed, you know, Michael and I talked almost every, well, we texted almost every day during the period.

• 51:14Speaker 02

  Charles, at one point, took me, there was the ICML conference, the machine learning conference was in Atlanta. I was the chair, the general chair of the conference. Charles was my publicity chair, something like that, or fundraising chair. Yeah. But he decided it'd be really funny if he didn't actually show up for the conference in his own home city. So he didn't. But he did at one point pick me up at the conference in his Tesla and drove me to the Atlanta mall and forced me to buy an iPhone because he didn't like how it was to text with me and thought it would be better for him if I had an iPhone. The text would be somehow smoother.

• 51:54Charles Isbell?

  And it was.

• 51:55Speaker 02

  And it was.

• 51:56Charles Isbell?

  And it is, and his life is better.

• 51:57Speaker 02

  And my life is better. And so, yeah, but but it was, yeah, Charles forced me to get an iPhone so that he could text me more efficiently. I thought that was an interesting moment.

• 52:06Charles Isbell?

  It works for me. Anyway, so we kept talking the whole time. And then eventually we did the we did the teaching thing and it was great. And there's a couple of reasons for that, by the way. One is, I really wanted to do something different. Like, you've got this medium here. People claim it can change things. What's a thing that you could do in this medium that you could not do otherwise? Besides edit, right? I mean, what could you do? And and being able to do something with another person was that kind of thing? It's very hard. I mean, you can take turns, but teaching together, having conversations is very hard. Right? So that was a cool thing. The second thing, give me an excuse to do more stuff with him.

• 52:37Speaker 02

  Yeah, I always thought he makes it sound brilliant. And it is, I guess. But it's at the time, it really felt like I've got a lot to do. Charles is saying, and it would be great if Michael could teach the course and I could just.

• 52:51Charles Isbell?

  Hang out.

• 52:52Speaker 02

  Yeah, just kind of coast on that.

• 52:54Charles Isbell?

  Well, that's what the second class was more like that. Because the second.

• 52:56Speaker 02

  Second class was explicit like that. But the first class, it was at least half.

• 53:00Charles Isbell?

  Yeah, but I do all the stuff.

• 53:01Speaker 02

  The structure, the structure that we came up.

• 53:02Lex Fridman?

  Once again, letting the the facts get in the way.

• 53:05Speaker 02

  Good story.

• 53:06Lex Fridman?

  Good story.

• 53:07Speaker 02

  I should just let Charles talk to us.

• 53:09Charles Isbell?

  But that's the facts as he saw. But it so that was that was kind of.

• 53:12Lex Fridman?

  That's your facts.

• 53:13Charles Isbell?

  Yeah, that was sort of true for 7642, which is the reinforcement learning class, because that was really his class.

• 53:17Lex Fridman?

  You started with reinforcement learning?

• 53:18Charles Isbell?

  No, we started with I did intro machine learning, 7641, which is supervised learning, unsupervised learning and reinforcement learning and decision making. Cram all that in there, the kind of assignments that we talked about earlier. And then eventually, about a year later, we did a follow on 7642, which is reinforcement learning and decision making. The first class was based on something I had been teaching at that point for well over a decade. And the second class was based on something Michael had been teaching. Actually, I learned quite a bit teaching that class with him. But he drove most of that. But the first one, I drove most of it was all my material, although I had stolen that material originally from slides I found online from Michael, who had originally stolen that material from, I guess, slides he found online, probably from Andrew Moore, because the jokes were the same anyway. At least some of the, at least when I found the slides, some of the stuff was there.

• 54:02Charles Isbell?

  Yes, every machine learning class taught in early 2000s stole from Andrew Moore.

• 54:06Speaker 02

  Particular joke or two.

• 54:08Charles Isbell?

  The at least the structure. Now, I did, and he did actually a lot more with reinforcement learning and such and game theory and those kinds of things. But, you know, we all sort of.

• 54:16Lex Fridman?

  You mean in the research world?

• 54:17Speaker 02

  No, no, no.

• 54:18Charles Isbell?

  No, I mean, in teaching that class.

• 54:19Speaker 02

  The coverage was different than.

• 54:21Charles Isbell?

  Than what other people were doing. Most people were just doing supervised learning and maybe a little bit of, you know, clustering and whatnot. But we took it all the way to.

• 54:27Speaker 02

  A lot of it just comes from Tom Mitchell's book.

• 54:29Charles Isbell?

  Oh, no. Yeah, except well, half of it comes from Tom Mitchell's book, right? But the other half doesn't. This is why it's all readings, right? Because certain things weren't invented when Tom wrote this.

• 54:38Speaker 02

  Yeah, okay, that's true.

• 54:39Charles Isbell?

  Right? But it was quite good. But there's a reason for that besides, you know, just I wanted to do it. I wanted to do something new and I wanted to do something with him, which is a realization, which is despite what you might believe, he's an introvert and I'm an introvert, or I'm on the edge of being an introvert anyway. But both of us, I think, enjoy the energy of the crowd, right? There's something about talking to people and bringing them in to whatever we find interesting that is empowering, energizing or whatever. And I found the idea of staring alone at a computer screen and then talking off of materials less inspiring than I wanted it to be.

• 55:19Speaker 02

  And I had in fact done a MOOC for Udacity on algorithms. And it was a week in a dark room, talking at the screen, writing on the little pad. And I didn't know this was happening, but they had watched, the crew had watched some of the videos while, you know, like in the middle of this. And they're like, something's wrong. You're sort of shutting down. And I think a lot of it was I'll make jokes and no one would laugh.

• 55:48Lex Fridman?

  Yeah.

• 55:48Speaker 02

  And I felt like the crowd hated me. Now, of course, there was no crowd. So like it wasn't rational. But it's little, each time I tried it and I got no reaction, it just was taking the energy out of my performance, out of my presentation.

• 56:03Charles Isbell?

  Such a fantastic metaphor for grad school. Anyway, by working together, we could play off each other and have.

• 56:09Speaker 02

  And keep the energy up because you can't, you can't let your guard down for a moment with Charles. He'll just overpower you.

• 56:15Charles Isbell?

  I have no idea what you're talking about. But we would work really well together, I thought. And we knew each other, so I knew that we could sort of make it work. Plus, I was the associate dean, so they had to do what I told them to do. We had to do that, we had to make it work. And so it worked out very well, I thought, well enough that we.

• 56:29Speaker 02

  With great power comes great power.

• 56:31Charles Isbell?

  That's right. And we became smooth and curly. And that's when we did the overfitting thriller video.

• 56:40Speaker 02

  Yeah, we, yeah, yeah, that's a thing.

• 56:42Lex Fridman?

  So what, can we just like smooth and curly? Where would that come from?

• 56:46Speaker 02

  Okay, so that happened, it was completely spontaneous.

• 56:49Lex Fridman?

  These are nicknames you go by.

• 56:50Speaker 02

  Yeah.

• 56:50Charles Isbell?

  It's what the students call us.

• 56:53Speaker 02

  He was lecturing. So the way that we structured the lectures is one of us is the lecturer and one of us is basically the student. And so the, he was lecturing on-

• 57:02Charles Isbell?

  The lecturer prepares all the materials, comes up with the quizzes, and then the student comes in not knowing anything. So it's, you know, just like being on campus.

• 57:09Speaker 02

  Yeah.

• 57:10Charles Isbell?

  And I was doing game theory in particular.

• 57:12Speaker 02

  Yeah.

• 57:12Charles Isbell?

  The prisoner's dilemma.

• 57:13Speaker 02

  Prisoner's dilemma. And so he needed to set up a little prisoner's dilemma grid. So he drew it and I could see what he was drawing. And the prisoner's dilemma consists of two players, two parties. So he decided he would make little cartoons of the two of us. And so there was two criminals, right, that were deciding whether or not to rat each other out. One of them, he drew as, you know, a circle with a smiley face and a kind of goatee thing. Smooth head. And the other one with all sorts of curly hair. And he said, this is smooth and curly. I said, smooth and curly? He said, no, no, smooth with a V. It's very important that it have a V.

• 57:48Lex Fridman?

  And that stuck. I actually watched that video.

• 57:49Speaker 02

  And then the students really, the students really took to that. Like they've related, they found that relatable.

• 57:53Charles Isbell?

  He started singing Smooth Criminal by Michael Jackson.

• 57:55Lex Fridman?

  Yeah, yeah, yeah. And that, those names stuck.

• 57:58Speaker 02

  So that, so we now have a video series, the, an episode, our kind of first actual episode should be coming out today. Smooth and Curly on Video, where the two of us discuss West, episodes of Westworld. We watched Westworld and we're like, huh, what does this say about computer science and AI?

• 58:16Charles Isbell?

  And we've never, we did not watch it. I mean, I know it's on season three or whatever. We haven't. As of this recording, it's on season three. And-

• 58:22Speaker 02

  We've watched now two episodes total.

• 58:24Charles Isbell?

  Yeah, I think I watched three.

• 58:25Lex Fridman?

  What do you think about Westworld?

• 58:27Speaker 02

  Two episodes in, so I could tell you so far-

• 58:30Lex Fridman?

  Yeah.

• 58:30Speaker 02

  I'm just guessing what's gonna happen next. It seems like bad things are gonna happen with the robots uprising.

• 58:35Lex Fridman?

  Boiler alert.

• 58:37Charles Isbell?

  So I have not, I have not, I mean, you know, I vaguely remember a movie existing, so I assume it's, it's related to that, but-

• 58:43Speaker 02

  That was more my time than your time, Charles.

• 58:44Charles Isbell?

  That's right, because you're much older than I am. I think the important thing here is that it's narrative, right? It's all about telling a story. That's the whole driving thing. But the idea that they would give these reveries, that they would make people, they would make them-

• 58:56Speaker 02

  Let them remember.

• 58:57Charles Isbell?

  Remember the awful things that hap-

• 58:59Speaker 02

  Things that happened.

• 59:00Charles Isbell?

  Who could possibly think that was gonna, I gotta, I mean, I don't know, I've only seen the first two episodes or maybe the third one. I think I've only seen.

• 59:05Speaker 02

  You know what it was? You know what the problem is? That the robots were actually designed by Hannibal Lecter.

• 59:10Charles Isbell?

  That's true.

• 59:12Speaker 02

  So like, what do you think's gonna happen? Bad thing.

• 59:15Charles Isbell?

  It's clear that things are happening and characters being introduced and we don't yet know anything, but still, I was just struck by how it's all driven by narrative and story. And there's all these implied things like programming hap- the programming interface is talking to them about what's going on in their heads, which is both, I mean, artistically, it's probably useful to film it that way, but think about how it would work in real life. That just seems very crazy. But there was, we saw in the second episode, there's a screen, you could see things.

• 59:40Speaker 02

  They were wearing like.

• 59:41Charles Isbell?

  State in the world. It was quite interesting to just kind of ask this question so far. I mean, I assume it veers off into Never Never Land at some point, but.

• 59:48Speaker 02

  So we don't know, we can't answer that question.

• 59:50Lex Fridman?

  I'm also a fan of a guy named Alex Garland. He's a director of Ex Machina. And he is the first, I wonder if Kubrick was like this actually, is he like studies, what would it take to program in AI systems? Like he's, he's curious enough to go into that direction. On the Westworld side, I felt there was more emphasis on the narratives than like actually asking like computer science questions. Like, how would you build this? How would you?

• 1:00:21Charles Isbell?

  How would you debug it? I still think, to me, that's the, the key issue. They were terrible debuggers.

• 1:00:26Lex Fridman?

  Yeah.

• 1:00:27Speaker 02

  And well, they said specifically, so we make a change and we put it out in the world and that's bad because something terrible could happen. Like, if you're putting things out in the world and you're not sure whether something terrible is going to happen, you're probably, your process is probably.

• 1:00:37Charles Isbell?

  I just feel like there should have been someone whose sole job it was, was to walk around and poke his head in and say, what could possibly go wrong? Just over and over again.

• 1:00:45Lex Fridman?

  I would have loved if there was an, and I did watch a lot more, I'm not giving anything away. I would have loved it if there was like an episode where like, like the new intern is like debugging a new model or something and like it just keeps failing and and they're like, all right, and then it's more turns into like a episode of Silicon Valley or something like that.

• 1:01:03Charles Isbell?

  Yes.

• 1:01:03Lex Fridman?

  This is a, versus like this ominous AI systems that are constantly like threatening.

• 1:01:09Lex Fridman?

  The the fabric of this world that's been created.

• 1:01:11Speaker 02

  Yeah.

• 1:01:12Charles Isbell?

  Yeah. And you know, the other, the, this, this reminds me of something that, so I agree with that. That should be very cool, at least, well, for the small percentage of people who care about debugging systems. But the other thing is.

• 1:01:22Speaker 02

  Debugging, the series.

• 1:01:24Charles Isbell?

  It falls into, think of the sequels, fear of the debug.

• 1:01:26Speaker 02

  Oh, my gosh.

• 1:01:27Charles Isbell?

  And anyway, so.

• 1:01:29Speaker 02

  It's a nightmare show. It's a horror movie.

• 1:01:32Charles Isbell?

  I think that's where we lose people, by the way, early on, is the people who either decide, either figure out debugging or think debugging is terrible. This is.

• 1:01:37Speaker 02

  Where we lose people in computer science.

• 1:01:39Charles Isbell?

  This, this is part of the struggle versus suffering, right? You, you get through it and you kind of get the skills of it, or you're just like, this is dumb. And this is a dumb way to do anything. And I think that's when we lose people. But, um, I, well, I'll leave it at that. But I think that, I think that, that there's something really, really neat about framing it that way. But what I don't like about all of these, all of these things, and I love Tex Machina, by the way. Although the ending was very depressing.

• 1:02:06Lex Fridman?

  Well, again, one of the things I have to talk to Alex about, he says that the thing that nobody noticed. He put in is the, at the end, spoiler alert, the, the robot turns and looks at the camera and smiles.

• 1:02:23Charles Isbell?

  Right.

• 1:02:24Lex Fridman?

  Briefly. And to him, he thought that his definition of passing the touring, the general version of the Turing test or the consciousness test is smiling for no one.

• 1:02:37Speaker 02

  Hmm. Oh.

• 1:02:38Lex Fridman?

  Like, like, not, oh, you know, it's, it's like the Chinese room kind of experiment. It's not always trying to act for others.

• 1:02:47Charles Isbell?

  Right.

• 1:02:47Lex Fridman?

  But just on your own, being able to have a relationship with the actual experience and just like take it in. I don't know. He said like, nobody noticed.

• 1:02:56Charles Isbell?

  I mean, I.

• 1:02:56Lex Fridman?

  The magic of it.

• 1:02:57Charles Isbell?

  I have this vague feeling that I remember the smile, but now you, now you've just put the memory in my head. So probably not. But I do think that that's interesting, although by looking at the camera, you are smiling for the audience, right? You're breaking the fourth wall. It seems, I mean, well, that's a, that's a limitation of the medium, but I, I like that idea. But here's the problem I have with all of those movies, all of them, is that, but I know why it's this way. And I enjoy those movies and Westworld is, uh, it sets up the problem of AI as succeeding and then having something we cannot control, but it's. That's not the bad part of AI. The bad part of AI is the stuff we're living through now, right? It's the using the data to make decisions that are terrible. It's not the intelligence that's going to go out there and surpass us and, you know,

• 1:03:41Charles Isbell?

  take over the world or, you know, lock us into a room to starve to death slowly over multiple days. It's instead the the tools that we're building that are allowing us to make the terrible decisions we would have less efficiently made before, right? You know, computers are very good at making us more efficient, including being more efficient at doing terrible things. And and that's the part of the AI we have to worry about. It's not the, you know, true intelligence that we're going to build sometime in the future, probably long after we're around. But, you know, I I I just, I think that whole framing of it sort of misses the point, even though it is inspiring. And I was inspired by those ideas, right? That I got into this in part because I wanted to build something like that.

• 1:04:25Charles Isbell?

  Philosophical questions are interesting to me, but but, you know, that's not where the terror comes from. The terror comes from the everyday.

• 1:04:30Lex Fridman?

  And you can construct situations in the subtlety of the interaction between AI and the human, like with with social networks, all the stuff you're doing with interactive artificial intelligence. But, you know, I feel like Cal 9000 came a little bit closer to that when in 2001 Space Odyssey, because it felt like a personal assistant. You know, it felt like closer to the AI systems we have today and and the real things we might actually encounter, which is over relying in some fundamental way on our like dumb assistants or on social networks, like over offloading too much of us onto, you

• 1:05:14Lex Fridman?

  know, onto things that require internet and power and so on, and thereby becoming powerless as a standalone entity. And then when that thing starts to misbehave in some subtle way, it creates a lot of problems. And those problems are dramatized when you're in space because you don't have a way to walk away.

• 1:05:36Charles Isbell?

  Well, as the man said, once you, once we started making the decisions for you, it stopped being your world, right? That's the matrix, Michael, in case you don't.

• 1:05:43Lex Fridman?

  I didn't judge it.

• 1:05:44Charles Isbell?

  You don't remember. But on the other hand, I could say no, because isn't that what we do with people anyway? You know, this kind of the shared intelligence that is humanity is relying on other people constantly. I mean, we we hyper specialize, right? As individuals, we're still generally intelligent. We make our own decisions in a lot of ways, but we leave most of this up to other people. And that's perfectly fine. And by the way, everyone does necessarily share our goals. Sometimes they seem to be quite against us. Sometimes we make decisions that others would see as against our own interests. And yet we somehow manage it, manage to survive. I'm not entirely sure why an AI would actually make that worse.

• 1:06:22Charles Isbell?

  Or even different, really.

• 1:06:24Lex Fridman?

  You mentioned the matrix. Do you think we're living in a simulation?

• 1:06:29Speaker 02

  It does feel like a thought game more than a real scientific question.

• 1:06:35Lex Fridman?

  Well, I'll tell you why. Like, I think it's an interesting thought experiment to see what you think from a computer science perspective. It's a good experiment of how difficult would it be to create a sufficiently realistic world that us humans would enjoy being in?

• 1:06:51Lex Fridman?

  That's almost like.

• 1:06:52Speaker 02

  If we're living in a simulation, then I don't believe that we were put in the simulation. I believe that it's just physics playing out and we came out of that. Like, I don't, I don't, I don't think.

• 1:07:03Lex Fridman?

  So you think you have to build the universe?

• 1:07:05Speaker 02

  I think that the universe itself, we can think of that as a simulation. And in fact, what I try, sometimes I try to think about to understand what it's like for a computer to start to think about the world. I try to think about the world. Things like quantum mechanics, where it doesn't feel very natural to me at all. And it really strikes me as I don't understand this thing that we're living in. It has, there's weird things happening in it that don't feel natural to me at all. Now, if you want to call that as the result of a simulator, okay, I'm fine with that. But like, I don't.

• 1:07:39Lex Fridman?

  There's the bugs in the simulation.

• 1:07:41Charles Isbell?

  There's the bugs. I mean, the interesting thing about simulation is that it might have bugs. I mean, that's the thing that I, the.

• 1:07:47Speaker 02

  But there would be bugs for the people in the simulation. They're just, that's just reality.

• 1:07:51Charles Isbell?

  Unless you were aware enough to know that there was a bug. But I, I think.

• 1:07:55Lex Fridman?

  Back to the matrix.

• 1:07:56Charles Isbell?

  Yeah, the way you put the question.

• 1:07:57Speaker 02

  I don't think that we live in a, in a simulation created for us. Okay, I would say that.

• 1:08:01Charles Isbell?

  I think that's interesting. I've actually never thought about it that way. I mean, you, the way you asked the question, though, is could you create a world that is enough for us humans? It's an interestingly sort of self referential question because.

• 1:08:13Charles Isbell?

  The beings that created the simulation probably have not created the simulation that's realistic for them. But we're in the simulation and so it's realistic for us. So we could create a simulation that is fine for the people in the simulation, as it were, that would not necessarily be fine for us as the creators of the simulation.

• 1:08:29Lex Fridman?

  But, well, you can, you can forget, I mean, if when you go into the, if you play video games of virtual reality, you can, if there was some suspension of disbelief, or whatever. It becomes the world. It becomes the world, even like in brief moments, you, you forget that another world exists. I mean, that's what like good stories do, they pull you in. The question is, is it possible to pull, you know, our brains are limited. Is it possible to pull the brain in to where we actually stay in that world longer and longer and longer and longer? And like, not only that, but we don't want to leave. And so especially, this is the key thing about the developing brain, is if we journey into that world early on in life, often.

• 1:09:12Charles Isbell?

  How would you even know?

• 1:09:13Lex Fridman?

  Yeah. Yeah, so I, but like, from a video game design perspective, from a Westworld perspective, it's, I think, I think it's an important thing for even computer scientists to think about, because it's clear that video games are getting much better. And virtual reality, although it's been ups and downs, just like artificial intelligence, it feels like virtual reality will be here in a very impressive form if we were to fast forward 100 years into the future, in a way that might change society fundamentally. Like, if I were to, I'm very limited in predicting the future, as all of us are, but if I were

• 1:09:51Lex Fridman?

  to try to predict, like, in which way I'd be surprised to see the world 100 years from now, it'd be that, or impressed, it'd be that we're all no longer living in this physical world, that we're all living in a virtual world.

• 1:10:09Charles Isbell?

  You really need to be calculating God. By Sawyer.

• 1:10:15Charles Isbell?

  It's a, he'll, he'll read it in a night. It's a very easy read, but it's a, assuming you're that kind of reader. But it's a, it's a good story and it's kind of about this, but not in a way that it appears. And I really enjoyed the thought experiment. And I think it's pretty sure it's Robert Sawyer. But anyway, he's, he's apparently Canadians top science fiction writer, which is why the story mostly takes place in Toronto. But it's a, it's a very good, it's a very good sort of story that, that sort of imagines this. Very different kind of simulation hypothesis sort of thing from say, um, the egg. For example, you know, you know, I'm talking about the short story.

• 1:10:56Charles Isbell?

  By the guy who did The Martian. Who wrote The Martian? You know, I'm talking.

• 1:11:01Lex Fridman?

  The Martian.

• 1:11:01Speaker 02

  Matt Damon.

• 1:11:02Charles Isbell?

  No, the book.

• 1:11:04Lex Fridman?

  So we had this whole discussion that Michael doesn't, uh, doesn't partake in this exercise of reading.

• 1:11:10Charles Isbell?

  Yeah, he doesn't seem to like it, which seems very strange to me, considering how much he has to read. I read all the time. I used to read 10 books at every week when I was, when I was in sixth grade or whatever. I was a lot of it, science fiction, a lot of it, a lot of history that I, I love to read. But anyway, you should recalculate in God. I think you'll, you'll, it's very easy read. Like I said, I think you'll enjoy sort of the ideas that it presents.

• 1:11:33Lex Fridman?

  Yeah, I think the, the thought experiment is quite interesting. One thing I've noticed about people growing up now, I mean, we talked about social media, but video games is a much bigger, bigger, bigger and bigger part of their lives. And then, and the video games have become much more realistic. I think it's possible that the three of us are not, uh, maybe the two of you are not familiar exactly with the numbers we're talking about here. I think the number of people.

• 1:12:01Charles Isbell?

  It's bigger than movies, right? It's, it's, it's huge. I used to do a lot of the narrative, computational narrative stuff.

• 1:12:07Speaker 02

  I understand that economists can actually see the impact of video games on the labor market. That, that there are, there, there's fewer young men of a certain age participating in like paying jobs than you'd expect. And, and that they trace it back to video games.

• 1:12:25Charles Isbell?

  I mean, the problem with Star Trek was not warp drive or teleportation. It was the holodeck. Like if you have the holodeck.

• 1:12:35Charles Isbell?

  That's it. That's it. You go in the holodeck, you never come out. I mean, it just never made. Once I saw that, I thought, okay, well, so this is the end of humanity as we know it, right? They've invented the holodeck.

• 1:12:46Lex Fridman?

  Because that feels like the singularity, not some AGI or whatever. It's some possibility to go into another world that can be artificially made better than this one.

• 1:12:55Charles Isbell?

  Mm hmm. And slowing it down so you live forever, or speeding it up so you appear to live forever, or making the decision of when to die.

• 1:13:03Lex Fridman?

  And then most of us will just be old people on the porch yelling at the kids these days in their virtual reality worlds.

• 1:13:11Speaker 02

  But they won't hear us because they've got headphones on.

• 1:13:14Lex Fridman?

  So, I mean, rewinding back to MOOCs, is there lessons that you've, speaking of kids these days?

• 1:13:22Speaker 02

  There you go. That was a transition.

• 1:13:23Charles Isbell?

  That was fantastic.

• 1:13:25Lex Fridman?

  I'll fix it in post.

• 1:13:28Speaker 02

  That's Charles's favorite phrase.

• 1:13:31Lex Fridman?

  Fix it in post?

• 1:13:31Speaker 02

  Fix it in post.

• 1:13:32Charles Isbell?

  Fix it in post. It's all, when we were recording, all the time, whenever the editor didn't like something or whatever, I would say, we'll fix it in post. He hated that. He hated that more than anything.

• 1:13:41Speaker 02

  Because it was Charles's way of saying, I'm not going to do what you can. You know, you're on your own for this one.

• 1:13:46Charles Isbell?

  But it always got fixed in post.

• 1:13:48Lex Fridman?

  Exactly. So is there something you've learned about, I mean, it's interesting to talk about MOOCs. Is there something you've learned about the process of education, about thinking about the present? I think there's two lines of conversation to be had here is the future of education in general that you've learned about, and more presciently is the education in the times of COVID.

• 1:14:15Charles Isbell?

  Yeah. The second thing in some ways matters more than the first for, at least in my head for the, not just because it's happening now, but because I think it's, it's reminded us of a lot of things. Coincidentally, today, there's an article out by a good friend of mine, who's also a professor at Georgia Tech, but more importantly, a writer and editor at The Atlantic, I mean, Ian Bogos. And the title is something like, Americans Will Sacrifice Anything For The College Experience. And it's about why we went back to college and why people wanted us to go back to college. And it's not, you know, greedy presidents trying to get the last dollar from someone. It's because they want to go to college and what they're paying for is not the classes. What they're paying for is the college experience. It's not the education, it's being there.

• 1:14:58Charles Isbell?

  I've believed this for a long time, that we continually make this mistake of people want to go back to college as being people want to go back to class. They don't, they want to go back to campus. They want to move away from home. They want to do all those things that people experience. It's a rite of passage. It's a, it's a identity, if I can, if I can steal some of Ian's words here. And I think that's right. And I think what we've learned through COVID is it has made it, the disaggregation was not the disaggregation of the education from the place, the university place, and that you can get the best anywhere you want to. In terms of, there's lots of reasons why that is not necessarily true. The disaggregation is having it shoved in our faces that the reason to go, again, that

• 1:15:39Charles Isbell?

  the reason to go to college is not necessarily to learn, it's to have the college experience. And that's very difficult for us to accept, even though we behaved that way, most of us, when we were undergrads. Hmm. You know, a lot of us didn't go to every single class. We learned and we got it and we look back on it and we're happy we had the learning experience as well, obviously particularly us, because this is the, the kind of thing that we do. And my guess is that's true of, of the vast majority of your audience. But that doesn't mean the, I'm standing in front of you telling you this is the thing that people are excited about. Hmm. And that's why they want to be there, primarily why they want to be there. So to me, that's what COVID has forced us to deal with, even though I think we're still all in deep denial about it and hoping that it'll go back to that.

• 1:16:24Charles Isbell?

  And I think about 85% of it will, we'll be able to pretend that that's really the way it is again, and we'll forget the lessons of this. But technically what will come out of it. Yeah. Or technologically what will come out of it is a way of providing a more dispersed experience through online education and these kinds of, of remote things that we've learned. And we'll have to come up with new ways to engage them. In the experience of college, which includes not just the parties or the whatever kids do, but the learning part of it, so that they actually come out four, or five, or six years later with having actually, having actually learned something. So I think the world will be radically different afterwards, and I think technology will matter for that, just not in the way that the people who were building the technology originally imagined it would be.

• 1:17:06Charles Isbell?

  And I think this would have been true even without COVID, but COVID has accelerated that reality. So it's happening in two or three years, or five years, as opposed to 10 or 15.

• 1:17:16Speaker 02

  That was an amazing answer that I did not understand.

• 1:17:20Charles Isbell?

  So.

• 1:17:21Speaker 02

  It was passionate and.

• 1:17:22Charles Isbell?

  Shots fired.

• 1:17:23Speaker 02

  But I don't know, I just didn't, no, I'm not trying to criticize it. I think I'm, I don't think I'm getting it. So you mentioned disaggregation.

• 1:17:29Charles Isbell?

  Mm hmm.

• 1:17:30Speaker 02

  So what's that?

• 1:17:31Charles Isbell?

  Well, so, you know, the power, the power of technology that if you go on the West Coast and hang out long enough is all about, we're going to disaggregate these things together, the books from the bookstore, you know, that kind of a thing. And then suddenly Amazon controls the universe, right? And technology is a disruptor, right? And people have been predicting that for higher education for a long time, but certainly in.

• 1:17:48Speaker 02

  So is this, is this the sort of idea like students can aggregate on a campus someplace and then take classes over the network anywhere?

• 1:17:58Charles Isbell?

  Yeah, this is what people thought was going to happen, or at least people claimed was going to happen, right? That, you know.

• 1:18:01Speaker 02

  Because my daughter is essentially doing that now. She's on one campus, but learning in a different campus.

• 1:18:05Charles Isbell?

  Sure, and COVID makes that possible, right? COVID makes that.

• 1:18:09Speaker 02

  Legal.

• 1:18:10Charles Isbell?

  All but avoidable, right? But, but the idea originally was that, you know, you and I were going to create this machine learning class, and it was going to be great, and then no one else would be the machine learning class everyone takes, right? That was never going to happen, but, you know, something like that, you could see.

• 1:18:22Speaker 02

  But I feel like you didn't address that. So why, why, why is it that, why, why?

• 1:18:26Charles Isbell?

  I don't think that will be the thing that happens.

• 1:18:28Speaker 02

  So the college experience, maybe I, maybe I missed what the college experience was. I thought it was peers, like people hanging around.

• 1:18:34Charles Isbell?

  A large part of it is peers. Well, it's peers and independents.

• 1:18:38Speaker 02

  Yeah, but none of that, you can do classes online for all of that.

• 1:18:42Charles Isbell?

  No, no, no, because.

• 1:18:43Speaker 02

  No, definitely.

• 1:18:44Charles Isbell?

  We're social people, right?

• 1:18:45Speaker 02

  So when we take the classes, that also has to be part of an experience.

• 1:18:49Charles Isbell?

  It's in a context, and the context is the university. And by the way, it actually matters that Georgia Tech really is different from Brown.

• 1:18:57Speaker 02

  I see, because then students can choose the kind of experience they think is going to be best for them.

• 1:19:02Charles Isbell?

  Okay, I, I think we're giving too much agency to the students in making an informed decision.

• 1:19:07Speaker 02

  Okay.

• 1:19:07Charles Isbell?

  But the truth, but yes, they will make choices and they will have different experiences and some of those choices will be made for them. Some of them will be choices they're making because they think it's this, that or the other. I just don't want to say, I don't want to give the idea.

• 1:19:18Speaker 02

  It's not homogeneous.

• 1:19:19Charles Isbell?

  Yes, it's certainly not homogeneous, right? I mean, Georgia Tech is different from Brown. Brown is different from pick your favorite state school in Iowa, Iowa State, okay? Which I guess is my favorite state school in Iowa.

• 1:19:32Speaker 02

  Sure.

• 1:19:32Charles Isbell?

  But, you know, these are all different. They have different contexts and a lot of those contexts are, they're about history, yes, but they're also about the location of where you are. They're about the larger group of people who are around you, whether you're in Athens, Georgia, and you're basically the only thing that's there. As a university, you're responsible for all the jobs, or whether you're at Georgia State University, which is an urban campus. Where you're surrounded by, you know, six million people in your campus, where it ends and begins in the city, ends and begins, we, we don't know. It, it actually matters whether it's a small campus or a large campus, I mean, these things matter.

• 1:20:03Lex Fridman?

  Why is it that if you go to Georgia Tech, you're like forever proud of that? And you like say that to people at dinner, like bars and whatever? And if you, not to, you know, if you get a degree at an online university somewhere, you don't, that's not a thing that comes up at a bar.

• 1:20:23Charles Isbell?

  Well, it's funny you say that. So the students who take our online masters by several measures are more loyal than the students who come on campus, certainly for the master's degree. The reason for that, I think, and you'd have to ask them, but based on my conversations with them, I, I feel comfortable saying this. Is because this didn't exist before. I mean, we talk about this online masters and that it's reaching, you know, 11,000 students and that's an amazing thing. And we're admitting everyone we believe who can succeed. We got a 60% acceptance rate. It's amazing, right? It's also a $6,600 degree. The entire degree costs $6,600, $7,000, depending on how long you take, a dollar degree, as opposed to $46,000 that costs you to come on campus. So that feels, and I can do it while I'm working full time and I've got a family and a mortgage and all these other things.

• 1:21:08Charles Isbell?

  So it's an opportunity to do something you wanted to do, but you didn't think was possible. Without giving up two years of your life, as well as all the money and everything else in the life that you have built. So I think. We created something that's had an impact, but importantly, we gave a set of people opportunities they otherwise didn't feel they had. So I think people feel very loyal about that. And my biggest piece of evidence for that, besides the surveys, is that we have somewhere north of 80 students, maybe 100 at this point, who graduated but come back and TA for this class for basically minimum wage, even though they're working full time, because they believe, they believe in, in sort of having that opportunity. They want to be a part of something. Now, will they, will generation three feel this way?

• 1:21:50Charles Isbell?

  15 years from now, will people have that same sense? I don't know. But right now, they, they kind of do. And so it's not the online, it's, it's a matter of feeling as if you're a part of something, right? We're all very tribal.

• 1:22:02Lex Fridman?

  Yeah.

• 1:22:02Charles Isbell?

  Right? And I think there's something very tribal about being a part of something like that. Being on campus makes that easier. Going through a shared experience makes that easier. It's harder to have that shared experience if you're alone, looking at a computer screen. We can create ways to make that.

• 1:22:18Lex Fridman?

  But is it possible?

• 1:22:18Charles Isbell?

  It is possible.

• 1:22:19Lex Fridman?

  That's, the question is, it still is the intuition to me, and it was at the beginning, when I saw something like the online masters program, is that this is going to replace universities?

• 1:22:32Charles Isbell?

  And it won't replace universities, but it

• 1:22:34Lex Fridman?

  But like, why is it, why?

• 1:22:35Charles Isbell?

  Because it's living in a different part of the ecosystem, right? The people who are taking it are already adults. They've, they've gone through their undergrad experience. Their, I think their, their goals have shifted from when they were 17. They have other things that, that are going.

• 1:22:48Lex Fridman?

  Right.

• 1:22:48Charles Isbell?

  But it does do something really important, something very social and very important, right? You, you know, this whole thing about, um, you know, don't build the sidewalks, just leave the grass and the students will, or the people will walk and you put the sidewalks where they create paths, this kind of thing.

• 1:23:00Lex Fridman?

  That's interesting, yeah.

• 1:23:02Charles Isbell?

  Their architects apparently believe that's the right way to do things. The metaphor here is that we, we, we created this environment. We didn't quite know how to think about the social aspect, but you know, we didn't have time to solve all, do all the social engineering, right? The students did it themselves. They created.

• 1:23:21Charles Isbell?

  You know, these groups like on Google Plus, there were like thirty something groups created in the first year because somebody had these Google Plus. And they created these groups and they divided up in ways that made sense. We live in the same state or we're working on the same things. We have the same background, whatever. And they created these social things. We sent them T-shirts. And they were, we have all these great pictures of students putting on their T-shirts as they travel around the world. I climbed to this mountaintop. I'm putting this T-shirt on. I'm a part of this. They were, they were a part of them. They created the social environment on top of the social network and the social media that existed to create this sense of belonging and being a part of something. They found a way to do it. Right. And I think they had other.

• 1:24:00Charles Isbell?

  It scratched an itch that they had, but they had scratched some of that itch that might have required they be physically in the same place long before. Right. So I think yes, it's possible and it's more than possible. It's necessary. But I don't think it's going to replace the university as we know it. The university as we know it will change. But there's just a lot of power in the kind of rite of passage and kind of going off to yourself. Now, maybe there'll be some other rite of passage that'll happen.

• 1:24:28Lex Fridman?

  That's the question.

• 1:24:29Charles Isbell?

  That'll drive it somewhere else.

• 1:24:30Lex Fridman?

  You can separate. So the university is such a fascinating mess of things. So just even the faculty position is a fascinating mess. Like it doesn't make any sense. It's stabilized itself. But like, why are the world-class researchers spending a huge amount of time or their time teaching and service? Like you're doing like three jobs.

• 1:24:54Charles Isbell?

  Yeah.

• 1:24:54Lex Fridman?

  And, and I mean, it turns, it's maybe an accident of history or human evolution. I don't know. It seems like the people who are really good at teaching are often really good at research. There seems to be a parallel there. But like, it doesn't make any sense that you should be doing that. At the same time, it also doesn't seem to make sense that your place where you party

• 1:25:18Lex Fridman?

  is the same place where you go to learn calculus or whatever.

• 1:25:22Charles Isbell?

  But it's a safe space.

• 1:25:23Lex Fridman?

  Safe space for everything.

• 1:25:25Charles Isbell?

  Yeah, relatively speaking, it's a safe space. Now, by the way, I feel the need very strongly to point out that we are living in a very particular weird bubble. Right? Most people don't go to college. And by the way, the ones who do go to college, they're not 18 years old, right? They're like 25 or something. I forget the numbers. You know, the places where we've been, where we are, uh, they look like whatever we think the traditional movie version of universities are. But for most people, it's not that way at all. By the way, most people who drop out of college, it's entirely for financial reasons, right? The, the, the, so, you know, we're talking about a particular experience. Um, and so for that set of people, which is very small, but larger than it was a decade or two or three or four, certainly ago.

• 1:26:10Charles Isbell?

  I don't think that will change. My concern, which I think is kind of implicit in some of these questions, is that somehow we will divide the world up further. Uh, into the people who get to have this experience and get to have the network and they sort of benefit from it. And everyone else, while increasingly requiring that they have more and more credentials in order to get a job as a barista. Right? You gotta have a master's degree in order to, to work at Starbucks. Um, and we're going to force people to do these things, but they're not going to get to have that experience. And there'll be a small group of people who do continue to, you know, positive feedback loop, et cetera, et cetera, et cetera. I worry a lot about that, which is why for me, um, and by the way, here's an answer to your question about faculty. Which is why to me that you have to focus on access and the mission. I think the reason, whether it's good, bad, or strange, I mean, I agree, it's strange.

• 1:26:54Charles Isbell?

  But I think it's useful to have the faculty member, particularly at large R1 universities where we've all had experiences. Uh, that, um, you tie what they get to do and with the fundamental mission of the university. And let the mission drive. What I hear when I talk to faculty is they love their PhD students because they're creating, they're reproducing, basically, right? And it lets them do their research and multiply. But they understand that the mission is the undergrads. And so they will do it without complaint, mostly, because it's a part of the mission and why they're here. And they have experiences with it themselves. And it was important to get them, get them where they were going. The people who tend to get squeezed in that, by the way, are the master students, right? Who are neither the PhDs who are like us, nor the undergrads.

• 1:27:37Charles Isbell?

  We, we have already bought into the idea that we, we have to teach, though. That's increasingly changing. Anyway, I, I think tying that mission in really matters. And it gives you a way to unify people around making it an actual higher calling. Education feels like more of a higher calling to me than, than even research. Because education, you cannot treat it as a hobby if you're going to do it well.

• 1:27:59Lex Fridman?

  But, but that's the, that's the pushback on this whole system is that you should, education should be a full-time job. Right? And like, it almost like research is a distraction from that.

• 1:28:13Charles Isbell?

  Yes, although I think most of our colleagues, many of our colleagues would say that research is a job and education is distraction.

• 1:28:19Lex Fridman?

  Right, but that's the beautiful dance. It seems to be that, that tension in itself is, seems to work, seems to bring out the best in, in the faculty. Or like the ones that.

• 1:28:31Charles Isbell?

  But I will point out two things. One thing I'm going to point out, and the other thing I want Michael to point out, because I think Michael is much closer to the, to the, to sort of the, the ideal professor in some sense than I am.

• 1:28:41Lex Fridman?

  Well, he is the platonic sense of a professor.

• 1:28:44Speaker 02

  I don't know what he meant by that, but he's, he is a dean, so he has a different experience.

• 1:28:48Charles Isbell?

  I'm giving him, I'm giving him time to think of the profound thing he's going to say.

• 1:28:51Lex Fridman?

  That's good.

• 1:28:51Charles Isbell?

  But let me, let me point this out, which is that we have lecturers in the College of Computing where I am. There's 10 or 12 of them depending on how you count, as opposed to the 90 or so tenure track faculty. Those 10 lecturers who only teach, well, they don't only teach, they also do service. They, some of them do research as well, but primarily they teach. They teach 50%, over 50% of our credit hours. And we teach everybody, right? So they're doing not just, they're doing more than eight times the work of the tenure track faculty. But just more closer to nine or 10. And that's including our grad courses, right? So they're doing this, they're teaching more, they're touching more than anyone. And they're beloved for it.

• 1:29:33Charles Isbell?

  I mean, so we recently had a survey, you do these alumni, everyone does these alumni surveys. You hire someone from the outside to do whatever. And I was really struck by something. So all these really cool numbers, I'm not going to talk about it because you know, it's all internal confidential stuff. But one thing I will talk about is there was a single question we asked our alumni. And these are people who graduated, you know, born in the 30s and 40s, all the way up to people who graduated last week, right? Well, last semester.

• 1:29:55Speaker 02

  Okay, good.

• 1:29:57Charles Isbell?

  Time flies. Yeah, time flies. And it was a question, name this a single person who had a strong positive impact on you, something like that.

• 1:30:06Speaker 02

  I think it was special impact?

• 1:30:09Charles Isbell?

  Yeah, special impact on you. And then, so they got all the answers from people and they created a word cloud. There was clearly a word cloud created by people who don't do word clouds for a living because they had one person whose name like appeared like nine different times, like Philip, Phil, Dr. Phil, you know, but whatever. But they got all this and I looked at it and I noticed something really cool. The five people from the College of Computing I recognized were in that cloud. And four of them were lecturers, the people who teach. Two of them relatively modern, both were chairs of our Division of Computing Instruction. One just, one retired, one is going to retire soon. And the other two were lecturers I remembered from the 1980s. Two of those four actually have- By the way, the fifth person was Charles.

• 1:30:54Charles Isbell?

  That's not important. The thing is, I don't tell people that, but the two of those people our teaching awards are named after. Thank you, Michael. Two of those, our teaching awards are named after, right? So when you ask students, alumni, people who are now 60, 70 years old even, you know, who touched them? They say the dean of students. They say the big teachers who taught the big introductory classes that got me into it. There's a guy named Richard Bark who's on there, who's, who's, you know, who's known as a great teacher. The Phil Adler guy who- Who I probably just said his last name wrong, but I know the first name's Phil because he kept showing up over and over again. Famous.

• 1:31:28Speaker 02

  Adler is what it said.

• 1:31:29Charles Isbell?

  Okay, good.

• 1:31:29Speaker 02

  But different people spelled it differently, so he appeared multiple times.

• 1:31:32Charles Isbell?

  Right, so he was a clearly, he was a professor in the business school. But when you read about him, I went to read about him because I was curious who he was, you know, it's all about his teaching and the students that he touched, right? So whatever it is that we're doing and we think we're doing that's important, or why we think the universities function, the people who go through it.

• 1:31:51Speaker 02

  Yeah.

• 1:31:52Charles Isbell?

  They remember the people who were kind to them, the people who taught them something, and they do remember it, they remember it later. I think that's important. And so the mission matters.

• 1:32:01Lex Fridman?

  Yeah, not to completely lose track of the fundamental problem of how do we replace the the party aspect of universities.

• 1:32:11Speaker 02

  That's right.

• 1:32:12Lex Fridman?

  Before we go to the what makes the platonic professor, do you do you think, like, what in your sense is the role of moocs in this whole picture during COVID? Like, are we, should we desperately be clamoring to get back on campus? Or is this a stable place to be for a little while?

• 1:32:33Speaker 02

  I don't know. I know that it's that it's the online teaching experience and learning experience has been really rough. I think that that people find it to be a struggle in a way that's not a happy, positive struggle, that when you got through it, you just feel like glad that it's over as opposed to I've achieved something. So, you know, I worry about that, but, you know, I worry about just even before this happened, I worry about lecture teaching as how how well is that actually really working as far as a way to do education, as a way to inspire people? I mean, all the data that I'm aware of seems to indicate, and this kind of fits, I think, with Charles's story, is that people

• 1:33:16Speaker 02

  Respond to connection, right? They actually feel if they're if they feel connected to the person teaching the class, they're more likely to go along with it. They're more they're more able to retain information. They're more motivated to be involved in the class in some way. And and that really matters. It people.

• 1:33:34Lex Fridman?

  You mean to the human themselves?

• 1:33:36Speaker 02

  Yeah.

• 1:33:37Lex Fridman?

  So but can't you do that actually perhaps more effectively? Online, like you mentioned, science communication. So I I literally, I think, learned linear algebra from Gilbert Strang by watching MIT Open Courseware when I was a drug like and he was a personality. He was a bit like a, you know, tiny in in this tiny little world of math. He's a bit of a rock star, right? So you kind of look up to that to that person. Can't that replace the in person education?

• 1:34:09Charles Isbell?

  It can help. I will point out something. I can't share the numbers, but the we have surveyed our students. And even though they have feelings about what I would interpret as connection. I like that word in the different modes of classrooms. There's no difference between how they how well they think they're learning. For them, the thing that makes them unhappy is the situation they're in. And I think the last lack of connection, it's not whether they're learning anything. They seem to think they're learning something anyway. Right? And in fact, they seem to think they're learning it equally well. Presumably because the faculty are putting in or the instructors, more generally speaking, are putting in the energy and effort to try to make certain that they're what they've curated

• 1:34:53Charles Isbell?

  can be expressed to them in a useful way. But the connection is missing. And so there's huge differences in what they prefer. And as far as I can tell, what they prefer is more connection, not less. That connection just doesn't have to be physically in a classroom. I mean, look, I used to teach 348 students on a machine learning class on campus. Do you know why? That was the biggest classroom on campus.

• 1:35:13Charles Isbell?

  They're sitting in a theater, they're sitting in theater seats. I'm literally on a stage looking down on them and talking to them, right? There's no, I mean, we're not sitting down, having a one-on-one conversation, reading each other's body language, trying to communicate and going, we're not doing any of that. So, you know, if you're on, if you're past the third row, it might as well be online anyway, is the kind of thing that people have said. Daphne has actually said some version of this, that online starts on the third row or something like that. And I, and I think that's, that's not, yeah, I, I like it. I think it captures something important. But people still came, by the way, they, even the people who had access to our material would still come to class.

• 1:35:49Lex Fridman?

  I mean, there's a certain element about looking to the person next to you.

• 1:35:52Charles Isbell?

  Yeah.

• 1:35:53Lex Fridman?

  It's just like their presence there, their, their boredom and like when the parts are boring and their excitement when the parts are exciting, like in sharing in that, like unspoken kind of, uh, yeah, communication, like in part, the connection is with the other people in the room.

• 1:36:11Charles Isbell?

  Watching this, watching the circus on TV alone is not really, have you ever been to a movie theater and been the only one there at a comedy? It's not as funny as when you're in a room full of people all laughing.

• 1:36:24Lex Fridman?

  Well, you need, maybe you need just another person. It's like, as opposed to many, maybe, maybe there's some kind of.

• 1:36:30Charles Isbell?

  Well, there's different kinds of connection, right?

• 1:36:32Speaker 02

  And there's different kinds of comedy.

• 1:36:36Speaker 02

  Well, in the sense that.

• 1:36:37Lex Fridman?

  As we're learning today.

• 1:36:39Speaker 02

  I wasn't sure if that was gonna land, but, um, just the idea that, that different jokes, I, I've, I've now done a little bit of stand-up and so different jokes work in different sized crowds too.

• 1:36:51Charles Isbell?

  No, that's true.

• 1:36:51Speaker 02

  Where sometimes if, you know, if it's a big enough crowd, then even a really subtle joke can take root someplace and then that cues other people and it kind of, there's a whole statistics of, I did this terrible thing to my.

• 1:37:04Charles Isbell?

  So when I was really young, I decided that my brother was only laughing at sitcoms when I laughed. Like he was taking cues from me. So I like purposely didn't laugh just to see if I was right.

• 1:37:15Speaker 02

  And did you laugh at non funny things?

• 1:37:17Charles Isbell?

  Yes.

• 1:37:17Speaker 02

  You really wanted to do both sides?

• 1:37:18Charles Isbell?

  I did both sides. And, uh, and at the end of it, I told him what I did.

• 1:37:22Speaker 02

  Oh, that's so.

• 1:37:23Charles Isbell?

  He was very upset about this.

• 1:37:24Speaker 02

  Yeah.

• 1:37:24Charles Isbell?

  And from that day on.

• 1:37:26Speaker 02

  He's, he lost his sense of humor.

• 1:37:27Charles Isbell?

  No, no, no, no. Well, yes. But from that day on, he, he, he laughed on his own. He stopped taking cues from me.

• 1:37:33Speaker 02

  I see.

• 1:37:33Charles Isbell?

  So I want to say that, you know, it was a good thing that I did, but.

• 1:37:36Speaker 02

  Yes, yes. You saved that man's life.

• 1:37:38Charles Isbell?

  Yes, but it was mostly mean, but it's true though. It's true, right? That, that people, I, I think you're right.

• 1:37:44Speaker 02

  But, okay. So that, where does that get us? That gets us the idea that, I mean, certainly movie theaters are a thing. Right? Where people like to be watching together, even though the people on the screen can't, aren't really co-present with the people in the audience. The audience is co-present with itself.

• 1:37:59Lex Fridman?

  By the way, and that point, it's, it's an open question that's being raised by this, whether movies will no longer be a thing because Netflix's audience is growing. So that's, it's a, it's a very parallel question for education. Will movie theaters still be a thing?

• 1:38:13Speaker 02

  Right.

• 1:38:14Lex Fridman?

  In 2021.

• 1:38:15Speaker 02

  I think the argument is that there is a feeling of being in the crowd that isn't replicated by being at home watching it. And that there's value in that. And then I think just.

• 1:38:24Charles Isbell?

  But. But.

• 1:38:26Speaker 02

  It scales better online.

• 1:38:27Charles Isbell?

  But I feel like we're having a conversation about whether concerts will still exist after the invention of the record or the CD or wherever it is, right?

• 1:38:38Speaker 02

  They won't.

• 1:38:39Charles Isbell?

  You're right. Concerts are dead.

• 1:38:41Speaker 02

  Well, okay. Before, I think the joke is only funny if you say it before now.

• 1:38:46Lex Fridman?

  Right. Yeah, that's true.

• 1:38:48Speaker 02

  Like three years ago. It's like, well, no, obviously.

• 1:38:50Lex Fridman?

  I'll wait to publish this until we have a vaccine.

• 1:38:54Charles Isbell?

  You know, we'll fix it in post. But I think the, the important thing is.

• 1:38:57Speaker 02

  Fix the virus post.

• 1:38:59Charles Isbell?

  Concerts changed, right? First of all, movie theaters weren't this way, right? In like the sixties and seventies, they weren't like this. Like blockbusters were basically what? Well, Jaws and Star Wars created blockbusters, right? Um, before then, there weren't like the whole summer, shared summer experience didn't exist in our lifetimes, right? Certainly you were well into adulthood by the time this was true, right? So it's just a very different, it was, it's very different. So what the, what we're, we've been experiencing in the last 10 years is not like the majority of human history. But more importantly, concerts, right? Concerts mean something different. Most people don't go to concerts anymore. Like there's an age where you care about it. You sort of stop doing, but you keep listening to music or whatever and da, da, da, da, da, da.

• 1:39:38Charles Isbell?

  So I think that's a painful way of saying that, um, it will change. It was not the same thing as a going away. Replace is too strong of a word, but it will change. It has to.

• 1:39:52Lex Fridman?

  I actually like to push back. I wonder because I think you're probably just throwing that, your intuition out.

• 1:39:57Charles Isbell?

  Oh, I definitely.

• 1:39:58Lex Fridman?

  And it's possible concerts, more people go to concerts now, but obviously much more people listen to, well, this is dumb than before there was records. Uh, it's, it's possible to argue that, uh, if you look at the data, uh, that it just expanded the pie of what music listening means. So it's possible that like universities grow in this, in the parallel or the theaters grow, but also more people get to watch movies, more people get to, uh, like be educated.

• 1:40:30Charles Isbell?

  Yes, I, I hope that is true.

• 1:40:31Speaker 02

  Yeah. And to the extent that we can grow the pie and have education be not just something you do for four years when you're done with your other education, um, but it'd be a more lifelong thing, that would have tremendous benefits, especially as the, the economy and the world change rapidly. Like people need opportunities to stay abreast of these changes. And so, I don't know, I could, I could, it's all part of the, the ecosystem.

• 1:40:58Charles Isbell?

  It's all to the good. I mean, you know, I, I'm not going to have an argument about whether we, we lost fidelity. We went from LaserDisc to DVDs or record players to CDs. I mean, I'm, I'm willing to grant that that is true, but convenience matters. And the ability to do something that you couldn't do otherwise because of that convenience matters. And you can tell me I'm only getting 90% of the experience, but I'm getting the experience. I wasn't getting it before, or it wasn't lasting as long, or it wasn't as easy. I mean, this just seems, this just seems straightforward to me. It's gonna, it's going to change. It is for the good that more people get access. And it is our job to do two separate things. One, to educate them and make access available. That's our mission. But also, for very simple selfish reasons, we need

• 1:41:41Charles Isbell?

  to figure out how to do it better so that we individually stay in business. We can do both of those things at the same time. They are not in, they may be in tension, but they are not mutually exclusive.

• 1:41:52Lex Fridman?

  So you've educated some scary number of people.

• 1:41:56Charles Isbell?

  Mm hmm.

• 1:41:59Lex Fridman?

  So you've seen a lot of people succeed, find their path through life. Is there a advice that you can give to a young person today about computer science education, about education in general, about life, about whatever the journey that one takes in their, maybe in their teens, in their early twenties, sort of in those underground years, as you try to go through the essential process of partying and not going to classes, and yet somehow trying to get a degree?

• 1:42:37Charles Isbell?

  If you get to the point where you're, you're, you're far enough up in the, in the hierarchy of needs that you can actually make decisions like this, then find the thing that you're passionate about and pursue it. And sometimes it's the thing that drives your life, and sometimes it's secondary. And you'll do other things because you've got to eat, right? You got a family, you got to feed, you got people you have to help or whatever. And I understand that, and it's not easy for everyone. But always take a moment or two to pursue the things that. You love the things that bring passion and happiness to your life. And if you don't, I know that sounds corny, but I genuinely believe it. And if you don't have such a thing, then you're lying to yourself. You have such a thing. You just have to find it. And it's okay if it takes you a long time to get there.

• 1:43:20Charles Isbell?

  Rodney Dangerfield became a comedian in his 50s, I think. Certainly wasn't his 20s. And lots of people failed for a very long time before getting to where they were going. You know, I try to have hope. And it wasn't obvious. I mean, you know, you and I talked about the experience that I had a long time ago with with a particular police officer. Was it my first one? It wasn't my last one. But, you know, in my view, I wasn't supposed to be here after that. And I'm here. So it's all gravy. So you might as well go ahead and grab life as you can because of that. That's that's sort of how I see it. While recognizing, again, the delusion matters, right? Allow yourself to be deluded. Allow yourself to believe that it's all going to work out.

• 1:44:02Charles Isbell?

  Just don't be so deluded that you you miss the obvious. And you're going to be fine. It's going to be there. It's going to be there. It's going to work out. What do you think?

• 1:44:12Speaker 02

  I like to say, choose your parents wisely. Because that has a big impact on your life. Yeah, I mean, you know, I mean, there's a whole lot of things that you don't get to pick. And whether you get to have, you know, one kind of life or a different kind of life can depend a lot on things out of your control. But I really do believe in the in the passion and excitement thing. My I was talking to my mom on the phone the other day. And essentially, what came out is that computer science is really popular right now. And I get to be a professor teaching something that's very attractive to people.

• 1:44:53Speaker 02

  And she was like, trying to give me some appreciation for how foresightful I was for choosing this line of work, as if somehow I knew that this is what was going to happen in 2020. But that's not how it went for me at all. Like, I studied computer science because I was just interested. It was just so interesting to me. I didn't.

• 1:45:16Speaker 02

  I didn't think it would be particularly lucrative.

• 1:45:18Lex Fridman?

  Yeah.

• 1:45:19Speaker 02

  And I've done everything I've can to keep it as unlucrative as possible.

• 1:45:23Lex Fridman?

  Yeah.

• 1:45:24Speaker 02

  Some of my, you know, some of my friends and colleagues have not done that. And I pride myself on my ability to remain unrich. But, but, but I do believe that, like, I'm glad. I mean, I'm glad that it worked out for me. It could have been like, oh, what I was really fascinated by is this particular kind of engraving that nobody cares about. But so I got lucky and the thing that I cared about happened to be a thing that other people eventually cared about. But I don't think I would have had a fun time choosing anything else. Like, this was the thing that kept me interested and engaged.

• 1:45:57Lex Fridman?

  Well, one thing that people tell me, especially around early undergraduate, and the internet is part of the problem here, is they say they're passionate about so many things. How do I choose a thing? Which is a harder thing for me to know what to do with. Is there any?

• 1:46:16Charles Isbell?

  I mean, don't you know, I mean, you know, look, a long time ago, I walked down a hallway and I took a left turn.

• 1:46:23Lex Fridman?

  Yeah.

• 1:46:24Charles Isbell?

  I could have taken a right turn. And my world could be better or it could be worse. I have no idea. I have no way of knowing.

• 1:46:30Speaker 02

  Is there anything about this particular hallway that's relevant? Or you're just in general, choices?

• 1:46:33Charles Isbell?

  Yeah, you were on the left.

• 1:46:34Lex Fridman?

  It sounds like you regret not taking the right turn.

• 1:46:36Charles Isbell?

  Oh, no, not at all.

• 1:46:37Lex Fridman?

  You brought it up.

• 1:46:38Charles Isbell?

  Well, because there was a turn, a turn there. On the left was Michael Lemon's office, right? I mean, these sorts of things happen, right?

• 1:46:44Lex Fridman?

  Yes.

• 1:46:44Charles Isbell?

  But here's the thing.

• 1:46:45Speaker 02

  On the right, by the way, there was just a blank wall. It wasn't a huge choice.

• 1:46:49Charles Isbell?

  It would have really hurt.

• 1:46:49Lex Fridman?

  He tried first.

• 1:46:50Charles Isbell?

  No, but it's, it's, it's true, right? That, you know, I, I think about Ron Brockman, right? I, I went, I took a trip I wasn't supposed to take. And I ended up talking to, to Ron about this. And I ended up going down this entire path that allowed me, I think, get tenure. But by the way, I decided to say yes to something that didn't make any sense. And I went down this educational path. But it would have been, you know, who knows, right? Maybe if I hadn't done that, I would be a billionaire right now. I'd be Elon Musk. My life could be so much better. My life could also be so much worse. You know, you've just got to feel that sometimes you have decisions you're gonna make. You cannot know what's gonna do. You should think about it, right? Some things are clearly smarter than other things. You gotta play the odds a little bit.

• 1:47:34Charles Isbell?

  But in the end, if you've got multiple choices, there are lots of things you think you might love. Go with the thing that you actually love, the thing that jumps out at you, and sort of pursue it for a little while. The worst thing that'll happen is you took a left turn instead of a right turn, and you ended up merely happy.

• 1:47:47Lex Fridman?

  Beautiful. So, so accepting, so taking the step and just accepting, accepting that that don't like question. I like to think that life is long, and there's time to actually pursue. Every once in a while, you have to put on a leather suit and make a thriller video.

• 1:48:07Charles Isbell?

  Every once in a while.

• 1:48:08Lex Fridman?

  If I ever get a chance again, I'm doing it. Yeah. I was told that you actually danced, but that part was edited out.

• 1:48:18Charles Isbell?

  I don't dance. There was a thing where we did do the zombie thing. We did do the zombie thing. That wasn't edited out. It just wasn't put into the final thing. I'm quite happy. But there was a reason for that, too, right? Like, I wasn't wearing something right? There was a reason for that. I can't remember what it was.

• 1:48:35Lex Fridman?

  No leather suit.

• 1:48:37Charles Isbell?

  Is that what it was? I can't remember. Anyway, the right thing happened.

• 1:48:40Lex Fridman?

  Exactly. You took the left turn and it ended up being the right thing. So a lot of people ask me that are a little bit tangential to the programming, the computing world, and they're interested to learn programming, like all kinds of disciplines that are outside of the particular discipline of computer science. What advice do you have for people that want to learn how to program? Or want to either taste this little skill set or discipline? Or try to see if it can be used somehow in their own life?

• 1:49:13Charles Isbell?

  What stage of life are they in?

• 1:49:16Lex Fridman?

  It feels, well, one of the magic things about the internet of the people that write me is, I don't know.

• 1:49:22Charles Isbell?

  Hmm. Because my answer is different for, for, for, my daughter is taking AP Computer Science right now. Hi, Johnny. She's, uh, she's amazing and doing amazing things. And my son's beginning to get interested and I'll be really curious where, where he takes it. I think he's, his mind actually works very well for this sort of thing and she's doing great. But one of the things I have to tell her all the time, she points, well, I want to make a rhythm game. So I want to go for two weeks and then build a rhythm game. Show me how to build a rhythm game. And then start small, learn the building blocks and how, take the time, have patience. Eventually you'll build a rhythm game. I was in grad school when I suddenly woke up one day over the Royal East. And I thought, wait a minute, I'm a computer scientist. I should be able to write Pac-Man in an afternoon. And I did, not with great graphics. It was actually a very cool game.

• 1:50:07Charles Isbell?

  I had to figure out how the ghost moved and everything. And I did it in an afternoon in Pascal on an old Apple 2GS. But if I had started out trying to build Pac-Man, I think it probably would have ended very poorly for me. Luckily, back then, there weren't, you know, these magical devices we call phones and software everywhere to give me this illusion that I could create something by myself from the basics inside of a weekend like that. I mean, that was a culmination of years and years and years, right? Before I decided, oh, I should be able to write this and I could. So, you know, my advice, if you're early on is, you know, you've got the internet. There are lots of people there to give you the information. Find someone who cares about this. Remember, they've been doing it for a very long time. Take it slow, learn the little pieces, get excited about it, and then keep the big project

• 1:50:51Charles Isbell?

  you want to build in mind. You'll get there soon enough because, as a wise man once said, life is long. Sometimes it doesn't seem that long, but it is long and you'll have enough time to, to build it all out. It, all the information is out there, but start small. You know, generate Fibonacci numbers. It's not exciting, but it'll get you.

• 1:51:13Charles Isbell?

  Well, there's only one programming language, it's Lisp. But if you have to pick a programming language, I guess in today's day, what would I do? I guess I'd do.

• 1:51:21Speaker 02

  Python is basically Lisp, but with with better syntax.

• 1:51:25Charles Isbell?

  Blasphemy.

• 1:51:26Speaker 02

  Yeah, with C syntax, how about that?

• 1:51:28Charles Isbell?

  So you're gonna argue that C syntax is better than anything? Anyway, also, I'll go, I'm gonna answer Python despite what he said.

• 1:51:34Speaker 02

  Tell me, tell your story about the, somebody's dissertation that had a Lisp program in it.

• 1:51:39Charles Isbell?

  It was so funny, this is, this is Dave's, Dave's dissertation was like, Dave McAllister, who was a professor at MIT for a while, and then he came to Bell Labs. And now he's at, now he's at Technology, Technical Institute of Chicago. A brilliant guy, such an interesting guy. Anyway, his thesis, it was a theorem prover. And he decided to have as an appendix, his actual code, which of course was all written in Lisp, because of course it was. And like, the last 20 pages are just right parentheses. It's wonderful. Like, they, that's programming, right there. Pages of, pages of right parentheses. Anyway, Lisp is the only real language, but I understand that that's not necessarily the place where you start. Python is just fine. Python is good.

• 1:52:23Charles Isbell?

  If you're, you know, of a certain age, if you're really young and trying to figure it out, graphical languages that let you kind of see how the thing works, and that's fine, too. They're all fine. It almost doesn't matter. But there are people who spend a lot of time thinking about how to build languages that get people in. The question is, are you trying to get in and figure out what it is? Or do you already know what you want? And that's why I asked you what stage of life people are in. Because if you're different stages of life, you would, you would attack it differently.

• 1:52:48Lex Fridman?

  The answer to that question of which language keeps changing, I mean, there's some value to exploring, a lot of people write to me about Julia, there's, there's these like, more modern languages, they keep being invented, Rust and, and Kotlin. And there's stuff that, for people who love functional languages like Lisp, that apparently there's echoes of that, but much better in, in the modern languages. It's worthwhile to. Especially when you're learning languages, it feels like it's okay to try one that's not like the popular one.

• 1:53:22Charles Isbell?

  Oh, yeah, but you know.

• 1:53:23Speaker 02

  And I think you get that you get that way of thinking almost no matter what language. And if you if you push far enough, like it can be assembly language, but you need to push pretty far before you start to hit the really deep concepts that you would get sooner in other languages. But like, I don't know, computation is kind of computation is kind of touring equivalent is kind of computation. And so it's so it matters how you express things, but you have to build out that mental structure in your mind. And I don't I don't think it super matters which language.

• 1:53:53Charles Isbell?

  I mean, it matters a little because some things are just at the wrong level of abstraction. I think assemblies at the wrong level of abstraction for someone coming in new. I think that if you start.

• 1:54:01Speaker 02

  For someone coming in new.

• 1:54:03Charles Isbell?

  Yes, for frameworks, big frameworks are are quite a bit. You know, you've got to get to the point where I want to learn a new language means I just pick up a reference book and I think of a project and I go through it in a weekend. But you got to you got to get there. You're right, though, the languages that are designed for that are it almost doesn't matter. Pick the ones that people have built tutorials and infrastructure around to help you get kind of kind of ease into it because it's hard. I mean, I did this little experiment once. I was teaching intro to CS in the summer as a favor. Which is anyway.

• 1:54:36Speaker 02

  Pleasant memories.

• 1:54:37Charles Isbell?

  I was teaching intro to CS as a favor. And it was very funny because I'd go in every single time and I would think to myself, how am I possibly going to fill up an hour and a half talking about for loops, right? And there wasn't enough time. Took me a while to realize this, right? There are only three things, right? There's reading from a variable, writing to a variable and conditional branching. Everything else is syntactic sugar, right? The syntactic sugar matters, but that's it. And when I say that's it, I don't mean it's simple. I mean, it's hard, like conditional branching, loops, variable, those are really hard concepts. So you shouldn't be discouraged by this. Here's a simple experiment. I'm going to ask you a question now. You ready? X equals three.

• 1:55:16Speaker 02

  Okay.

• 1:55:17Charles Isbell?

  Y equals four.

• 1:55:19Speaker 02

  Okay.

• 1:55:20Charles Isbell?

  What is X?

• 1:55:21Lex Fridman?

  Three.

• 1:55:22Charles Isbell?

  What is Y?

• 1:55:23Lex Fridman?

  Four.

• 1:55:24Charles Isbell?

  Y equals X.

• 1:55:25Lex Fridman?

  I'm going to mess this up.

• 1:55:26Charles Isbell?

  No, that's oh, it's easy to do. Y equals X.

• 1:55:28Lex Fridman?

  Y equals X.

• 1:55:29Charles Isbell?

  What is Y?

• 1:55:32Lex Fridman?

  Uh, three.

• 1:55:33Charles Isbell?

  That's right. X equals seven.

• 1:55:36Charles Isbell?

  What is y?

• 1:55:37Lex Fridman?

  That's one of the trickiest things to get for programmers, that there's a memory and the variables are pointing to a particular thing in memory. And sometimes the languages hide that from you and they bring it closer to the way you think mathematics works.

• 1:55:51Charles Isbell?

  Right. So in fact, Mark Guzdal, who worries about these sorts of things, or used to worry about these sorts of things anyway, had this kind of belief that actually people, when they see these statements, X equals something, Y equals something, Y equals X, that you have now made a mathematical statement that Y and X are the same.

• 1:56:10Speaker 02

  Which you can if you just put like an anchor in front of it.

• 1:56:12Charles Isbell?

  Yes, but people, that's not what you're doing.

• 1:56:14Speaker 02

  Yeah.

• 1:56:14Charles Isbell?

  Right? I thought, and I kind of asked the question, and I think I had some evidence for this, I'm sorry, a study, is that most of the people who didn't know the answer, weren't sure about the answer, they had used spreadsheets.

• 1:56:26Lex Fridman?

  Ah, interesting.

• 1:56:28Charles Isbell?

  And so it's a, it's a, it's, it's, you know,

• 1:56:32Charles Isbell?

  it's by, it's by reference or by name, really. Right? And so depending upon what you think they are, you get completely different answers. The fact that I could go, or one could go, two thirds of the way through a semester, and people still hadn't figured out in their heads, when you say Y equals X, what that meant, tells you it's actually hard. Because all those answers are possible. And in fact, when you said, oh, if you just put an ampersand in front of it, I mean, that doesn't make any sense for an intro class. And of course, a lot of languages don't even give you the ability to think about it in terms of ampersand. Do we want to have a 45 minute discussion about the difference between equal EQ and equal in Lisp?

• 1:57:06Lex Fridman?

  Yeah.

• 1:57:07Charles Isbell?

  I know you do.

• 1:57:10Charles Isbell?

  But you know, you could do that. It's, this is actually really hard stuff. So you shouldn't be, it's not too hard. We all do it, but you shouldn't be discouraged. It's why you should start small, so that you can figure out these things. You have the right model in your head. So that when you write the language, you can execute it and build the machine that you want to build, right?

• 1:57:27Lex Fridman?

  Yeah. The funny thing about programming and those very basic things is the, the, the very basics are not often made explicit, which is actually what drives everybody away from basically any discipline, but programming is just another one. Like even a simpler version of the equal sign that I kind of forget is in mathematics, equals is not assignment.

• 1:57:50Charles Isbell?

  Yeah.

• 1:57:51Lex Fridman?

  Like, I think basically every single programming language with just a few handful of exceptions equals its assignment.

• 1:57:59Charles Isbell?

  Mm hmm.

• 1:57:59Lex Fridman?

  You have some other operator for equality.

• 1:58:02Charles Isbell?

  Yeah.

• 1:58:03Lex Fridman?

  And, you know, even that, like, everyone kind of knows it.

• 1:58:06Charles Isbell?

  Mm hmm.

• 1:58:07Lex Fridman?

  Once you started doing it, but like, you need to say that explicitly, or you just realize it, like, yourself. Otherwise, you might be stuck for, you said, like, half a semester. You could be stuck for quite a long time. And I think also part of the programming is being okay in that state of confusion for a while. It's to the debugging point. It's like, I just wrote two lines of code. Why doesn't this work? And staring at that for like hours.

• 1:58:39Lex Fridman?

  And trying to figure out. And then every once in a while, you just have to restart your computer and everything works again. And then, and then you just kind of stare into the void with the tear slowly rolling down your eye.

• 1:58:51Charles Isbell?

  By the way, the fact that they didn't get this actually had no impact on, I mean, they were still able to do their assignments.

• 1:58:56Lex Fridman?

  Right.

• 1:58:57Charles Isbell?

  Because it turns out their misunderstanding wasn't being revealed to them.

• 1:59:01Lex Fridman?

  Yes.

• 1:59:02Charles Isbell?

  By the problem sets we were giving them.

• 1:59:05Lex Fridman?

  It's profound, actually, yeah.

• 1:59:06Charles Isbell?

  I wrote a program a long time ago, actually, for my master's thesis, and in C++ I think, or C, I guess it was C. And it was all memory management and terrible. And it wouldn't work for a while.

• 1:59:21Charles Isbell?

  And it was some kind of, it was clear to me that it was overwriting memory, and I just couldn't, I was like, look, I got a paper due, time for this. So I basically declared a variable at the front in the main that was like 400K, just an array. And it worked. Because wherever I was scribbling over memory, it would scribble into that space and it didn't matter. And so I never figured out what the bug was. But I did create something to sort of deal with it.

• 1:59:46Lex Fridman?

  To work around it.

• 1:59:47Charles Isbell?

  And it, you know, that's crazy. That's crazy. It was okay, because that's what I wanted. But I knew enough about memory manage to go, you know, management to go, you know, I'm just going to create an empty array here and hope that that deals with the scribbling memory problem. And it did. That takes a long time to figure out. And by the way, the language you first learned probably is garbage collection anyway, so you're not even going to come up across, you're not going to come across that problem.

• 2:00:08Lex Fridman?

  So we talked about the Minsky idea of hating everything you do and hating yourself. So let's end on a question that's gonna make both of you very uncomfortable.

• 2:00:19Charles Isbell?

  Okay.

• 2:00:19Lex Fridman?

  Which is, what is your, Charles, what's your favorite thing that you're grateful for about Michael? And Michael, what is your favorite thing that you're grateful for about Charles?

• 2:00:34Charles Isbell?

  Well, that answer is actually quite easy. It's friendship.

• 2:00:39Lex Fridman?

  He's still the easy answer.

• 2:00:40Charles Isbell?

  I did.

• 2:00:40Speaker 02

  Yeah, I can tell you what I hate about Charles. He steals my good answers. The thing I like most about Charles, he sees the world in a similar enough, but different way that it's sort of like having another life. It's sort of like I get to experience things that I wouldn't otherwise get to experience because I would not naturally gravitate to them that way. And so he just, he just shows me a whole other world. It's awesome.

• 2:01:04Charles Isbell?

  Yeah, the inner product is not zero, for sure. It's not quite one. Point seven, maybe? Just enough that you can learn.

• 2:01:15Speaker 02

  Just enough that you can learn.

• 2:01:17Lex Fridman?

  That's the definition of friendship. The inner product is point seven.

• 2:01:20Charles Isbell?

  Yeah, I think so. That's the answer to life, really.

• 2:01:22Speaker 02

  Charles sometimes believes in me when I have not believed in me. He can, he also sometimes works as an outward confidence that he has so much, so much confidence and self, I don't know.

• 2:01:34Charles Isbell?

  Awareness.

• 2:01:34Speaker 02

  Comfortableness.

• 2:01:36Charles Isbell?

  Okay, let's go with that.

• 2:01:37Speaker 02

  That I feel better a little bit. If he, if he thinks I'm okay, then maybe I'm not as bad as I think I am.

• 2:01:44Lex Fridman?

  At the end of the day, luck favors the Charles. It's a huge honor to talk with you. Thank you so much for taking this time, wasting your time with me. It was an awesome conversation. You guys are an inspiration to a huge number of people and to me. So really enjoyed this. Thanks for talking.

• 2:02:02Charles Isbell?

  I enjoyed it as well. Thank you so much. And by the way, if luck favors the Charles, then it's certainly the case that I've been very lucky to do.

• 2:02:07Speaker 02

  Oh.

• 2:02:08Lex Fridman?

  I'm gonna edit that part out.

• 2:02:12Lex Fridman?

  Thanks for listening to this conversation with Charles Isbell and Michael Littman. And thank you to our sponsors, Athletic Greens, super nutritional drink, Eight Sleep, self cooling mattress, masterclass online courses from some of the most amazing humans in history, and Cash App, the app I use to send money to friends. Please check out the sponsors in the description to get a discount and to support this podcast. If you enjoy this thing, subscribe on YouTube, review it with five stars on Apple Podcasts, follow on Spotify, support it on Patreon, or connect with me on Twitter at Lex Fridman. And now let me leave you with some words from Desmond Tutu.

• 2:02:55Lex Fridman?

  Don't raise your voice, improve your argument. Thank you for listening and hope to see you next time.