#183 – Po-Shen Loh: Mathematics, Math Olympiad, Combinatorics & Contact Tracing

Po-Shen Loh is a mathematician at CMU and coach of the USA International Math Olympiad team. Please support this podcast by checking out our sponsors: – The Jordan Harbinger Show: https://jordanharbinger.com/lex/ – Onnit: https://lexfridman.com/onnit – BetterHelp: https://betterhelp.com/lex to get 10% off – Eight Sleep: https://www.eightsleep.com/lex and use code LEX to get special savings – LMNT: https://drinkLMNT.com/lex to get free sample pack EPISODE LINKS: Po’s Twitter: https://twitter.com/poshenloh Po’s Website: https://www.poshenloh.com/ Daily Challenges: https://daily.poshenloh.com/ NOVID: https://www.novid.org/ PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ YouTube Full Episodes: https://youtube.com/lexfridman YouTube Clips: https://youtube.com/lexclips SUPPORT & CONNECT: – Check out the sponsors above, it’s the best way to support this podcast – Support on Patreon: https://www.patreon.com/lexfridman – Twitter: https://twitter.com/lexfridman – Instagram: https://www.instagram.com/lexfridman – LinkedIn: https://www.linkedin.com/in/lexfridman – Facebook: https://www.facebook.com/lexfridman – Medium: https://medium.com/@lexfridman OUTLINE: Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time. (00:00) – Introduction (08:51) – Planes and bridges (12:28) – Writing a computer game from scratch (14:54) – Programming competitions (18:28) – Math is hard (23:59) – Contact tracing that preserves privacy (1:01:16) – Math Olympiad (1:16:57) – Hard math problem (1:24:14) – Is math discovered or invented? (1:29:09) – Intelligence (1:35:59) – Math education (1:40:10) – How to learn math (1:49:06) – Combinatorics (1:52:13) – Voting trees (2:02:36) – Stochastic coalescence (2:12:23) – P=NP (2:16:39) – Tolkien and WWII (2:19:00) – Advice for young people (2:21:05) – Meaning of life

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Lex Fridman?

  The following is a conversation with Po Chen Lo, a professor of mathematics at Carnegie Mellon University, national coach of the USA International Math Olympia Team, and founder of XP that does online education of basic math and science. He's also the founder of Novid, an app that takes a really interesting approach to contact tracing, making sure you stay completely anonymous, and it gives you statistical information about COVID cases in your physical network of interactions. So you can maintain privacy, very important, and make informed decisions. In my opinion, we'd desperately need to solutions like this in early 2020. And unfortunately, I think we will again need it for the next pandemic.

• 0:44Lex Fridman?

  To me, solutions that require large-scale, distributed coordination of human beings need ideas that emphasize freedom and knowledge. Quick mention of our sponsors: Jordan Harbinger Show, Onnit, BetterHelp, Aidsleep, and Element. Check them out in the description to support this podcast. As a side note, let me say that Po and I filmed a few short videos about simple, beautiful math concepts that I will release soon. It was really fun. I really enjoyed Po sharing his passion for math with me in those videos. I'm hoping to do a few more short videos in the coming months that are educational in nature on AI, robotics, math, science, philosophy,

• 1:26Lex Fridman?

  or if all else fails, just fun snippets into my life on music, books, martial arts, and other random things. If that's of interest to anyone at all. As usual, I'll do a few minutes of ads now. I try and make these interesting, but I give you timestamps, so if you skip, please still check out the sponsors by clicking the links in the description. It is the best way to support this podcast. I'm very picky about the sponsors we take on, so hopefully, if you buy their stuff, you will find value in it, just as I have. This episode is sponsored by the Jordan Harbinger Show. Go to jordanharbinger.com/lex, subscribe to it, listen, you will not regret it.

• 2:10Lex Fridman?

  He is a great interviewer and I especially like his Feedback Fridays episodes where this combination of fearlessness and thoughtfulness is especially on display, touching topics of sex, corruption, mental disorders, hate, love, and everything in between. I've overheard somebody talking about this idea that some podcasts that they enjoy are educational and some are just fun to listen to. I think I connect with this kind of dichotomy and I think the best podcasts are the ones that combine both. I actually think it's almost impossible to combine both in any one specific segment of a podcast, but you almost want to be alternating, oscillating between the two.

• 2:54Lex Fridman?

  Just shooting the shit and having fun for a few minutes and then going back into serious scientific or historic exploration of a concept. Jordan does a pretty good job of this, I think. So go to jordanharbinger.com/lex. It's how he knows I sent you. That's jordanharbinger.com/lex. This episode is also sponsored by Onnit, nutrition, supplement, and fitness company. They make AlphaBrain, which is a nootropic that helps support memory, mental speed, and focus. I think of those three, for me, focus is probably the only thing I can count on improving and working on and focusing on.

• 3:34Lex Fridman?

  I think my memory has a very interesting quality to it, that I forget most things, or rather, I forget the details of most things, but somehow they stay in there almost as platonic forms. Like the names disappear, the details, like the dates and the context disappears, but the core concept, the core idea stays. It's a really weird kind of memory. I'm not sure if it's genetic or something I just haven't practiced enough of. Anyway, go to lexfridman.com/onnit for their semi-annual sale that includes 25% off supplements, 25% off nutrition, and many more exciting discounts.

• 4:16Lex Fridman?

  I'm trying a new thing with a URL, lexfridman.com/onnit. Which reminds me, I'm here in Austin now and Onnit has a gym here, which I should probably go check out. This episode is sponsored by BetterHelp, spelled H E L P, help. They figure out what you need and match you with a licensed professional therapist in under 48 hours. A bunch of people have been asking me about David Goggins and where did the podcast go? Are you still doing the challenge? Yes, we're still going to do the challenge and yes, we're still going to do the podcast. I'm thinking it's probably now going to happen in July, August. We're probably going to do some kind of long distance run together and do the podcast afterwards.

• 5:00Lex Fridman?

  I'm also now starting to slowly, slowly get back in better shape and thinking about competing in jujitsu again. Doing a few tournaments here and there to challenge myself in the physical space. I think it's really good for the mind to challenge the body. And I think it's good for the body to challenge the mind, those to go well together. Competing is one of the best ways to do both, actually. But you know what else is good? Talk therapy. BetterHelp is easy, private, affordable and available worldwide. So if you don't want to compete in a jujitsu tournament, try out BetterHelp. Check them out at betterhelp.com/lex. That's betterhelp.com/lex. This episode is also sponsored by Eight Sleep and its Pod Pro mattress.

• 5:45Lex Fridman?

  It controls temperature with an app, is packed with sensors and can cool down to as low as 55 degrees on each side of the bed separately. I'm recently more and more discovering the incredible, the amazing power of the power nap. Even like 10, 20 minutes can completely transform my mood from kind of drained, worn out, moving and thinking slowly to just excited to take on the day. It's kind of incredible. And the other thing that people suggested and I've tried, and I think it works, is taking like an energy drink or drinking a cup of coffee right before the nap. And that way, when you wake up, There's an extra boost. I don't know if it's a placebo effect or psychological,

• 6:30Lex Fridman?

  but it definitely has worked for me. But when I do the power nap, if it's not on the floor, under the desk, I go with the A sleep mattress. It really is one of the things in my life that brings a smile to my face. Anyway, they have the Pod Pro cover, so you can just add that to your mattress without having to buy theirs. But their mattress is pretty nice, too, just so you know. It can track a bunch of metrics like heart rate variability, but cooling alone is worth the money. Go to 8sleeve.com/lex to get special savings. That's 8sleeve.com/lex. Finally, this episode is brought to you by Element Electrolyte Drink Mix, spelled L-M-N-T. To do low carb diets correctly, the number one thing you have to get right is electrolytes,

• 7:15Lex Fridman?

  specifically sodium, potassium, and magnesium. I think their slogan actually is Salty AF, which is very clever, very clever, Element. They have a new flavor, my new favorite, actually, watermelon salt. I actually really like watermelon when it comes to things that are refreshing. So if I'm looking for a refreshing drink, watermelon is a good to go to flavor. And it really is magical how much difference a bit of salt can make. Salt and water. Having done all kinds of diet stuff combined with performance and exercise, it really is incredible that the times I felt shitty is when I didn't have enough water and enough salt in a good combination of the two. And the flip side of that is been incredible to see how much the

• 7:59Lex Fridman?

  entire thing flips in a matter of minutes when you drink enough water and enough salt. So if you're doing keto stuff, if you're doing fasting, or if you're somebody that exercises a lot, you have to get the electrolytes right. Element works great for me. Olympians use it, tech people use it. I swear by this stuff. Try it at drinklmnt.com/lex. That's drinklmnt.com/lex. This is the Lex Fridman Podcast, and here is my conversation with Po Chen Lo.

• 8:51Lex Fridman?

  You know, you mentioned you really enjoy flying and experiencing different people in different places. There's something about flying for me, I don't know if you have the same experience, that every time I get on an airplane, it's incredible to me that human beings have actually been able to achieve this. And when I look at like, what's happening now with humans traveling out into space, I see it as all the same thing. It's incredible that humans are able to get into a box and fly in the air and safely and land. And the same, it seems like, and everybody's taking it for granted. So when I observe them, it's quite fascinating because I see that cleanly mapping to the world where we're now on, in

• 9:36Lex Fridman?

  rockets and traveling to the moon, traveling to Mars. And at the same kind of way, I can already see the future where we will all take it for granted. So I don't know, I don't know if you have, you personally, when you fly, have the same kind of magical experience of like, how the heck did humans actually accomplish this?

• 9:56Speaker 01

  So I do, especially when there's turbulence, which is, you know, like on the way here, there was turbulence and the plane jiggled, even the flight attendant had to hold on to the side. And I was just thinking to myself, it's amazing that this happens all the time and the wings don't fall off, you know, like, given how many planes are flying. But then I often think about it and I'm like, you know, a long time ago, I think people didn't trust elevators in a 40-story building in New York City. And now we just take it completely for granted that you can step into this shaft, which is 40 floors up and down, and it will just not fail.

• 10:30Lex Fridman?

  Yeah, again, I'm the same way with elevators, but also buildings. When I'll stand on the 40th floor and wonder...

• 10:39Lex Fridman?

  How the heck are we not falling right now? Yeah. How amazing it is with the high winds, like structurally, just the earthquakes and the vibrations. I mean, natural vibrations in the ground. Like, how is this? How are all of these? You go to like New York City, all of these buildings standing. I mean, to me, one of the most beautiful things, actually mathematically too, is bridges. I used to build bridges in high school from like toothpicks, just like out of the pure joy of like physics making some structure really strong. Understanding, like from a civil engineering perspective, what kind of structure will be stronger than another kind of structure, like suspension bridges? And then you see that at scale,

• 11:24Lex Fridman?

  humans being able to span a body of water with a giant bridge. And it's, I don't know, it's so humbling. It makes you realize how dependent we are on each other. Sort of, I talk about love a lot, but there's a certain element in which we little ants have just a small amount of knowledge about our particular thing. And then we're depending on a network of knowledge that other experts hold. And then most of our lives, most of the quality of life we have has to do with the richness of that network of knowledge, of that collaboration, and then sort of the ability to build on top of it.

• 12:09Lex Fridman?

  Levels of abstractions. You start from like bits in a computer, then you can have assembly, then you can have C++. So you have an operating system, then you can have C++ and Python. Finally, some machine learning on top. All of these are abstractions. And eventually we have AI that runs all of us humans. But anyway. But speaking of abstractions in programming, in high school, you wrote some impressive games for MS-DOS. I got a chest to, in browser somehow. It's magic. Got a chest to play them.

• 12:39Lex Fridman?

  Alien Attack 1, 2, 3, and 4. What's the hardest part about programming those games? And maybe can you tell the story about about building those games?

• 12:48Speaker 01

  Sure. I actually tried to do those in high school because I was just curious if I could. Yeah.

• 12:54Lex Fridman?

  It's a good starting point for anything, right?

• 12:56Speaker 01

  Yeah, yeah, yeah. It's like, could you? But the appealing thing was also it was a soup to nuts kind of thing. So something that has always attracted me is I like beautiful ideas. I like seeing beautiful ideas, but I actually also like seeing execution of an idea all the way from beginning to end in something that works. So for example, in high school, I was lucky enough to grow up in the late 90s when even a high school student could hope to make something sort of comparable to the shareware games that were out there. Not, I say the word sort of, like still quite far away, but at least I didn't need to hire a 3D CG artist. There weren't enough pixels to draw anyway, even I can draw, right? Bad art, of course.

• 13:37Speaker 01

  But the point is, I wanted to know, is it possible for me to try to do those things where back in those days, you didn't even have an easy way to draw letters on the screen in a particular font? You couldn't just say, import a font. It wasn't like Python. So for example, back then, if you played those games in the in the web browser, which is emulating the the old school computer, those, even the letters you see, those are made by individual calls to draw pixels on the screen.

• 14:04Lex Fridman?

  So you built that from scratch, almost building a computer graphics library from scratch?

• 14:08Speaker 01

  Yes. The primitive that I got to use was some code I copied off of a book in assembly of how to put a pixel on a screen in a particular color.

• 14:16Lex Fridman?

  And the programming language was Pascal?

• 14:19Speaker 01

  Ah, yeah. The first one was in Pascal, but then the other ones were in C++ after that.

• 14:25Lex Fridman?

  How's the emulation in the browser work, by the way? Is that trivial? Because it's pretty cool. You get to play these games that have a very much 90s feeling to them.

• 14:33Speaker 01

  Ah, so it's literally making an MS-DOS environment, which is literally running the old dot EXE file.

• 14:39Lex Fridman?

  Wow, in the browser. This is, that could be more amazing than the airplane. So it wasn't so much about the video games. It was more about can you build something really cool from scratch?

• 14:51Speaker 01

  Yes.

• 14:52Lex Fridman?

  And you did a bunch of programming competitions?

• 14:57Lex Fridman?

  What was your interest, your love for programming? What did you learn through that experience? Especially now that as much of your work has taken a long journey through mathematics.

• 15:10Speaker 01

  I think I always was amazed by how computers could do things fast. If I wanted to make it an abstract analysis of why it is that I saw some power in the computer. Because if the computer can do things so many times faster than humans, where the hard part is telling the computer what to do and how to do it. If you can master that, asking the computer what to do, then you could conceivably achieve more things. And those contests I was in, those were the opposite in some sense of making a complete product, like a game is a product. Those contests were effectively write a function to do something extremely efficiently. And if you are able to do that, then you can unlock more of the power of the computer.

• 15:53Lex Fridman?

  But also doing it quickly, there's a time element from the human perspective to be able to program quickly. There's something nice, so there's like almost like an athletics component to where you're almost like an athlete seeking optimal performance as a human being, trying to write these programs. And at the same time, it's kind of art because you're, the best way to write a program quickly is to write a simple program. You still have a damn good solution. So it's not necessarily you have to type fast. You have to think through a really clean, beautiful solution. I mean, what do you think is the use of those programming competitions? Do you think they're ultimately something you would recommend for students,

• 16:38Lex Fridman?

  for people interested in programming or people interested in building stuff?

• 16:41Speaker 01

  Yes, I think so because especially with the work that I've been doing nowadays, even trying to control COVID. Something that was very helpful from day one was understanding that the kinds of computations we would want to do, we could conceivably do on like a four-core cloud machine on Amazon Web Services out to a population which might have hundreds of thousands or millions of people. The reason why that was important to have that back-of-the-envelope calculation with efficient algorithms is because if we couldn't do that, then we would bankrupt ourselves before we could get to a big enough scale. If you think about how you grow anything from small to big, if in order to grow it from small to big, you also already need 10,000 cloud servers, you'll never get to big.

• 17:26Lex Fridman?

  And also, the nice thing about programming competitions is that you actually build a thing that works. So you you finish it. There's a completion thing and you realize, I think there's a magic to it, where you realize that it's not so hard to build something that works, to have a system that successfully takes in inputs and produces outputs and solves a difficult problem. And that directly transfers to building a startup, essentially, that can help some aspect of this world, as long as it's mostly be based on software engineering. Things get really tricky when you have to manufacture stuff. That's why people like Elon Musk are so impressive, that they, it's not just software. Tesla

• 18:10Lex Fridman?

  Autopilot is not just software. It's like you have to actually have factories that build cars. And there's like a million components involved in the machinery required to assemble those cars and so on. But in software, one person can change the world, which is incredible. But on the mathematics side, what, if you look back, or maybe today, what made you fall in love with mathematics?

• 18:37Speaker 01

  For me, I think I've always been very attracted to challenge, as I already indicated with writing the program. I guess if I see something that's hard or supposed to be impossible, it certain, sometimes I say, maybe I want to see if I can pull that off. And with the mathematics, the math competitions presented problems that were hard, that I didn't know how to start, but for which I could conceivably try to learn how to solve them. So I mean, there are other things that are hard, called like get something to Mars, get people to Mars. And I didn't, and I still don't think that I'm able to solve that problem. On the other hand, the math problems struck me as things which are hard, and with significant

• 19:19Speaker 01

  amount of extra work, I could figure it out. And maybe they would actually even be useful. Like that mathematical skill is the core of lots of other things.

• 19:30Lex Fridman?

  That's really interesting. Maybe you could speak to that because a lot of people say that math is hard as a kind of negative statement.

• 19:40Lex Fridman?

  It always seemed to me a little bit like that's kind of a positive statement, that all things that are worth having in this world are hard. I mean, everything that people think about that they would love to do, whether it's sports, whether it's art, music, and all the sciences, they're going to be hard if you want to do something special. So is there something you could say to that idea that math is hard? Should it be made easy? Or should it be hard?

• 20:11Speaker 01

  Ah, so I think maybe I want to dig in a little bit onto this hard part and say, I think the interesting thing about the math is that you can see a question that you didn't know how to start doing it before. And over a course of thinking about it, you can come up with a way to solve it. And so you can move from a state of not being able to do something to a state of being able to do something where you help to take yourself through that instead of somebody else spoon feeding you that technique. So actually, here I'm already digging into maybe part of my teaching philosophy also, which is that I actually don't want to ever just tell somebody, here's how you do something.

• 20:56Speaker 01

  I actually prefer to say, here's an interesting question. I know you don't quite know how to do it. Do you have any ideas? This is, I'm actually coming up with, I'm actually explaining another way that you could try to do teaching. And I'm contrasting this to a method of, watch me do this, now practice it 20 times. I'm trying to say a lot of people consider math to be hard because maybe they can't remember all of the methods that were taught. But for me, I look at the hardness and I don't think of it as a memory hardness. I think of it as a, can you invent something hardness? And I think that if we can teach more people how to do that art of invention in a pure cognitive way, not as hard as the actual hardware stuff, right?

• 21:38Speaker 01

  But like in terms of the concepts and the thoughts and the mathematics, teaching people how to invent, then suddenly, actually, they might not even find math to be that.

• 21:48Speaker 01

  Tyrism is hard anymore.

• 21:49Lex Fridman?

  Right.

• 21:50Speaker 01

  But that rewardingness hard of I have the capability of looking at something which I don't know what to do and coming up with how to do it. I actually think we should be doing that, giving people that capability.

• 22:02Lex Fridman?

  So hard in the same way that invention is hard, that is ultimately rewarding. So maybe you can dig in that a little bit longer, which is,

• 22:14Lex Fridman?

  do you see basically the way to teach math is to present a problem and to give a person a chance to try to invent a solution with minimal amount of information first? Is that basically, how do you build that muscle of invention in a student?

• 22:33Speaker 01

  Yes, so the way that, I guess I have two different sort of ways that I try to teach. Actually, one of them is, in fact, this semester, because all my classes were remotely delivered. I even threw them all onto my YouTube channel. So you can see, you can see how I teach at Carnegie Mellon. But I'd often say, hey, everyone, let's try to do this. Any ideas? And that actually changes my role as a professor from a person who shows up for class with a script of what I want to talk through. Actually, I don't have a script. The way I show up for class is there is something that we want to learn how to do, and we're going to do it by improv. I'm talking about the same method as improv comedy, which is where you tell me some ideas and I'll try to yes and them. You know what I mean?

• 23:16Speaker 01

  And then together, we're going to come up with a proof of this concept where you were deeply involved in creating the proof. Actually, every time I teach the class, we do every proof slightly differently because it's based on how the students came up with it. And that's how I do it when I'm in person. I also have another line of courses that we make that is delivered online. Those things are where I can't do it live, but the teaching method became also similar. It was just, here's an interesting question. I know it's out of reach. Why don't you think about it? And then automatic hints, we feed automatic hints through the internet to go and let the person try to invent.

• 23:55Lex Fridman?

  So that's like a more rigorous prodding of invention. But you did mention disease and COVID and you've been doing some very interesting stuff from a mathematical but also software engineering angle of coming up with ideas. It's back to the, I can, I see a problem, I think I can help. So you stepped into this world. Can you tell me about your work there under the flag of NOVID and both the software and the technical details of how the thing works?

• 24:29Speaker 01

  Sure, sure. So first I want to make sure that I say this is actually a team effort. I happen to be the one speaking, but there's no way this would exist without an incredible team of people who inspire me every day to work on this, but I'll speak on behalf of them. So the idea was indeed that we stepped forward in March of last year when the world started to become, our part of the world started to become, our part meaning the United States, started to become paralyzed by COVID. The shutdown started to happen. And at that time, it started as a figment of an idea, which was network theory, which is the area of math that I work in, could potentially be combined with smartphones and some kind of health information anonymized. Exactly how?

• 25:14Speaker 01

  We didn't know yet. We tried to crystallize it. And many months into this work, we ended up accidentally discovering a new way to control diseases, which is now what is the main impetus of all of this work, is to take this idea and polish it and hopefully have it be useful not only now, but for future pandemics. The idea is really simple to describe. Actually, my main thing in the world is I come up with obvious observations. That's, that's, I'll explain it now.

• 25:42Lex Fridman?

  Einstein did the same thing and he wrote a few short papers.

• 25:47Speaker 01

  But so the idea is like this. If we describe how usually people control disease for a lot of history, it was that you'd find out who was sick, you'd find out who they have been around, and you try to remove all of those people from society against their will. Yes. Now, that's the problem. The against their will part gives you the wrong kind of a feedback loop, which makes it hard to control the disease, because then the people you're trying to control keep getting other people sick. You can see already how I'm thinking and talking about this. Feedback loops. This is actually related to something you said earlier about even like how skyscrapers stay in the air. The whole point is control theory. You actually want to, or even how an airplane stays.

• 26:30Speaker 01

  You need to have control loops which are feedbacking in the right way. And what we observed was that the feedback control loop for controlling disease by asking people to be removed from society against their will was not working. It was running against human incentives, and you suddenly are trying to control seven billion, eight billion people in ways that they don't individually want to necessarily do. So here's the idea. And this is inspired by the fact that at the core of our team were user experience designers. That's actually, in fact, the first thing I knew we needed when we started was to bring user experience at the core. Okay. But so the idea was suppose there was a pandemic. Suppose hypothetically, there was a pandemic. What would you want?

• 27:15Speaker 01

  You'd want a way to be able to live your life as much as possible and avoid getting sick. Can we make an app to help you avoid getting sick? Notice how I've just articulated the problem. It is not can we make an app so that after you are around somebody who's sick, you can be removed from society? It's can we make an app so that you can avoid getting sick? That would run a positive, I don't know if I want to call it positive or negative, but it would run a good feedback loop. Okay. So then how would you do this? The only problem is that you don't know who's sick. Because especially with this disease, if I see somebody who looks perfectly healthy, the disease spreads two days before you have any symptoms. And so it's actually not possible.

• 27:59Speaker 01

  That's where the network theory comes in. You caught it from someone. What if we changed the paradigm and we said, whenever there's a sickness, tell everybody how many physical relationships separate them from the sickness? That is the trivial idea we added. The trivial idea was the distance between you and a disease is not measured in feet or seconds. It's measured in terms of how many close physical relationships separate you, like these six degrees of separation, like LinkedIn. Simple idea. What if we told everyone that? It turns out that actually unlocks some interesting behavioral feedback loops,

• 28:40Speaker 01

  which, for example, let me, let me now jump to a non-COVID example to show why this maybe could be useful. Actually, we think it could be quite useful. Imagine there was Ebola or some hemorrhagic fever. Imagine it spread through contact, through the air, in fact. Pretend, pretend.

• 28:57Speaker 01

  That's a disastrous disease. It has high fatality rate and as you die, you're bleeding out of every orifice.

• 29:08Speaker 01

  Okay?

• 29:08Lex Fridman?

  Yeah, not pleasant.

• 29:09Speaker 01

  Not pleasant. So the question is, suppose that such a disease broke,

• 29:15Speaker 01

  who would want to install an app that would tell them how many relationships away from them this disease had struck?

• 29:22Lex Fridman?

  A lot of people.

• 29:23Speaker 01

  A lot of people. In fact, almost, I don't want to say almost everyone. That's a very strong statement, but a very large number of people.

• 29:30Lex Fridman?

  That's fascinating framing, like the more deadly and transmissible the disease, the stronger the incentive to install it in a positive sense, the, in the good feedback loop sense. That's a really good example. It's a really good way to frame it because with COVID, it was not as deadly as potential pandemics could have been, viruses could have been. So it's sometimes muddled with how we think about it. But yeah, this is a really good framing. If the virus was a lot more deadly, you want to create a system that has a set of incentives that it quickly spreads to the population where everybody is using it and is contributing in a positive way to the system.

• 30:11Speaker 01

  Exactly. And actually, that point you just made, I don't take credit for that observation. There was another person I talked to who pointed out that it's very interesting that this feedback loop is even more effective when the disease is worse. And that's actually not a bad characteristic to have in your feedback loop if you're trying to help civilization keep running.

• 30:31Lex Fridman?

  Yeah, it's a really, it's in this dynamic. Like, people figure out, they dynamically figure out how bad the disease is. The more it spreads, and the deadlier it is as the people observe it, as long as the spread of information, like semantic information, natural language information, is closely aligned with the reality of the disease, which is a whole nother conversation, right? We, that's, we might, maybe we'll chat about that, how we sort of make sure there's not misinformation while there's accurate information. But that aside, okay, so this is a really nice property.

• 31:06Speaker 01

  Right, and, and, and just going on on that, actually just talking more about what that could do and why we're so excited about it. It's that not only would people want to install it, what would they do? If you start to see that this disease is getting closer and closer, we, we surveyed informally people, but they said, as we saw it getting closer, we would hide, we would try to not have contacts. But now, you notice what this has just achieved. The whole goal on, on, on this whole exercise was, you got the people who might be sick, and you got everyone else, Set A and Set B. Set A is the people who might be sick, Set B is everyone else. And for the entirety of the past, contact tracing approaches,

• 31:47Speaker 01

  you tried to get Set A to do things that might not be to their liking or their will, because that's removing them from society.

• 31:54Lex Fridman?

  Yes.

• 31:55Speaker 01

  We found out that there's two ways to separate Set A from Set B. You can also let the people at Set B, at the fringe of Set A, attempt to remove themselves from this interface. It's just, it's the symmetry of A and B separation. Everyone was looking at A, we look at B, and suddenly B isn't their incentive to do so.

• 32:14Lex Fridman?

  Beautiful. So there's a virus that jumps from human to human. So there's a network, sometimes called graph, of the spread of a virus. It hops from person to person to person to person. And each one of us individuals are sitting, or plop, plopped into that network. We have close friends and relations and so on. It's kind of fascinating to actually think about this network. And we can maybe talk about the shapes of this kind of network. Because I was trying to think exactly this, like, how many people do I, well, some kind of an introvert, not kind of, I'm very much an introvert, but, so can I be explicit about the kind of people I meet in regular life, say, when it was completely opened up, there's

• 32:59Lex Fridman?

  no pandemic.

• 33:02Lex Fridman?

  There is a kind of network, and there's maybe, in the graph, theoretic sense, there's some weights or something about how close that relationship is in terms of the frequency of visits, the duration of visits, and all of those kinds of things. So you're saying, we might want to be, to create, on top of that network, a spread of information to let you know, as the virus travels through this network, how close is it getting to you? And the number of hops away it is on that network is really powerful information that creates a positive feedback loop, where you can act essentially anonymously and on your

• 33:47Lex Fridman?

  own. Like, nobody's telling you what to do, which is really important, is decentralized, and not, whatever the opposite of authoritarian is, but you get to, sort of, the American way, you get to choose to do it yourself, you have the freedom to do it yourself, and you're incentivized to do it, and you're most likely going to do it, to protect yourself against you getting the disease, as the closer it gets to you, based on the information that you have. But can you maybe elaborate? First of all, brilliant. Whenever I saw the thing you're working on, so, forget for COVID, this is, of course,

• 34:30Lex Fridman?

  really relevant for COVID, but it's also probably relevant for future diseases as well. So this, that was the thing I'm nervous about, is like, if this whole, if our society shut down because of COVID, like, what the heck is gonna happen when there's a much deadlier disease? Like, this, this is disappointing, the whole time... 2020, the whole time I'm just sitting like this, like is the incompetence of everybody except the people developing vaccines. The biologists are the only ones that got their stuff together. But in terms of institutions and all that kind of stuff, it's just been terrible. But this is exactly the power of information and the power of information that doesn't limit personal freedom.

• 35:15Lex Fridman?

  So your idea is brilliant. Okay. Mathematically, can you maybe elaborate? What are we talking about? Like, how do you actually make that work? What's involved?

• 35:25Speaker 01

  Sure. First, I'm going to reply to something you said about the freedom inside this because actually that was the idea. The idea is this is game theory, right? And effectively, what we did is analogous to free market economy as opposed to central planning.

• 35:41Lex Fridman?

  Yeah.

• 35:42Speaker 01

  If you just line up the set of incentives correctly so that people have in their purely selfish behavior are contributing to the optimization of the global function.

• 35:54Lex Fridman?

  Yes.

• 35:54Speaker 01

  That's it. And the point of what we do, I guess, in mathematics is we try to explore the search space to go and find out as many possibilities as there are. And in this case, it's an applied, in this case, it's an applied search space. That's why the inputs from design, user experience design and actual people are important. But you asked about, I guess, the mathematical or the technical things underpinning it. So I think the first thing I'll say is we wanted to make this thing not require your personal information. And so in order to do that, what gave me the confidence to, I guess, lead our team to run at the beginning is we saw that this could be done without using GPS information. So technically, what's going on is if two smartphones, it's a smartphone app, if two

• 36:36Speaker 01

  smartphones have this thing installed, they just communicate with each other by Bluetooth to go and find out how far, they can detect nearby things by Bluetooth. And then they can find out that these two phones were approximately such and such distance apart. And that kind of relative proximity information is enough to construct this big network.

• 36:58Lex Fridman?

  Okay, so the physical network is constructed based on proximity that's through Bluetooth and you don't have to specify your exact location, it's the proximity.

• 37:08Speaker 01

  I'm not using the Pythagorean theorem, basically. I mean, if I just knew the GPS coordinates, we could use the Pythagorean theorem too. Sorry, that's just how I call it. Distance formula, whatever you want to call it.

• 37:20Lex Fridman?

  Yeah, so we're not doing the old Pythagorean based violation of privacy. Okay. But so is that? Is that enough to form? To give you enough information about physical connection to another human being? Is there a time element there? Is there so? So, okay. That sounds like a really strong, like low hanging fruit. Like if you have that, you could probably go really, really far. My natural question is, is there extra information you can add on top of that? Like the duration of the physical proximity?

• 38:00Speaker 01

  So first of all, we actually do estimate the duration. But the way we estimate the duration is like how a movie is filmed. In the sense that every so often, every few minutes, we check what's nearby. It's like how a movie is filmed. You take lots of snapshots.

• 38:14Lex Fridman?

  Yes.

• 38:15Speaker 01

  So there's no way in a battery efficient way to really keep track of that proximity. However, fortunately, we're using probability. The fact is, the paradigm that we're using is it's not super important if you run into that person only for 10 minutes at the grocery store. If that's a stranger that you run into 10 minutes in this grocery store, that's not going to be relevant for our paradigm. Because our paradigm is not telling you who were you around before and might therefore have gotten infected by already. Ours is about predicting the future. We changed from, I mean, the standard paradigm was what already happened, quick damage control. Ours is predict the future. If you run into that person once in the grocery store today and never see them again, it's irrelevant for predicting the future.

• 38:59Speaker 01

  And therefore, for ours, what really matters is the many hours around the other person. At which point, if you're scanning every five to eight minutes.

• 39:08Lex Fridman?

  That's going to come out in the problem, like statistically speaking, it's going to come out as a strong relationship. And a person in the grocery store is going to wash out as not as not an important physical relationship. I mean, this is brilliant. What? How difficult is it to make work? So you said, one, there's a mathematical component that we just kind of talked about. And then there's the user experience component. So how difficult is it to go, just like you built the video game Alien Attack, from zero to completion? What's involved? How difficult is it?

• 39:42Speaker 01

  So I'm going to answer that question in terms of building the product. But then I'm also going to acknowledge that just having an app doesn't make it useful. Because that's actually maybe the easy part. If you know what I mean. There's like all of this stuff about rollout, adoption and awareness. But let's focus on the app part first. So that's again why I said that the team is incredible. So we have a bunch of people who, let's just say that the technology that we use to make it is not the standard way you make an app. If you think about a standard iOS app or Android app, those are a user interface that contacts a web server and sends some information back and forth. We're doing some stuff that has to hook into the operating system of saying,

• 40:25Speaker 01

  let's go use Bluetooth for something it wasn't really meant for. Right? So there's that part.

• 40:30Lex Fridman?

  By the way, what is the app called?

• 40:31Speaker 01

  Oh, it's called NOVID, COVID with an N.

• 40:36Lex Fridman?

  Very nice. So you have to hook into Bluetooth. You're saying you have to do that beyond the permissions that are like at the very surface level provided on the phone?

• 40:48Speaker 01

  Well, I don't want to call them permissions. I just want to say that's not what you usually do with Bluetooth.

• 40:52Lex Fridman?

  Gotcha.

• 40:53Speaker 01

  Usually with Bluetooth, you say, do I have headphones nearby?

• 40:56Lex Fridman?

  Yes.

• 40:57Speaker 01

  Okay, I'm done. You don't go and say, do I have headphones nearby? Or do I have another phone nearby which is doing something? And then keep asking that same question.

• 41:04Lex Fridman?

  Keep asking the question.

• 41:05Speaker 01

  Right? So this is actually not easy. And I mean, there were some parts of it which actually a lot of people had tried unsuccessfully. Actually, it's known that, for example, the UK was trying to do something similar. And the problem they ran into was when you program things on iOS, iOS is very good at making it hard to do things in the background. And so there was quite a lot of effort required to go and make this thing work.

• 41:32Lex Fridman?

  So the whole point, this thing would run in the background and iOS, I mean, most. Android probably as well, right? But yeah, iOS certainly makes it difficult for something to run in the background, especially when it's eating up your battery, right?

• 41:47Speaker 01

  Ah, well, we wanted to make sure we didn't eat up the battery. So that one, we can, we actually are very proud of the fact that ours uses very little battery, actually, even if compared to Apple's own system, so.

• 41:59Lex Fridman?

  Beautiful. So what else is required to make this thing work?

• 42:01Speaker 01

  Right, so the key was that you had to do significant amount of work on the actual mobile app development, which fortunately, the team that we brought was this kind of general thinkers, where we would dig in deep into the operating system documentation and the API libraries. So we got that working. But there's another angle, which is you also need the servers to be able to compute fast enough, which is tying back to this old school computer programming competitions and Math Olympians. In fact, our team that was working on the algorithm and backend side included several people who had been in these competitions from before, which I happen to know because I do coach the team for the math.

• 42:40Lex Fridman?

  Yes.

• 42:40Speaker 01

  And so we were able to bring people in to build servers, a server infrastructure in C++ actually, so that we could support significant numbers of people without needing tons of servers.

• 42:52Lex Fridman?

  Is there some distributed algorithms working here? Or you basically have to keep in the same place the entire graph as it builds? Because especially the more and more people use it, the bigger, the bigger the graph gets. I mean, this is very difficult scaling problem, right?

• 43:09Speaker 01

  Ah, so that's actually why this computer algorithm competition stuff was handy. It's because there are only about seven to eight gigabit people in the world.

• 43:20Lex Fridman?

  Yeah.

• 43:21Speaker 01

  That's not that many. So if you can make your algorithms linear time or almost linear time, a computer operates in gigahertz.

• 43:29Lex Fridman?

  Yeah.

• 43:29Speaker 01

  I only need to do one run, one recalculation every hour in terms of telling people how far away these dangers are.

• 43:36Lex Fridman?

  Yes.

• 43:37Speaker 01

  So I suddenly have 3,600 seconds and my CPU cores are running in gigahertz. And at most, there are eight gigabit people.

• 43:46Lex Fridman?

  Well, you're skipping over the fact that there's n squared potential connections between people. So how do you get around the fact that, you know, that we, you know, the potential set of relationship any one of us could have is eight billion. So it's eight billion times squared. That's the potential amount of data you have to be storing and computing over and constantly updating.

• 44:12Speaker 01

  So the way we dealt with that is we actually expect that the typical network is very sparse. The technical term sparse would mean that the average degree or the average number of connections that a person has is going to be at most like a hundred strong connections that you care about. If you think of it almost in terms of the heavy hitters, actually, in most people's lives, if we just kept track of their top hundred interactions, that's probably most of the signal.

• 44:42Lex Fridman?

  Yeah, yeah, I'm saddened to think that I might not be even in the double digits, but.

• 44:49Speaker 01

  Oh, I was intentionally giving a crazy number to account for college students.

• 44:55Lex Fridman?

  You call, oh, those are who you call on the heavy hitters, the people who are like the social butterflies. Yeah.

• 44:59Speaker 01

  Yeah, yeah.

• 45:00Lex Fridman?

  I need to, I'd love to know that information about myself, by the way. That, do you expose the graph, like how many, like about yourself, how many connections you have?

• 45:13Speaker 01

  We do expose to each person how many direct connections they have.

• 45:17Lex Fridman?

  That's great.

• 45:17Speaker 01

  But for privacy purposes, we don't tell anybody who their connections, like how their connections are interconnected.

• 45:23Lex Fridman?

  Yes, gotcha.

• 45:24Speaker 01

  But at the same time, we do expose also to everyone an interesting chart that says, here's how many people you have that you're connected to directly. Here's how many at distance two, meaning via people. And then here's how many at distance three. And the reason we do that is that actually ends up being a dynamic that also boosts adoption. It drives another feedback loop. The reason is because we saw, actually, when we deployed this in some universities, that when people see on their app that they are indirectly connected to hundreds or thousands of other people, they get excited and they tell other people, hey, let's download this app.

• 45:57Lex Fridman?

  Yeah.

• 45:58Speaker 01

  But you know, we also saw in those examples, especially looking at the screenshots people gave, that is hit as soon as the typical person has two or three other direct connections on the system. Because that means that our app has reached a virality R naught of two to three. The key is we were making a viral app to fight a virus spreading on the same network that the virus spreads on.

• 46:22Lex Fridman?

  So you're trying to out-virus the virus?

• 46:24Speaker 01

  That's right.

• 46:26Speaker 01

  That's exactly right.

• 46:27Lex Fridman?

  Okay, great. What have you learned from this whole experience in terms of,

• 46:33Lex Fridman?

  let's say for COVID, but for future pandemics as well? Is it possible to use the power of information here of networked information as the virus spreads and travels in order to basically keep the society open? Is it possible for for people to protect themselves with this information? Or do you still have to have most like in this overarching policy of everybody should stay at home? All that kind of thing?

• 47:01Speaker 01

  We are trying to answer that question right now. So the answer is, we don't know yet. But that's actually why we're very happy that now the idea has started to become more widely known. And we're already starting to collaborate with epidemiologists. Again, I'm just a mathematician, right? And a mathematician should not be the person who is telling everybody, this will definitely work. But because of the potential power of this approach, especially the potential power of this being an end game for COVID, we have gotten the interest of real researchers. And we're now working together to try to actually understand the answer to that question. Because you see, there's a theory. So what I can share is the mathematics of here's why there's some hope that this would work. And that's because I'm talking about end game now.

• 47:46Speaker 01

  End game means you have very few cases. But everywhere, we're always thinking, once there's few cases, then does that mean we now open up? Once you open up in the past, then the cases go up again until you have to lock down again. And now, when we talk about the dynamic process that makes, it's guaranteeing you always have cases until you have the great vaccines, which is, you know, we both got vaccinated. This is this is good. But at the same time, why I'm thinking this is still important is because we know that many vaccine makers have said they're preparing for the next dose next year. And if we have a perpetual thing where you just always need a new vaccine every year, it could actually be beneficial to make sure we have as many other techniques as possible for parts of the world that can't afford, for example, that kind of distribution.

• 48:30Lex Fridman?

  Yeah, so actually, no matter, no matter how deadly the virus is, no matter how many things, whether you have a vaccine or not, it's still useful to be having this information.

• 48:39Speaker 01

  Yes.

• 48:39Lex Fridman?

  Is to stay home or not, depending on how risky like. I'm a big fan, just like you said, of having the freedom for you to decide how risk averse you want to be, right? Depending on your own conditions, but also on the state of like what you, just how dangerously you like to live.

• 48:57Speaker 01

  So I think that actually makes a lot of sense. And I also think that since we're, when you think of disease spreading, it spreads in aggregate in the sense that if there are some people who maybe are more risk tolerant because of other things in their life, well, there might also be other people who are less risk tolerant. And then those people decide to isolate. But what matters is in the aggregate that this R not of the infection spreading drops below one. And so the key is if you can empower people with that power to make that decision, you might actually still be able to drive that R not down below one.

• 49:34Lex Fridman?

  Yeah. And also, this is me talking, is people get a little bit nervous, I think, with information somehow mapping to privacy violation. But I, first of all, in the approach you're describing, that's respecting anonymity. But I would love to have information from the very beginning, from March and April of last year, almost like a map of like where it's risky and where it's not to go. And not map based on sort of the exact location of people, but where people usually hang out kind of thing.

• 50:15Lex Fridman?

  Just, it may be not necessarily about actual location, but just maybe activities, like just to have information about what is, what is good to do and not, you know, in terms of like safety. Is it okay to run outside and not? Is it okay to go to a restaurant and not? I just feel like we're operating the blind. And then what you had is a very imperfect signal, which is like basically politicians desperately trying to make statements about what is safe and not. They don't know what the heck they're doing. They have a bunch of smart scientists telling them stuff and the scientists themselves also. Very important, don't always know what they're doing.

• 50:57Lex Fridman?

  Epidemiology is not, is as much an art as a science. You're desperately trying to predict the future, which nobody can do. And then you're trying to speak with some level of authority. I mean, if I were to criticize scientists, they spoke with too much authority. It's okay to say, I'm not sure. But then they think like, if I say, I'm not sure, then there's going to be a distrust. What they realize is when you're wrong and you say, I'm sure, it's going to lead to more distrust. So there's this imperfect, like, just chaotic, messy system of people trying to figure out with very little information. And what you're proposing is just a huge amount of information. And information is power.

• 51:38Lex Fridman?

  Is there challenges with adoption that you see in the future here? So there's, maybe we could speak to, there's approaches, I guess, from Google. There's different people that've tried similar kind of ideas. Not, you have quite a novel idea, actually. But speaking, the umbrella idea of contact tracing. Is there, is there something you can comment about why their approaches haven't been fully adopted? Is there challenges there? Is there, is there reasons why Novit might be a better idea moving forward? And in general, just about adoption?

• 52:16Speaker 01

  Yeah. So first of all, I want to say, I always have respect for the methods that other people use. And so it's, it's good to see that other people have been trying. But what we have noticed is that the difference between our value proposition to the user and the value proposition to the user delivered by everything that was made before is that unfortunately, the action of installing a standard contact tracing app will then tell you after you've already been exposed to the disease so that you can protect other people from you. And what that does to your own direct probability of getting sick? If you think about it, suppose you were making the decision, should I or should I not install one of those apps? What does that do to your own probability of getting sick?

• 53:02Speaker 01

  It's close to zero.

• 53:03Lex Fridman?

  This is the sad thing you're speaking to. Not sad, I suppose it's the way of the world is the only incentive there is to just help other people, I suppose, but a much stronger incentive is anything that allows you to help yourself.

• 53:20Speaker 01

  Yes, so what I'm saying is that, let's just say, free market capitalism was not based on altruism. I think it's based on if you make a system of incentives so that everybody trying to maximize their own situation somehow contributes to the whole, that's a game theoretic solution to a very hard problem. And so this is actually basically mechanism design. We've basically come up with a different mechanism, different set of incentives, which incentivizes the adoption. Because actually, whenever we've been rolling it out, usually the first question we ask people, like say in a university, is do you know what Novit does? And most of them have read about the other apps and they say, oh, Novit will tell you after you've been around someone so you can quarantine. And we have to explain to them, actually, Novit never wants to ask you to quarantine.

• 54:05Lex Fridman?

  Yeah.

• 54:05Speaker 01

  That's not the principle. Our principle isn't based on that at all. We just want to let you know if something is coming close so that you can protect yourself.

• 54:14Lex Fridman?

  If you want.

• 54:15Speaker 01

  Yeah, if you want, if you want, if you want. And then the quarantine is like, yes, in that case, if you're quarantining, it's because you're shutting the door from the inside, if that makes sense.

• 54:24Lex Fridman?

  Yes, exactly, exactly. I mean, this is brilliant. So what do you think the future looks like for future pandemics? What's your plan with Novit? What's your plan with these set of ideas?

• 54:36Speaker 01

  I am actually still an academic and a researcher. So the biggest work I'm working on right now is to try to build as many collaborations with other public health researchers at other universities to actually work on pilot deployments together in various places. That's the goal. That's actually ongoing work right now. And so, for example, if anyone's watching this and you happen to be a public health researcher and you want to be involved in something like this, I'm just going to say I'm still incentive thinking. There is something in it for the researchers, too. This could open up an entire new way of controlling disease. That's my hope. I mean, it might actually be true. And people who are involved in figuring out how to make this work. Well, it could actually be good for their careers, too. I always have to think, like, if a researcher was getting involved, what are they getting out of it?

• 55:21Lex Fridman?

  Oh, so you mean, like, from a research perspective, you can, like, publications and sets of ideas about how to, from a sort of network theory perspective, understand how we control the spread of a pandemic?

• 55:37Speaker 01

  Yes, and what I'm doing right now is, this is basically interdisciplinary research, where maybe our side is bringing the technology and the network theory, and the missing parts are epidemiology and public health expertise. And if the two things start to join, also because everywhere that you deploy, let's just say that the world is different in the Philippines as it is in the United States. And just the natures of the locality would mean that someone like me should not be trying to figure out how to do that. But if we can work with the researchers who are based there, now suddenly, we might come up with a solution that will help scale in parts of the world, where they aren't all getting the Moderna and Pfizer vaccines, which cost like $20 a pop in the US.

• 56:14Lex Fridman?

  So if they want to participate, who do they reach out to?

• 56:18Speaker 01

  Oh, that would just be us. I mean, the novid.org website has.

• 56:21Lex Fridman?

  novid.org?

• 56:21Speaker 01

  It has a feedback reach out form. And actually, we are, I mean, again, this is the DNA of being a researcher. I am actually very excited by the idea that this could contribute knowledge that will outlast all of our generations, like, all of our lifetimes.

• 56:37Lex Fridman?

  There you go. Reach out to novid.org. What about individual people? Should they install the app and try it out, or is this really geographically restricted?

• 56:47Speaker 01

  Oh, yeah, I didn't come on here to tell everyone to install the app. I did not come to tell everyone to install the app, because it works best if your local health authority is working with us.

• 56:56Lex Fridman?

  Gotcha.

• 56:57Speaker 01

  There's a reason. It's because, this is back to the game theory. If anyone could just say, "I'm positive." The high school senior prank would be to say that we have a massive outbreak on finals week. Let's not have final exams. So the way that our system works, it actually borrows some ideas, not borrows, we came up with them independently. But it's ideas similar to what Google and Apple do, which is that if the local health authority is working with this, they can, for everyone who's positive, give them a passcode that expires in a short time. So for ours, if you're on the app and saying, I'm positive, you can either just say that, and that's called unverified, or you can enter in one of these codes that you got from the local health authority. So basically, for anyone who's watching this, it's not that you should just go and download

• 57:41Speaker 01

  it, unless you want to go and look at it, that's cool. But if you, on the other hand, if you happen to know anyone at the local health authority which is trying to figure out how to handle COVID, well then, I mean, we'd be very happy to also work with you.

• 57:53Lex Fridman?

  Gotcha. So the verified there is really important because you're maintaining anonymity and because of that, you have to have some source of verification in order to make sure that it's not possible to manipulate. Because it's ultimately about trust and information, and so it could be, verification is really important there. So basically, individual people should ask their local health authorities to sign up to contact you. I hope this spreads. I hope this spreads for future pandemics, because I'm really, it's the amount, the millions of people who are hurt by this, I think are response to the virus, economically speaking.

• 58:37Lex Fridman?

  The number of people who lost their dream, lost their jobs, but also lost their dream entrepreneurs, you know, jobs often give meaning. There's people who financially and psychologically are suffering because of our, I'll say, incompetent response to the virus across the world, but certainly the United States, that should be the beacon of entrepreneurial hope for the world. So I hope that we'll be able to respond to these kinds of events much better in the future. And this is exactly the right kind of idea, and now is the time to do the investment. Let's step back to the beauty of mathematics.

• 59:20Lex Fridman?

  Maybe ask the big silly question first, which is, what do you find beautiful about mathematics?

• 59:28Speaker 01

  I think that being able to look at a complicated problem, which looks unsolvable, and then to be able to change the perspective, to come from a different angle and suddenly see that there's a nice solution. I don't mean that every problem in math is supposed to be this way, but I think that these reframings and changing of perspectives that cause difficult things to get simplified and crystallized and factored in certain ways is beautiful. Actually, that's related to what we were just talking about with even this fighting pandemics. The crystal idea was just quantify proximity by the number of relationships in the physical network instead of just by the feet and meters, right?

• 1:00:12Speaker 01

  It's just that if you change that perspective, now all of these things follow. And so mathematics, to me, is beautiful in the pure sense just for that.

• 1:00:22Lex Fridman?

  Yeah, it's quite interesting to see a human civilization as a network, as a graph, and our relationships as kind of edges in that graph. And to then do, outside of just pandemic, do interesting inferences based on that. This is true for Twitter, social networks and so on. How we expand the kind of things we talk about, think about sort of politically, if you have this little bubble, quote unquote, of ideas that you play with, it's nice from a recommender system perspective, how do you jump out of those bubbles? It's really fascinating. YouTube is working on that. Twitter is working on that, but not always so successfully.

• 1:01:07Lex Fridman?

  But there's a lot of interesting work from a mathematical and a psychological, sociological perspective there within those graphs. But if we look at the cleanest formulation of that, of looking at a problem from a different perspective, you're also involved with the International Mathematics Olympiad, which takes a small, clean...

• 1:01:32Lex Fridman?

  Problems that are really hard, but once you look at them differently can become easy. But that little jump of innovation is the entire trick. So maybe at the high level, can you say what is the International Mathematical Olympiad?

• 1:01:48Speaker 01

  Sure. So this is the competition for people who aren't yet in college, math competition, which is the most prestigious one in the entire world. It's the Olympics of mathematics, but only for people who aren't yet in college. Now, the kinds of questions that they ask you to do are not computational. Usually, you're not supposed to find that the answer is 42. Right? Instead, you're supposed to explain why something is true.

• 1:02:14Lex Fridman?

  Yes.

• 1:02:15Speaker 01

  And the problem is that at the beginning, when you look at each of the questions, first of all, you have four and a half hours to solve three questions. And this is one day. And then you have a second day, which is four and a half hours, three questions. But when you look at the questions, they're all asking you, explain why the following thing is true, which you've never seen before. And by the way, even though there are six questions, if you solve any one of them, you're a genius and you get an honorable mention. So this is hard to solve.

• 1:02:39Lex Fridman?

  Really hard problem. So what about, is it one person? Is it a team?

• 1:02:42Speaker 01

  Ah, so it's, each country can send six people and the score of the country is actually unofficial. There's not an official country versus country system, although everyone just adds up the point scores of the six people and they say, well, now which country stacked up where?

• 1:02:58Lex Fridman?

  Yeah. So maybe as a side comment, I should say that there's a bunch of countries, including the former Soviet Union and Russia, where I grew up, where this is one of the most important competitions that the country participates in. Like, it was a source of pride for a lot of the country. You look at the Olympic sports, like wrestling, weightlifting, there's certain sports and hockey that Russia and the Soviet Union truly took pride in. And actually, the Mathematical Olympiad.

• 1:03:36Lex Fridman?

  It was one of them for many years, is still one of them. And that's kind of fascinating. We don't think about it this way in the United States. Maybe you can correct me if I'm wrong, but it's not nearly as popular in the United States in terms of its integration into the culture, into just basic conversation, into the pride. Like, you know, if you win an Olympic gold medal or if you win the Super Bowl, you can walk around proud. I think that was the case with the Mathematical Olympiad in Russia. Not, not as much the case in the United States, I think. So I just want to give that a little aside because beating anybody from Russia, from the Eastern Republic or from China is very, very difficult.

• 1:04:19Lex Fridman?

  Like, if I remember correctly, you know, there's people, this was a multi-year training process. They train hard and this is, this is everything that they're focused on. My, uh, my dad was, was a participant in this and it's, I mean, it's, uh, it's as serious as Olympic sports. You think about like gymnastics, like young athletes participating in gymnastics. This is as serious as that, if not more serious. So I just want to give that a little bit of context because we're talking about serious, high-level math athletics almost here.

• 1:04:53Speaker 01

  Yeah, and actually I also think that it made sense from the Soviet Union's perspective because if you look at what these people do eventually, even though, let's look at the USSR's Olympiad, International Math Olympiad record, even though they, I say even though they won a lot of awards at the high school thing, many of them went on to do incredible things in research mathematics or research other things. And that's showing the generalization, generalizability of what they were working on. Because ultimately, we're just playing with ideas of how to prove things. And if you get pretty good at inventing creative ways to turn problems apart, split

• 1:05:35Speaker 01

  them apart, observe neat ways to turn messy things into simple crystals, well, if you're going to try to solve any real problem in the real world, that could be a really handy tool too. So I don't think it was a bad investment. I think clearly it worked well for Soviet Union.

• 1:05:52Lex Fridman?

  Yeah, so this, this is interesting. People sometimes ask me, you know, you grew up in, under communism, you know, was there anything good about communism?

• 1:06:03Lex Fridman?

  And it's difficult for me to talk about it because it's not communism is one of those things that's looked down on like without in absolutist terms currently. But you could still, in my perspective, talk about the actual, forget communism or whatever the actual term is, but you know, certain ways that the society function that we can learn lessons from. And one of the things in the Soviet Union that was highly prized is knowledge, not even knowledge, wisdom and the skill of invention, of innovation at a young age. So we're not talking about a selection process where you pick the best students in the school

• 1:06:47Lex Fridman?

  to do the mathematics or to read literature. It's like everybody did it. Everybody, it was almost treated as if anyone could be the next Einstein, anybody could be the next, I don't know, Hemingway, James Joyce. And so you're forcing an education on the populace and a rigorous, deep education, like as opposed to kind of like, oh, we want to make sure we, we teach to the weakest student in the class, which American systems can sometimes do because we don't want to leave anyone behind.

• 1:07:26Lex Fridman?

  The Russian system was anyone can be the strongest student and we're going to teach you the strongest student and we're going to just to pretend or force everybody, even the weakest student, to be strong. And what that results in, it's obviously, this is what people talk about is a huge amount of pressure. Like it's psychologically very difficult. This is why people struggle when they go to MIT, this very competitive environment. It can be very psychologically difficult, but at the same time, it's bringing out the best out of people. And that mathematics was certainly one of those things. And exactly what you're saying, which kind of clicked with me just now, as opposed to kind of a spelling bee in the United States, which I guess you spell, I'm horrible at

• 1:08:10Lex Fridman?

  this, but it's a competition about spelling, which I'm not sure, but you could argue doesn't generalize well to the future skills. Mathematics, especially this kind of mathematics, is essentially a formalized competition of invention, of.

• 1:08:26Lex Fridman?

  Of creating new ideas and that generalizes really, really well. So that's quite brilliantly put. I didn't really think about that. So this is not just about the competition. This is about developing minds that will come to do some incredible stuff in the future.

• 1:08:44Speaker 01

  Yeah, actually, I want to respond to a couple of things there. The first one is one which is this notion of whether or not that is possible in a non-authoritarian regime. I think it is, and that's actually why I spent some of my efforts before the COVID thing, actually trying to work towards there. The reason is because if you think about it, let's say in America, lots of people are pretty serious about training very hard for football or baseball or basketball. Basketball is very, very accessible, but lots of people are doing that. Why? Well, actually, I think that what was going on with authoritarian thing was, at least the message that was universally sent was being a good thinker and a creator of ideas is a good thing.

• 1:09:29Lex Fridman?

  Yes, exactly.

• 1:09:30Speaker 01

  There's no reason why that message can't be sent.

• 1:09:33Lex Fridman?

  That's right.

• 1:09:33Speaker 01

  Everywhere, and I think it actually should be. So that's the first thing. The second thing is what you commented about this thing about, you know, the generalizable skill and what could people do with Olympians afterwards? So that's actually my interest in the whole thing. I don't, I mean, I don't just coach students how to do problems. In fact, I'm not even the best person for that. I'm not the best at solving these problems. There are other people who are much better at making problems and teaching people how to solve problems. In fact, when I was, when the Mathematical Association of America, which is the group which is in charge of the US participation in these Olympians, when they were deciding whether or not to put me in back in 2013 as the head coach,

• 1:10:14Speaker 01

  I had a conversation with their executive director where I commented that we might do worse because my position was I don't, I mean, I actually didn't want to focus on winning. I said, if you're going to let me work with 60 very strong minds as picked through this system, because the coach works with these, gets to run a camp for these students. I said, I'm actually not going to define my success in terms of winning this contest. I said I wanted to maximize the number of the students that I read about in the New York Times in twenty years.

• 1:10:46Lex Fridman?

  Oh, yeah.

• 1:10:47Speaker 01

  And the executive director of the Mathematical Association of America was fully in support of this, because that's also how their philosophy is. So in America, the way we run this is we're actually not just training to win, even though the students are very good and they can win anyway. One reason, for example, I went and even did the COVID thing involving quite a few of them

• 1:11:09Speaker 01

  is so that hopefully some of them get ideas. Because in twenty, thirty years, I won't have the energy or the insight to solve problems. We'll have another catastrophe. And hopefully, some of these people will step up and do it.

• 1:11:20Lex Fridman?

  And ultimately have that long-term impact. I wonder if this is scalable to, because that's such a great metric for education. Not how to get an A on the test,

• 1:11:34Lex Fridman?

  but how to have, how to be on the cover of New York Times for inventing something new. And do you think that's generalizable to education beyond just this particular Olympia? Like, it's, even you saying this feels like a rare statement, almost like a radical statement as a goal for education.

• 1:11:53Speaker 01

  So actually, the way I teach my classes at Carnegie Mellon, which I will admit right away is not equivalent to the average in the world. But it's already not, it's already not just the top sixty in the country as picked by something. Let me just explain, I have exams in my class, which are ninety percent of the grade. So the exams are the whole thing, or most of the whole thing. And the way that I let students prepare for the exams is I show them all the problems I've ever given on the previous exams. And the whole, the exam that they will take is open notes. They can take all the notes they want on the previous problems. And the guarantee is that the exam problems this time will have no overlap with anything you have seen me give in the past, as well as no overlap with anything I taught in the class. So the entire exam is invention.

• 1:12:34Lex Fridman?

  Wow.

• 1:12:36Speaker 01

  But that's how I go, right? My point is, I have explained to people, when I teach you, I don't want you to have remembered a method I showed you. I want you to have learned enough about this area that if you face a new question, which I came up with the night before, by thinking about like, what could I ask that I have never asked before? Oh, that's cute. I wonder what the answer is. Ah, that's an exam problem. That's exactly what I do before the exam. And then that's what I want them to learn. And the first exam, usually people have a rough time because it's like, what kind of crazy class is this? The professor doesn't teach you anything for the exam. But then by the second or third, and by the time they finish the class, they have learned how to solve anything in the area.

• 1:13:16Lex Fridman?

  How to invent.

• 1:13:17Speaker 01

  How to invent in that area, yeah.

• 1:13:19Lex Fridman?

  Can we walk back to the Mathematical Olympiad?

• 1:13:22Speaker 01

  Yes.

• 1:13:22Lex Fridman?

  What's the scoring and format like? And also, what does it take to win?

• 1:13:28Speaker 01

  So, the way it works is that each of the six students

• 1:13:33Speaker 01

  do the problems, and there are six problems. All the problems are equally weighted, so each one's worth seven points. That means that your maximum score is six problems times seven points, which is the nice number of forty-two. And now, the way that they're scored, by the way, is there's partial credit. So the question is asking you, explain why this weird fact is true. Okay, if you explain why, you get seven points. If you make minor mistake, maybe you get six points. But if you don't succeed in explaining why, but you explain some other true fact, which is along the way of proving it, then you get partial credit. And actually, now, this is tricky because how do you score such a thing?

• 1:14:15Speaker 01

  It's not like it was, the answer was seventy-two and you wrote seventy-one, and it's close, right? The answer is seventy-two and you wrote thirty-six. Oh, but that's pretty close because you were, you know, that maybe you're just off by it. By the way, they're not numerical anyway, but I'm just giving some numerical analog to the way the scoring might work. They are all essays. And that's where, I guess, I have some role, as well as some other people who helped me in the US delegation for coaches. We actually debate with the country, which is organized.

• 1:14:44Speaker 01

  The country which is organizing the Olympiad brings about fifty people to help judge the written solutions. And you you schedule these half hour appointments where the delegation from one country sits down at a table like this. Opposite side is two or three people from the host country. And they're just looking over these exam papers saying, well, how many points is this worth based on some rubric that has been designed? And this is a negotiation process where we're not we're not trying to bargain and get the best score we can. In fact, sometimes we go to this table and we will say, we think we want less than what you gave us. This is how this is how our these are our principles. If you give us too much, we say, no, you gave us too much. We do that.

• 1:15:27Speaker 01

  However, the reason why this is an interesting process is because if you can imagine every country which is participating has its own language. And so if you're trying to grade the Mongolian scripts and they're written in Mongolian. If you don't read Mongolian, which most people don't, then the the coaches are explaining to you, this is what the student has written. It's actually quite an interesting process.

• 1:15:47Lex Fridman?

  It's almost like a like a jury.

• 1:15:50Speaker 01

  Yes.

• 1:15:51Lex Fridman?

  You have you have in the American legal system, you have a jury that where they're deliberating. But unlike a jury, there's the members of the jury are speaking different languages sometimes. That's fascinating. But I mean, it's it's hard to know what to do because it's probably really, really competitive.

• 1:16:13Lex Fridman?

  But your sense is that ultimately people like, how do you prevent manipulation here, right?

• 1:16:22Speaker 01

  Well, we just hope that it's not happening. So so we write in English. Therefore, everything that the US does, everyone can look at. So it's very hard.

• 1:16:32Lex Fridman?

  It's very hard for you to manipulate.

• 1:16:34Speaker 01

  We don't manipulate. We only hope that other people aren't. But at the same time, as you see, our philosophy was we want to use this as a way to develop general talent. And although we do this for the six people who go to the International Math Olympiad. We really want that everyone at any touch at any stage of this process gets some skills that can help to contribute more later.

• 1:16:57Lex Fridman?

  I don't know if you can say something insightful to this question, but what do you think makes a really hard math problem? On this Olympiad, maybe in the courses you teach or in general? What makes for a hard problem? You've seen, I'm sure, a lot of really difficult problems. What makes a hard problem?

• 1:17:16Speaker 01

  So I could quantify it by the number of leaps of insight, of changes of perspective that are along the way. And here's why. This is like a very theoretical computer science bit of looking at it. Okay? It's that each reframing of the problem and using of some tool, I actually call that a leap of insight. When you say, oh, wow, now I see I should kind of put these plugs into those sockets. It's like so.

• 1:17:39Lex Fridman?

  Yes.

• 1:17:40Speaker 01

  And suddenly I get to use that machine. Oh, but I'm not done yet. Now I need to do it again. Each such step is a large possible, large fan out in the search space. The number of these tells you the exponent. The base of the exponent is like how big, how many different possibilities you could try. And that's, that's actually why, like, if you have a three insight problem, that is not three times as hard as a one insight problem. Because after you've made the one insight, it's not clear that that was the right track necessarily. Well, unless you're very good at it.

• 1:18:12Lex Fridman?

  There's still a branching of possibilities.

• 1:18:17Lex Fridman?

  You're saying there's problems like on the Math Olympiad that requires more than one insight?

• 1:18:21Speaker 01

  Yes, those are the hard ones. And also, I can tell you how you can tell. So this is how I also taught myself math when I was in college. So if you are taking a, not taught myself, I was taking classes, of course. But I was trying to read the textbook. And I found out I was very bad at reading math textbooks. A math textbook has a long page of stuff that is all true, which after you read the page, you have no idea what you just read.

• 1:18:43Lex Fridman?

  Yeah. This is just a good summary of math textbooks.

• 1:18:47Speaker 01

  Okay. Yeah, because it's not clear why anything was done that way. And yes, everything is true, but how the heck did anyone think of that? So the way that I taught myself math eventually was, the way I read a math textbook is I would look at the theorem statement. I would look at the length of the proof. And then I would close the book and attempt to reprove it myself.

• 1:19:08Lex Fridman?

  Yeah.

• 1:19:09Speaker 01

  Now.

• 1:19:09Lex Fridman?

  That's brilliant.

• 1:19:10Speaker 01

  The length of the proof is telling you the number of insights. Because the length of the proof is linear in the number of insights. Each insight takes space.

• 1:19:19Lex Fridman?

  Yeah.

• 1:19:20Speaker 01

  And if I know that it's a short proof, I know that there's only one insight. So when I'm doing my own way of solving the problem, like finding the proof, I quit if I have to do too many plugins. It's equivalent to a math contest. In a math contest, I look, is it problem one, two, or three? That tells me how many insights there are. This is exactly what I did.

• 1:19:37Lex Fridman?

  That's brilliant. Linear and the number. I don't know. I think it's possible that that's true.

• 1:19:43Speaker 01

  Approximately, approximately.

• 1:19:44Lex Fridman?

  Approximately, yeah. Just, I don't know, it'd be, somebody out there is going to try to formally prove this.

• 1:19:50Speaker 01

  Oh, no, I mean, you're right. There are cases where maybe it's not quite linear, but in general.

• 1:19:54Lex Fridman?

  Well, some of it's notation, too, and some of it is style and all those kinds of things. But within a textbook.

• 1:19:59Speaker 01

  Within the same book.

• 1:20:00Lex Fridman?

  Within the same book with the same.

• 1:20:01Speaker 01

  Yes. Within the same book on the same subject. This is what I was using.

• 1:20:05Lex Fridman?

  That's hilarious.

• 1:20:06Speaker 01

  Because you know, if it's a two-page proof, you just know this is going to be insane. Right?

• 1:20:10Lex Fridman?

  That's the scary thing about insights. You look like Andrew Wiles working on Fermat's Last Theorem. Is you don't know, something seems like a good idea. And you have that idea and it feels like this is a leap. Like a totally new way to see it, but you have no idea if it's at all useful. Even if you think it's correct, you have no idea if this is like going to go down a path that's completely counterproductive or not productive at all. That's the crappy thing about invention is, like I have, I'm sure you do. I have a lot of really good ideas every single day.

• 1:20:52Lex Fridman?

  But like, and then I'll go inside my head along them, along that little trajectory. But it could be just a total waste. And it's, you know what that feels like? It just feels like patience is required, not to get excited at any one thing.

• 1:21:09Speaker 01

  So I think this is interesting because you raised Andrew Wiles. He spent seven years attacking the same thing.

• 1:21:14Lex Fridman?

  Yeah.

• 1:21:15Speaker 01

  Right? And so I think that what attracts professional researchers to this is because even though it's very painful, but you keep fighting with something, when you finally find the right insights and string them together, it feels really good. So.

• 1:21:30Lex Fridman?

  Well, there's also like short-term, it feels good to.

• 1:21:37Lex Fridman?

  Whether it's real or not, to pretend like you've solved something in the sense like you have an insight and there's a sense like this might be the insight that solves it. So at least for me, I just enjoy that rush of positivity, even though I know statistically speaking is probably going to be a dead end.

• 1:21:55Speaker 01

  I'm the same way. I'm the same way. In fact, that's how I know whether I might want to keep thinking about this general problem. It's like if I still see that I'm getting some insights, I'm not at a dead end yet. But that's also where I learned something from my PhD advisor. Actually, he was a real big inspiration in my life. His name is Benny Sudakov. In fact, he grew up in the former Soviet Union.

• 1:22:14Lex Fridman?

  Excellent.

• 1:22:15Speaker 01

  He was from Georgia. But he's an incredible person. But one thing I learned was choose the problems to work on that might matter if you succeed. Because that's why, for example, we dug into COVID. It was just, well, suppose we succeed in finding some interesting insight here. Well, it actually matters. That is worth a laugh.

• 1:22:38Lex Fridman?

  Yeah, and I think COVID, the way you're approaching COVID has two interesting possibilities. One, it might help with COVID or another pandemic. But two, I mean, just this whole network theory space, you might unlock some deep understanding about the interaction with human beings that might have nothing to do with the pandemic. There's a there's a space of possible impacts that may be direct or indirect. And the same thing is with Andrew Wiles' proof. I don't understand, but apparently the pieces of it are really impactful for for mathematics, even if the main

• 1:23:20Speaker 01

  Yes.

• 1:23:20Lex Fridman?

  theorem is not. So along the way, the the insights you have might be really powerful for Unexpected reasons.

• 1:23:30Speaker 01

  So I like what you said. This is something that I learned from another friend of mine, who's also, he's a very famous researcher. All these people are more famous than I am. His name is Jacob Fox. He's Jacob Fox at Stanford, also a very big inspiration for me. We were both grad students together at the same time.

• 1:23:43Lex Fridman?

  Well, most importantly, you're good at selecting good friends.

• 1:23:45Speaker 01

  Ah, yeah, well, that's, that's, that's the key. You've got to find good people to learn things from. But his thing was, he often said, you know, if you solve a math problem and have this math proof, math problem for him is like a proof, right? So suppose you came up with this proof. He always asks, what have we learned from this that we could potentially use for something else? It's not just, did you solve the problem that was supposed to be famous? It was, and is there something new in the course of solving this that you had to invent? That we could now use as a tool elsewhere?

• 1:24:14Lex Fridman?

  Yeah, there's this funny effect where just looking at different fields, where people discover parallels, they'll, they'll prove something, it'll be a totally new result. And then somebody later realizes this was already done 30 years ago in another discipline in another way. And it's really interesting. Now, we did this offline in another illustration he showed to me. It's interesting to see the different perspectives on a problem. It kind of points like, there's just like very few novel ideas. That everything else, that most of us are just looking at different perspective on the same idea.

• 1:24:54Lex Fridman?

  And it makes you wonder, this, this, uh, this old silly question that I have to ask you is, do you think mathematics is discovered or invented? Do you think we're creating new idea? We're building a set of knowledge that's, that's distinct from reality? Or are we actually like, is math almost like a shovel? Where we're digging to like this core set of truths that are, that were always there all along?

• 1:25:27Speaker 01

  So I personally feel like it's discovered. But that's also because, I guess, the way that I like to choose what questions to work on, are questions that maybe we'll get to learn something about. Why is this hard? I mean, I'm often attracted to questions that look simple but are hard. Right? And what could you possibly learn from that? Sort of like probably the attraction of Fermat's Last Theorem. As you mentioned, simple statement, why is it so hard? So I'm more on the discovered side. And I also feel like if we ever ran into an intelligent other species in the universe.

• 1:26:02Speaker 01

  Probably if we compared notes, there might be some similarities between both of us realizing that pi is important. Because you might say, why? Why humans, do humans like circles more than others? I think stars also like circles. I think planets like circles. They're not perfect circles, but nevertheless, the concept of a circle is just point and constant distance. Doesn't get any simpler than that.

• 1:26:24Lex Fridman?

  It's possible that like an alien species will have, depending on different cognitive capabilities and different perception systems, will be able to see

• 1:26:35Lex Fridman?

  things that are much different than circles. And so

• 1:26:40Lex Fridman?

  if it's discovered, it would still be pointing at a lot of same geometrical concepts, mathematical concepts, but it's interesting to think of how many things we would have to still align, not just based on notation, but based on understanding, like just like some basic mathematical concepts, like how much work is there going to be in trying to find a common language? I mean, this is, I think Stephen Wolfram and his son helped with the movie Arrival, like the developing an alien language, like how would aliens communicate with humans?

• 1:27:18Lex Fridman?

  It's fascinating because like math seems to be the most promising thing, but even like math, like how do you visualize mathematical ideas? It feels like there has to be an interactive component, just like we have a conversation. There has to be, this is something we don't, I think, think about often, which is like with somebody who doesn't know anything about math, doesn't know anything about English or any other natural language, how would we describe? We talked offline about visual proofs. How would we, through visual proofs, have a conversation? Where we say something, here's the concept, the way we see it.

• 1:28:00Lex Fridman?

  Does that make sense to you? And like, can you mess with that concept to make it sense for you? And then go back and forth in this kind of way. So purely through mathematics, I'm sure it's possible to have those kind of experiments with like tribes on Earth that don't, there's no common language. Through math, like draw a circle and see what they do with it. Right? Do some of these visual proofs, like the summation of the odds and the adds up to the squares. Yes, I wonder how difficult that is. Before one or the other species murders.

• 1:28:33Speaker 01

  That's a good question.

• 1:28:35Lex Fridman?

  I hope that the curiosity for knowledge will overpower the greedy. This is back to our game theory thing, that the curiosity of like discovering math together will overpower the desire for resources. And ultimately, like, you know, willing to commit violence in order to gain those resources. I think as we progress, become more and more intelligent as a species, I'm hoping we would value more and more the knowledge. Because we'll come up with clever ways to gain more resources, so we won't be so resource starved. I don't know. That's a hopeful message for when we finally meet aliens.

• 1:29:08Speaker 01

  Yeah, yeah.

• 1:29:09Lex Fridman?

  The cool thing about the Math Olympiad, I don't know if you know work from Francois Chollet from Google. He come up, he came up with this kind of IQ test slash, it kind of has similar aspects to it that also the Math Olympiad does. For, for AI. So he came up with these tests where they're very simple for humans, but very difficult for AI. To illustrate exactly why we're just not good at seeing a totally new problem. We, sorry, AI systems are not good at.

• 1:29:52Lex Fridman?

  Looking at a new problem that requires you to detect that there's a symmetry of some kind, or there's a pattern that hasn't seen before. The pattern is like obvious to us humans, but it's not so obvious to find that kind of, you're inventing a pattern that's there in order to then find a solution.

• 1:30:19Lex Fridman?

  I don't know if you can comment on, but from an AI perspective and from a math problem perspective, what do you think is intelligence? What do you think is the thing that allows us to solve that problem? And how hard is it to build a machine to do that? Asking for a friend?

• 1:30:38Speaker 01

  Yeah, well, so I guess, you see, because if I just think of the raw search space, it's huge. That's why you can't do it. And if I think about what makes somebody good at doing these things, they have this heuristic sense, it's almost like a good chess player, of saying, let's not keep analyzing down this way because there's some heuristic reason why that's a bad way to go.

• 1:30:56Lex Fridman?

  Yes.

• 1:30:56Speaker 01

  Where do they get that heuristic from? Now, that's a good question. I don't know. Because that, if you asked them to explain to you, they could probably say something in words that sounds like it makes sense. But I'm guessing that's only a part of what's really going on in their brain of evaluating that position. You know what I mean? If you ask Gary Kasparov, what is good or why is this position good? He will say something.

• 1:31:17Lex Fridman?

  Yeah.

• 1:31:17Speaker 01

  But probably not approximating everything that's going on inside. So there's basically a function being computed.

• 1:31:23Lex Fridman?

  Yeah.

• 1:31:24Speaker 01

  But it's hard to articulate what that function is. Now, the question is, could a computer get as good at computing these kinds of heuristic functions? Maybe. I'm not enough of an expert to understand, but one bit of me has always been a little bit curious of whether or not the human brain has a particular tendency due to its wiring to come up with certain kinds of things, which is just natural due to the way that the topology of the neurons and whatever is there, for which if you tried to just build from scratch a computer to do it, would it naturally have different tendencies? I don't know. This is just me being completely ignorant and just saying a few ideas.

• 1:32:01Lex Fridman?

  Well, this is a good thing that mathematics shows is we don't have to be, so math and physics, or mathematical physics operates in a world that's different than our. Descendant of a brains operating. So it allows us to have multiple, many, many dimensions. It allows us to work on weird surfaces. With like topology as a discipline is just weird to me. It's really complicated, but it allows us to work in that space. Differential geometry, all those kinds of things where it's totally outside of our natural day to day, four dimensional experience. Three D dimensional with time experience.

• 1:32:42Lex Fridman?

  So math gives us, gives me hope that we can, that we can see, we can discover the processes of intelligence outside the limited nature of our own like human experiences. But you said that you're not an expert. It's kind of funny. I, I find that we know so little about intelligence that I think, I honestly think like almost children are more expert at creating artificial intelligence systems than, than adults. I feel like we know so little, we really need to think outside the box.

• 1:33:25Lex Fridman?

  And those little, I found that people should check out Francois Chollet's little exams. But even just solving math problems. I don't know if you've ever done this for yourself, but when you solve a math problem, you kind of then trace back and try to figure out, where did that idea come from? Like what, how do, like what was I visualizing in my head? How did I start visualizing it that way? How, why did I start rotating that cube in my head in that way? Like what is that? If I were to try to build a program that does that, where did that come from?

• 1:34:04Speaker 01

  So this is interesting. So I tried to do this to teach middle school students how to learn how to create, create and think and invent. And the way I do it is there are these math competition problems. And I'm working in collaboration with the people who run those. And I will turn on my YouTube live and for the first time, look at those questions and live solve them. The reason I do this is to let the middle school students and the high school students and the adults who everyone wants to watch just see what exactly goes on through someone's head as they go and attempt to invent what they need to do to solve the question. So I've actually thought about that. I think that, first of all, as a teacher, I think about that because whenever I want

• 1:34:46Speaker 01

  to explain to a student how to do something, I want to explain how it made sense, why it's intuitive to do the following things, and why the wrong things are wrong, not just by this one short, fast way, well, by this is the right way, if that makes sense. So my point is, I'm actually always thinking about that, like, how would you think about these things? And then I eventually decided the easiest way to expose this would just be to go live on YouTube and just say, I've never seen any of these questions before, here we go.

• 1:35:15Lex Fridman?

  Don't you get, that's anxiety inducing for me. Don't you get trapped in a kind of like little dead ends of confusion, even on middle school problems?

• 1:35:27Speaker 01

  Yes, that's what the comments are for. The live comments come in and students say, try this.

• 1:35:32Lex Fridman?

  Oh, wow.

• 1:35:33Speaker 01

  It's actually pretty good. And I'll never get stuck. I mean, I'm willing to go on camera and say, I guess what, potion though, can't do this one, that's fine. But then what ends up happening is you will then see how maybe somebody's saying something and I look at the chat and I say, aha, that actually looks useful. Now, that also shows how not all ideas, not all suggestions are the same power, if that makes sense. Because if I actually do get stuck, I'll go fishing through the chat. Anyone got any ideas?

• 1:35:57Lex Fridman?

  I don't know if you can speak to this, but is there a moment for the middle school students, maybe high school as well, where there's like a turning point for them, where they maybe fall in love with mathematics, or they get it? Is there, is there something to be said about like discovering that moment and trying to grab them, to get them to understand that mathematics is, no matter what they want to do in life, could be part of their life?

• 1:36:30Speaker 01

  Yes, I actually do think that the middle school is exactly the right time, because that's the place where your mathematical understanding gets just sophisticated enough that you can start doing interesting things. Because if you're early on and counting, I'm honestly not very good at teaching you new insights. My wife is pretty good at that, but somehow, once you get to this part where you know what a fraction is, and when you know how to add and how to multiply and what the area of a triangle is, at that point, to me, the whole world opens up and you can start observing there are really nifty coincidences, the things that made the Greek mathematicians and the ancient mathematicians excited. Actually, back then, it was exciting to discover the Pythagorean theorem. It wasn't just homework.

• 1:37:15Lex Fridman?

  So is there, which discipline do you think has the most exciting coincidences? So is it geometry? Is it algebra?

• 1:37:27Lex Fridman?

  Is it calculus?

• 1:37:28Speaker 01

  Well, you see, you're asking me, and I'm the guy who gets the most excited when the combinatorics shows up in the geometry.

• 1:37:35Lex Fridman?

  Is it? Okay, so it's the combinatorics in the geometry. So first of all, the nice thing about geometry, this is the same nice thing about computer vision, is it's visual. So geometry, you can draw circles and triangles and stuff, so it naturally presents itself to the visual proof, right? But also, the nice thing about geometry, I think for me,

• 1:38:01Lex Fridman?

  is the earliest class, the earliest discipline, where there's, that's most amenable to the exploration, the invention, the proofs. The idea of proofs, I think, is most easily shown in geometry, because it's so visual, I guess. So that, to me, is like, if I were to think about when I first fell in love with math, it would be geometry. And sadly enough, that's not used, geometry only has a little, appears briefly in the journey of a student, and it kind of disappears. And not until much later, which, you know, there may be like differential geometry, I don't know where else it shows up.

• 1:38:45Lex Fridman?

  For me, in computer science, like, you could start to think about like computational geometry, or even graph theory is a kind of geometry, you could start to think about it visually, although it's pretty tricky. But yeah, it was always, that was the most beautiful one. Everything else, I guess calculus can be kind of visual too, that can be pretty beautiful. But is there? Something you try to look for in a student to see like, how can I inspire them at this moment? Or is this like individual student to student? Is there something you could say there?

• 1:39:21Speaker 01

  So first of all, I really think that every student can pick up all of this skill. I really do think so. I don't think it's something only for a few. And so if I'm looking for a student, actually oftentimes when I'm, if I'm looking at a particular student, the question is, how can we help you feel like you have the power to invent also? Because I think a lot of people are used to thinking about math as something where the teacher will show you what to do, and then you will do it.

• 1:39:46Lex Fridman?

  Yes.

• 1:39:47Speaker 01

  So I think that the key is to show that they have some, let them see that they have some power to invent. And at that point, it's often starting by trying to give a question that they don't know how to do. You want to find these questions that they don't know how to do, that they can think about, and then they can solve. And then suddenly they say, my gosh, I've had a situation, I've had an experience where I didn't know what to do, and after a while, I did.

• 1:40:10Lex Fridman?

  Is there advice you can give on how to learn math? For people, whether it's middle school, whether it's somebody, as an adult, kind of gave up on math maybe early on?

• 1:40:26Speaker 01

  I actually think that these math competition problems, middle school and high school are really good. They are actually very hard. So if you, if you haven't had this kind of experience before, and you grab a middle school math competition problem from the state level, which is used to decide who represents the state in the country, in the United States, for example, those are pretty tricky. And even if you're a professional, maybe not doing mathematical things, and you're not a middle school student, you'll struggle. So I find that these things really do teach you things by trying to work on these questions.

• 1:41:00Lex Fridman?

  Is there a Google-able term that you could use for the organization for the state competitions?

• 1:41:06Speaker 01

  Ah, yeah. So there are a number of different ones that are quite popular. One of them is called Math Counts, M A T H C O U N T S. And that's a big tournament, which actually has a state level. There's also a mathleague.org, mathleague, L E A G U E dot org. Also has this kind of tiered tournament structure. There's also the American Math Competitions, AMC eight. AMC also has AMC ten, that's for 10th grade and below, and AMC twelve. These are all run by the Mathematical Association of America. And these are always defined old questions.

• 1:41:40Lex Fridman?

  What about the daily challenges that you run? What are those about?

• 1:41:43Speaker 01

  We do that too. But I mean, the difference was ours isn't, that one's not free. So I should actually probably be careful. The things that I've just mentioned are also not free. Not all of those things I mentioned just now are free either.

• 1:41:54Lex Fridman?

  People can figure out what it's free or not. But this is really nice to know what's out there. But can you speak a little bit to the daily challenges?

• 1:42:01Speaker 01

  Sure, sure. So that's actually what we did when, I guess I was thinking about, how would I try to develop that skill in people? If we had the power to architect the entire system ourselves? So that's called the Daily Challenge with Posh and Low. It's not free because that's actually how I pay for everything else I do. So that was the idea. But the concept was, aha, now let's invent from scratch. So if we're going to go from scratch and we're going to use technology, what if we made every single lesson, something where first I say, hey, here's an interesting question. Recorded, of course, not live. But it's like, I say, hey, here's an interesting question. Why don't we think about this? But I know you don't know how to do it. So now you think, and a minute later, a hint pops on the screen. But you still think, and a minute later, a big hint pops on the screen. You still think.

• 1:42:46Speaker 01

  And then finally, after the three minutes, hopefully you got some ideas, you try to answer. And then suddenly, there's like this pretty extended explanation of, oh, yeah. So here's like multiple different ways that you can do the question. And by accident, you also just learned this other concept. That's what we did.

• 1:43:02Lex Fridman?

  So is this targeted towards middle school students, high school students?

• 1:43:06Speaker 01

  It's targeted towards middle school students with competitions. But there's a lot of high school students who didn't do competitions in middle school, where they would also learn how to think. If you can see, the whole concept was, can we teach people how to think? How would you do that? You need to give people the chance to, on their own, invent, without that kid in the front row answering every question in two seconds.

• 1:43:27Lex Fridman?

  And people can find it, I think, with daily dot.

• 1:43:31Speaker 01

  It's daily dot potionlow dot com. But if you go to find my website, you'll be able to find it.

• 1:43:36Lex Fridman?

  Beautiful. Can we zoom out a little bit? And so day to day, week to week, month to month, year to year. What does the lifelong educational process look like, do you think? For for yourself, but for for me, what would you recommend in the world of mathematics? Or sort of as opposed to studying for a test, but just like lifelong

• 1:44:04Lex Fridman?

  expanding of knowledge in that skill for invention?

• 1:44:09Speaker 01

  I think I often articulate this as can you always try to do more than you could do in the past?

• 1:44:18Lex Fridman?

  Yeah.

• 1:44:18Speaker 01

  But that that comes in many ways. And I will say it's great if one wants to build that with mathematics. But it's also great to use that philosophy with all other things. In fact, if I if I just think of myself, I just think, what do I know now that I didn't know a year ago? Or a month ago or a week ago? And not just know, but like, what do I have the capability of doing?

• 1:44:38Lex Fridman?

  Yes.

• 1:44:39Speaker 01

  And if you just have that attitude, it brings more.

• 1:44:42Lex Fridman?

  See, the thing is, there's also a habit, like it is a skill. Like, I've been using Anki. It's an app for helps you memorize things. And I've actually this few months ago, started doing this daily of setting aside time to think about an idea that's outside of my work. Like, let's say, let's pick, it's all over the place, by the way. But let's say, politics, like gun control. Is it good to have a lot of guns or not in society? And just I've set aside time every day. I do at least 10 minutes, but I try to do 30 where I think about a problem.

• 1:45:26Lex Fridman?

  And I kind of outlined for myself from scratch, from not looking anything up, just thinking about it, using common sense. And I think the practice of that is really important. It's the daily routine of it. It's the discipline of it. It's not just that I figured something out from that thinking about gun control. It's more that. That muscle is built too, it's that thinking muscle. So I'm kind of interested in, you know, math has, because especially because I've gotten specialized into machine learning and because I love programming so much. I've lost touch with math a little bit to where I feel quite sad about it and I want to fix

• 1:46:10Lex Fridman?

  that. Even just not math, like pure knowledge math, but math like these middle school problems. The challenges, right? Is that something you see a person be able to do every single day? Kind of just practice every single day for years?

• 1:46:26Speaker 01

  So I can give an answer to that that gives a practical way you could do it, assuming you have kids. So, no, you can do it.

• 1:46:34Lex Fridman?

  Step one, get kids.

• 1:46:35Speaker 01

  No, no, I was just saying this because I'm just thinking out loud right now, what could I do? What could I do to suggest? Because what I have noticed is that, for example, if you do have kids who are in elementary school or middle school, if you yourself go and look at those middle school math problems to think about interesting ways that you can teach your elementary school or middle school kid, it works. That's what my wife did. She never did any of those contests before, but now she knows quite a lot about them. And I didn't teach her anything. I don't, I don't do that. She just was messing around with them and taught herself all of that stuff. And that had the automatic daily. I'm always thinking, how do you make it practical, right?

• 1:47:10Lex Fridman?

  Yes.

• 1:47:11Speaker 01

  And the way to make it practical is if the timer on the automatically daily is that you are going to automatically daily do something with your own kid.

• 1:47:18Lex Fridman?

  Yes.

• 1:47:19Speaker 01

  Now it feeds back. Okay.

• 1:47:21Lex Fridman?

  And that includes the whole lesson that if you want to learn something, you should teach it.

• 1:47:24Speaker 01

  Oh, I strongly believe that.

• 1:47:25Lex Fridman?

  Yes.

• 1:47:26Speaker 01

  I strongly believe that.

• 1:47:28Lex Fridman?

  And so I currently don't have kids, so that's, maybe I should just get kids to help me with the math thing. But outside of that, I do want to integrate math into daily practice. So I'll definitely take out, I'll definitely check out the daily challenges and see, because, what is it, Grant Sanderson, we talked about offline, the three blue, one brown. He speaks to this as well, that his videos aren't necessarily, they don't speak to the thing that I'm referring to, which is the daily practice. They're more almost tools of inspiration. They kind of show you the beauty of a particular of problem.

• 1:48:09Lex Fridman?

  In mathematics, but they're not a daily ritual. And I'm in I'm in search of that daily ritual mathematics. It's not it's not trivial to find.

• 1:48:22Lex Fridman?

  But I hope I hope to find that because I think math gives you a perspective on the world that enriches everything else.

• 1:48:31Speaker 01

  So I like what you said about the daily also because that's also one reason why I put my Carnegie Mellon class online. It's not every day. It was every it's every other day. Semester is almost over. But the idea was, I guess my philosophy was if I'm already doing the the class, let's just like put it there, right? But I do know that there are people who have been following it, who are not in my class at all, who have just been following it because, yes, it's combinatorics. And the value of that is you could you don't really need to know calculus to follow it, if that makes sense. So it's actually something that people could follow. So again, and that that one's free. So that one's just there on YouTube.

• 1:49:06Lex Fridman?

  Well, speaking of combinatorics, what is it? What do you find interesting? What do you find beautiful about combinatorics?

• 1:49:14Speaker 01

  So combinatorics to me is is the study of things where they might be more finite and more discrete. What I mean is like if I look at a network, actually, a lot of times the combinatorics will boil down to something and the combinatorics I think about might be something related to graphs or networks. And they're very discrete because if you have a node, it's not that you have point seven of a node and point three of a node over there. It's like you got one node and then you jump one step to go to the next node. So that that notion is different from, say, calculus, which is very continuous, where you go and say, I have this speed which is changing over time. And now what's the distance I've traveled? That's the notion of an integral, where you have to think of subdividing time into

• 1:49:58Speaker 01

  very, very small pieces. So the the kinds of things that you do when you reason about these finite, discrete structures often might be iterative, algorithmic, inductive. These are ideas where I go from one step to the next step and so on and make progress. I guess I actually personally like all kinds of math. My area of research. Just ended up in here because I met a really interesting PhD advisor. Potential that's that's honestly the reason I went into that direction. I met a really interesting guy. He seemed like he did good stuff, interesting stuff, and he looked like he cared about students. And I said, let me just go and learn whatever you do. Even though my prior practice and preparation before my PhD was not combinatorics, but analysis, the continuous stuff.

• 1:50:43Lex Fridman?

  So the annoying thing about combinatorics and discrete stuff is it's often really difficult to solve from a sort of running time complexity perspective. Is there, could you speak to the idea of complexity analysis of problems? Do you find it useful to find it interesting? Do you do you find that lens of studying the difficulty of how difficult the computer science problem is a useful lens onto the world?

• 1:51:22Speaker 01

  Oh, very much so, because if you want to make something practical, which has large numbers of people using it, the computational complexity to me is almost question one. And that's again, that's at the origin of when we started doing this stuff with disease control. From the very beginning, the deep questions that were running through my mind were, would we be able to support a large population with only one server? And if the answer is no, we can't start because I don't have enough money.

• 1:51:55Lex Fridman?

  Yeah, and there the question is very much, you know, linear time versus anything. Anything slower than linear time.

• 1:52:04Speaker 01

  Yep.

• 1:52:05Lex Fridman?

  It's a very specific thing. You have a bunch of really interesting papers. If I could ask, maybe we could pull out some cool insights at the high level. Can you describe the data structure of a voting tree? And what are some interesting results on it? You have a paper that I noticed on it.

• 1:52:21Speaker 01

  Yeah, so this is an example of, I guess, how in math, we might say, here's an interesting kind of a question that we just can't seem to understand enough about. And maybe there's something else going on here. And the way to describe this is you could imagine trying to hold elections, where if you have only two candidates, that's kind of easy. You just run them against each other and see who gets more votes. But as you know, once you have more candidates, it's very difficult to decide who wins the election. And there's an entire voting theory around this. So a theoretical question became, what if you made like a system of runoffs? Like a system of head-to-head contests,

• 1:53:04Speaker 01

  which is structured like a tree, almost looking like a circuit? I'm using that way of thinking because it's sort of like in electrical engineering or computer science. You might imagine having a bunch of leads that carry signal, which are going through AND gates and OR gates and whatnot. And you manage to compute beautiful things. This is just from a purely abstract point of view. What if the inputs are candidates? And for every two candidates, it is known which of the candidates is more popular than the other. Now, can you build some kind of a circuit board, which says, first, candidate number four will play against five and see who wins and so on? Okay, so now what would be a nice outcome, right? This is a general question of could I make a big circuit board to feed an election into?

• 1:53:46Speaker 01

  Like, maybe one nice outcome would be whoever wins at least is preferred over a lot of people.

• 1:53:52Lex Fridman?

  Yes.

• 1:53:52Speaker 01

  So, for example, if you ran in 1,024 candidates, ideally, we would like a guarantee that says that the winner beats a lot of people. Actually, in any system where there are 1,024 candidates, there's always a candidate who beats at least 512 of the others. This is a mathematical fact, that there's actually always a person who beats at least half of the other people.

• 1:54:17Lex Fridman?

  I'm trying to make sense of that mathematical fact. Is this supposed to be obvious?

• 1:54:22Speaker 01

  No, but I can explain it. No, I can. The way it works is that, think of it this way. Every time, I think, imagine I have all these candidates and everyone is competing. Everyone is like compared with everyone else at some point. Well, think of it this way. Whenever there's a comparison, somebody gets a point. That's the one who is better than the other one. My claim is there's somebody whose score is at least half of how many other people there are.

• 1:54:50Lex Fridman?

  Yeah, I'm just trying to, like, my intuition is very close to that being true, but it's beautiful. I didn't at first, that's not an obvious fact.

• 1:54:59Speaker 01

  No, it's not.

• 1:55:00Lex Fridman?

  And it's, it feels like a beautiful fact.

• 1:55:03Speaker 01

  Well, let me explain it this way. Imagine that for every match, you didn't give one point, but you gave two points. You gave one point to each person. Now, that's not what we're really doing. We really want to give one point to the winner of the match.

• 1:55:18Speaker 01

  But instead, we'll just give two. If you gave two points to everyone on every matchup, actually, everyone has the same number of points. And the number of points they get is how many other people there are.

• 1:55:30Speaker 01

  Does that sort of make sense? I'm just like saying.

• 1:55:31Lex Fridman?

  No, no, everything, everything you're saying makes perfect sense.

• 1:55:33Speaker 01

  Okay, so the point is if for every comparison between two people, which I'm doing for every two people, I gave one point to each person, your score, everyone's score is the same. It's how many other people there are.

• 1:55:45Lex Fridman?

  Yes.

• 1:55:46Speaker 01

  Now, we only make one change. For each matchup, you give one point only to the winner. So we're awarding half the points. So now the deal is, if in the original situation, everyone's score was equal, which is how many other people there are, now there's only half the number of points to go around. So what ends up happening is that there's always going to be, like, the average number of points per person is going to be half of how many other people there are. And somebody is going to be above average.

• 1:56:15Lex Fridman?

  Somebody's going to be above that.

• 1:56:17Speaker 01

  At least average. Yeah, yeah, this is, this is this notion of expected value, that if I have a random variable which has an expected value, there's going to be some possibility in the probability space where you're at least as big as the expected value.

• 1:56:29Lex Fridman?

  Yeah, when you describe it like that, it's obvious. But when you were first saying in this little circuit that there's going to be one candidate better than, than half. That's not obvious.

• 1:56:40Speaker 01

  Yeah.

• 1:56:41Lex Fridman?

  It's funny. Math. This is nice. Okay, so you have this, but ultimately, you want, you're trying to with a voting tree. I don't know if you're trying this, but to have a circuit that's, that's like compressed, that's small.

• 1:56:55Speaker 01

  Well, you'd like it to be small.

• 1:56:56Lex Fridman?

  That achieves the, that achieves the, the same kind of, um. I mean, the smaller it is, the, if we look at practically speaking, the, the lower the cost of running the election, of running through, of computing the circuit.

• 1:57:11Speaker 01

  That is true, but actually, at this point, the, the, the reason the question was interesting.

• 1:57:15Lex Fridman?

  Yes.

• 1:57:16Speaker 01

  Is because the, there was no good guarantee that the winner of that circuit would have, like have beaten a lot of people. Let me give an example.

• 1:57:26Lex Fridman?

  Got it.

• 1:57:27Speaker 01

  The best known circuit, when we started thinking about this, was the circuit called, candidate one plays against candidate two, candidate three plays against four, and then the winners play against each other. And then, by the way, five plays against six, seven against eight, the winners play against each other. You understand, it's like a giant binary tree.

• 1:57:43Lex Fridman?

  Yeah, it's a binary, like a balanced binary tree?

• 1:57:46Speaker 01

  Yeah. It's a balanced binary tree, one, two, three, four, up to 2024, everyone going up to find the winner.

• 1:57:51Lex Fridman?

  Beautiful.

• 1:57:51Speaker 01

  Well, you know what? There's a system in the world where it could just be that there's a candidate called number one that just beats like 10 other people. Just the 10 that they need to beat on their way up. And they lose to everyone else. But somehow, they would get all the way up.

• 1:58:11Lex Fridman?

  Yes.

• 1:58:12Speaker 01

  My point is, it is possible to outsmart that circuit in one weird way of the world. Which makes that circuit a bad one, because you want to say, I will use this circuit for all elections. And you might have a system of inputs that go in there, where the winner only beat 10 other people, which is the people they had to beat on their way up.

• 1:58:34Lex Fridman?

  So you want to have a circuit where there's as many, like the final result is as strong as possible?

• 1:58:40Speaker 01

  Yes.

• 1:58:41Lex Fridman?

  And so is there, what, what ideas do you have for that?

• 1:58:44Speaker 01

  So, um, we actually only managed to improve it to square root of n. So if n is the number of vertices, n over 2 would be the ideal. We got it to n, we got it to square root of n.

• 1:58:55Lex Fridman?

  Versus log of n?

• 1:58:57Speaker 01

  Yeah, exactly.

• 1:58:58Lex Fridman?

  Yeah. Which is

• 1:59:00Speaker 01

  Well, that is halfway.

• 1:59:01Lex Fridman?

  It could be a lot.

• 1:59:03Speaker 01

  Yeah.

• 1:59:04Lex Fridman?

  Could be a big improvement. So that's a, okay, cool. Is there something you can say with words about what kind of circuit, what that looks like?

• 1:59:12Speaker 01

  I can give an idea of one of the tools inside.

• 1:59:14Lex Fridman?

  Yeah.

• 1:59:15Speaker 01

  But the actual execution ends up being more complicated. But one of the widgets inside this is building a system where you have like a candidate who plays like one part of the whole huge, huge tree is that that same candidate, let's call him seven. Seven plays against somebody. Let's make up some numbers. Let's call the others like letters. So seven plays against A.

• 1:59:38Speaker 01

  Seven's also going to play against B separately. And the winners of each of those will play each other. By the way, seven's also going to play C, seven's going to play D, and the winners are going to play each other, and the winners are going to play each other. We call this seven against all. Well, seven against like everyone from a bunch of.

• 1:59:55Lex Fridman?

  Got it. So there's some nice overlap between the matchups.

• 1:59:58Speaker 01

  Yes.

• 1:59:58Lex Fridman?

  That somehow has a nice feature to it.

• 2:00:00Speaker 01

  Yes, and I can tell you the nice feature because if at the base of this giant tree, at the base of this giant circuit, like this is a widget. We build the things out of widgets, so I'm just describing one widget. But in the base of this widget, you have lots of things which are seven against someone, seven against someone, seven against someone. In fact, every matchup at the bottom is seven against someone. What that means is if seven actually beat everyone they were matched up against, well, seven would rise to the top. So one possibility is if you see a seven emerge from the top, you know that seven actually beat everyone they were against. On the other hand, if anyone else is on top, let's call it F. If F is on top, how did F get there? Well, F beat seven on the way at the beginning.

• 2:00:43Speaker 01

  So the point is, the outcome of this circuit has a certain property. If you see a seven, you know that the seven actually beat a bazillion people. If you see anyone else, at least you know they beat seven.

• 2:00:54Lex Fridman?

  Yeah, then you can prove that it has a nice property. That's really interesting. Is there something you can say, perhaps going completely outside of what we're talking about, is how we may

• 2:01:08Lex Fridman?

  have mathematical ideas of improving the electoral process?

• 2:01:13Speaker 01

  That one, no. No, I can't give you that one.

• 2:01:15Lex Fridman?

  Okay. I mean, is there, like, do you ever see as? As there be, do you see as there being a lot of opportunities for improving how we vote? Like from your, I don't know if you saw parallels, but you know, it seems like if this actually kind of maps to your sort of COVID work, which is there's a network effect, right? It seems like we should be able to apply similar kind of effects of how we decide other things in our lives. And one of the big decisions we'll make is who represents us in government. Do you ever think about like mathematically about those kinds of systems?

• 2:01:55Speaker 01

  I think a little bit about those because where I went to college, the way we voted for student government was based on this. Is it called ranked choice where you eliminate, you eliminate the bottom and there are runoff elections? So, so that was the first time I ever saw that and I thought that made sense.

• 2:02:11Lex Fridman?

  Yeah.

• 2:02:11Speaker 01

  The only problem is it doesn't seem so easy to get something that makes sense adopted as a new voting system.

• 2:02:17Lex Fridman?

  That's a whole nother, that's not a math solution. That's a, well, it's math in the sense it's game theory. So you have to come up with incentive, it's mechanism design. You have to figure out how to trick us despite our basic human nature to adopt solutions that are better.

• 2:02:34Speaker 01

  Yeah.

• 2:02:35Lex Fridman?

  That's a whole nother conversation, I think. Can you, just because it sounded really cool, talk a little bit about stochastic coalescence? And you have a paper on showing that, you describe what it is, but I guess it's super linear, super logarithmic time and you came up with some kind of trick that make it faster. Just, can you just talk about it a little bit?

• 2:02:56Speaker 01

  Yeah, so this was something which came up when I was at Microsoft Research for a summer. And I'm putting that context because that shows that it has some practical motivation at some point.

• 2:03:08Speaker 01

  Actually, I think it's still.

• 2:03:09Lex Fridman?

  It doesn't need to, it doesn't need to, it can be beautiful and it's all right.

• 2:03:12Speaker 01

  Yeah, so the easiest way to describe this is suppose you got like a big crowd of people and everybody knows how many hours of sleep they got last night. And you want to know how many total hours of sleep were gotten by this big crowd, big crowd of people. At the beginning, you might say, that sounds like a linear time algorithm of saying, hey, you, how many hours you got? How many you got, how many you got, add, add, add. Yes. But there's a way to do this if you remember that there are people and they presumably know how to add. You could make a distributed algorithm to make this happen. For example, while we're thinking of these trees, imagine you had a thousand twenty four people. If you could just say, hey, person number one and person number two, you will add your hours of sleep. Person number two will go away and person number one is going to remember the sum.

• 2:03:55Speaker 01

  Person three and four add up and person three takes charge of remembering it. Person four goes away. Now this like person one knows the sum of these two. Person three knows the sum of those two. They talk. You see what I mean? It's like you're going up this tree, same tree that we talked about earlier. Built up a tree from the bottom up. Yeah, build a tree from the bottom up. And the beautiful thing is since everyone's doing stuff in parallel, the amount of time it takes to get the total sum is actually just the number of layers in the tree, which is ten. So now that's logarithmic time to add up the number of hours that people slept today. Sounds fantastic. Yes. There's only one problem. How do you decide who's person number one and person number two? Yes.

• 2:04:37Speaker 01

  So if, for example, we just went out into downtown and said, hey, get these thousand people go. Well, if you're going to go and say, and by the way, you're one and you're two and you're three, that's linear time. Yes. You, that's cheating. So now the question is how to do this in a distributed way. And there were some people who proposed a very elegant algorithm. And they wanted to analyze it. That's, so I came in onto the analyze side. But the elegant algorithm was like this. It was like, well, we don't actually know what this big nut, big, big tree is. There isn't any big tree. So what's going to happen is first, uh, everyone is going to decide right now. Oh, what, one important thing. Everyone is going to, at the very beginning of the whole game,

• 2:05:17Speaker 01

  they will have delegated responsibility to themselves as the one who knows the sum so far. So, so, so the point is there's, there's going to be, people are all going to have like a pointer which says, you are the one who knows my, you, you've taken care of my ticket, my number.

• 2:05:34Lex Fridman?

  Yeah, they select a representative for this particular piece of knowledge.

• 2:05:38Speaker 01

  Right. And at the very beginning, you're your own representative. The thing has to start simple, right? So at the beginning, you're your own representative.

• 2:05:43Lex Fridman?

  You're pointing to yourself. Got it.

• 2:05:45Speaker 01

  Yep. And now, the way this works is that at every time step, someone blares a ding dong on the town clock or whatever.

• 2:05:52Lex Fridman?

  Yes.

• 2:05:53Speaker 01

  And each person flips a coin themselves to decide, am I going to hunt for somebody to give my number to? And let them represent me? Or am I going to sit here and wait for someone to come? Okay. Okay. Well, they flipped their coin. Some of the people start asking other people, saying, hey, I'd like you to be my representative. Here is my number. But the problem is that there's limited bandwidth of the people who are getting asked. It's like you can't go out to prom with five people. But this is not what we're doing. We're adding numbers, okay? But you can only add one number.

• 2:06:27Lex Fridman?

  Yeah.

• 2:06:28Speaker 01

  So the person who has suddenly gotten asked by all these people, well, they'll have to decide who they're going to take it from. And they randomly just choose one. When they randomly choose one, all the others are rejected and they don't get to delegate anything in that round. But now, if this person has absorbed this one who said, okay, here, you take charge of my number. This person now updates their pointer. You're in charge. And this person adds the two numbers.

• 2:06:55Speaker 01

  That was the first round. In the next round, when they do the coin flipping, this person doesn't flip anymore because they're just delegating. It's that anyone who has the pointers themselves, that's like a person who is in charge of some number of informations, they flip the coin to decide, should I find other people who are agents? Or should I wait for people to ask me?

• 2:07:17Lex Fridman?

  Yes. Brilliant.

• 2:07:19Speaker 01

  This is somebody else's idea. And now the idea is, okay, if you just keep doing this process, what ends up happening? Oh, yeah. And also, by the way, if you decide that you want to go reach out to other people, here's the catch. When you're one of these agents saying, okay, I'm going to go look for someone, you have no idea who in this crowd is an agent or somebody who delegated it to someone else. You just pick a random person. When you pick the random person, if it lands on someone and the person says, oh, I actually delegated it to someone, then you follow the point. Walk up the delegation chain. Walk up delegation chain. And you can do like path compression in the algorithm to make it so you don't consistently do lots of walking up.

• 2:07:59Speaker 01

  But the bottom line is that what ends up happening is that you end up reaching out, whenever you're one of the ones reaching out, you can think of it as each agent is responsible for some number of people. It's almost like they're the leader of a bunch. As the process is evolving, you have these lumps. Each lump has an agent. And when the agent reaches out, they reach out to another lump, where the probability of them hitting that lump is proportional to the size of the lump. Yeah. That is the one funny thing about this process. This is not that they can reach out to a uniformly random lump where every lump has the same chance of getting reached out to. Yeah.

• 2:08:39Speaker 01

  The bigger the lump is, the less, the more likely it is that you end up reaching that lump. Which is a problem? Let me explain why that's a problem. Because you see, you're hoping that this has a small number of steps. Yeah. But here is a bad situation that could happen. Imagine if you had like, there are N people that you're adding up. Imagine that you have exactly square root of N lumps left, of which almost all of them are just one person who's still their own boss, their own manager. Except one giant one. One giant one. Now, what's going to happen? It's going to be a huge bottleneck because every round, the giant one can only absorb one of the others. Yes. And now you suddenly have time which is about square root of N.

• 2:09:23Speaker 01

  The square root of N is chosen because that is one where the lumps are such that you really are limited by this large one slowly sucking up the rest of them. So the heart of the question became, well, but is that just so unusual that we don't, that it doesn't usually happen? Because remember, you start with everyone just being independent. It's like a lot of lumps of size one. How naturally do the big lumps emerge? Yes. And so what that heart of the proof was, was showing that that was a joint work with E. L. Dabeski. That one was showing that.

• 2:09:55Speaker 01

  Actually, in that thing, the lumps do kind of get out of whack. And so it's not the purely logarithmic number of steps. But if you make one very slight change, which is if you are one of the agents and you have just been propositioned, possibly relayed along by a couple of different people. If you just say, don't take a random one, but accept the smallest lump.

• 2:10:20Speaker 01

  That actually does enough to even.

• 2:10:21Lex Fridman?

  That distributes the lump sizes.

• 2:10:23Speaker 01

  Yep.

• 2:10:23Lex Fridman?

  That's, I mean, yeah, it's fascinating how with the distributed algorithms, a little.

• 2:10:26Speaker 01

  Yes.

• 2:10:28Lex Fridman?

  Make all the difference in the world. Yeah, actually, by the way, this does, this back to our voting conversation. This makes me think of like, these networking systems are so fascinating to study. They immediately spring to mind ideas of how to have representation. Like I, maybe as opposed to me voting for a president, I want to vote for, for like, for you, Paul, to represent me, maybe on a particular issue. And then you'll delegate that further and then we naturally construct those kinds of networks. Because that, that feels like I can have a good conversation with you and figure out that you know what you're doing and I could delegate it to you.

• 2:11:09Lex Fridman?

  And in that way, construct a representative government, a representative decision maker. That feels, that feels really nice as opposed to like us, like a tree of height one or something. Where it's like, everybody's just, it feels like there's a lot of room for layers of representation to form organically from the bottom up. I wonder if there are systems like that. This is the cool thing about the internet and the digital space where we're so well-connected. Just like with the Novid app to distribute information about the, the, the, the spread of the disease, we can in the same way, in a distributed sense, form anything like.

• 2:11:50Lex Fridman?

  Any kind of knowledge bases that are formed in a decentralized way. And in a hierarchical way, as opposed to sort of old way where there's no mechanism for large scale, fast distributed transactional information. This is really interesting. This is where almost like network graph theory becomes practical.

• 2:12:15Speaker 01

  Yeah.

• 2:12:16Lex Fridman?

  Most of that exciting work was done in the 20th century, but most of the application will be in the 21st, which is cool to think about. Let me ask the most ridiculous question. You think P equals NP?

• 2:12:26Speaker 01

  Wow. I don't know. I mean, I would say. I know there are enough people who have very strong interest in trying to show that it is. I'm talking about government agencies.

• 2:12:42Lex Fridman?

  For security purposes.

• 2:12:45Speaker 01

  For security purposes.

• 2:12:46Lex Fridman?

  And most computer scientists, we should say, believe that P equals NP. My question, almost like, this is back to our aliens discussion. You want to think outside the box, the low probability event. What is the world? What kind of discoveries would lead us to prove that P does not equal to NP? Like, there could be giant misunderstandings or gaps in our knowledge about computer science, about theoretical computer science, about computation, which allow us to think, like, flatten all problems.

• 2:13:21Speaker 01

  Yeah, so I don't know the answer to this question. I think it's very interesting, but I actually, I know, let's put it this way. By being at Carnegie Mellon and being around the theoretical computer scientists, I know enough about what I don't know to say.

• 2:13:34Lex Fridman?

  To be humble.

• 2:13:35Speaker 01

  I'm the wrong person to answer this question.

• 2:13:38Lex Fridman?

  Yeah.

• 2:13:39Speaker 01

  It's a great one.

• 2:13:40Lex Fridman?

  Well, Scott Aronson, who's now here at UT Austin, he used to be at MIT, puts the probability of P not equals to NP at 3%.

• 2:13:51Lex Fridman?

  He put, you know, I always love it when you ask, it's very rare in science and academics because you're, because most folks are humble in, in the face of the mystery, the uncertainty of everything around us, to have the, both the humor and the guts to say, like, what are the chance that there's aliens in our galaxy, intelligent alien civilizations? As opposed to saying, I don't know, it could be, you know, it could be zero, it could be, depending on the factor, saying it's 2.5%. There's something very pleasant about just having, it's the, it's the number thing. There's power to the number, it's just like 42, it's like, why 40, I don't know, but

• 2:14:35Lex Fridman?

  it's a powerful number. And then everything, this is the power of the human psychology, is once you have the number 42, it's not that the number has meaning, but because it's placed in a book with humor around it, it has the meme effect of actually creating reality. I mean, you could say that 42 has a strong contribution of helping us colonize Mars, because it created, it gave the whatever existential crisis to many of us, including Elon Musk, when he was young, you know, reading a book like that, and then like now 42 is now part of his humor, he doesn't shut up about, he's constantly joking about, and

• 2:15:17Lex Fridman?

  that humor is spreading through our minds, and somehow this like silly number just had an effect. In that same way, after Scott told me like the 3% chance, it's stuck in my head, and I think it's been having a ripple effect on everybody else, the believing that P is not equal to NP, Scott almost has a joke saying it's 3%, is make, is actually motivating a large number of researchers to work on it.

• 2:15:43Speaker 01

  3% is high.

• 2:15:44Lex Fridman?

  It's very high.

• 2:15:45Speaker 01

  Because for the potential impact that that would have.

• 2:15:48Lex Fridman?

  But then 3% is not that high because it's only, you know, it's like, we're not very good. I feel like humans are only able to really think about like 1%, 50%. And we kind of, I think a lot of people round 3% up to 50%, like in our minds, like 3%.

• 2:16:07Speaker 01

  It could happen.

• 2:16:08Lex Fridman?

  It could happen. And it could happen is like, yeah, like half the time will probably happen. So we're not very good at that. That's, that's the other thing with the pandemic is we're not the exponential growth that we also talked about offline is something that we can't quite intuit. And that's something we probably should if we're to predict the future, to anticipate the future and to understand how to create technologies that'll let us sort of control the future.

• 2:16:39Lex Fridman?

  Can I ask you for some recommendations, maybe for books or movies in your life? Long ago, when you were baby Poe or today, that you found insightful or you learned, you learned a lot from, you would recommend to others?

• 2:16:59Speaker 01

  Yeah. So I think, I don't necessarily have an exact name of these old things, but I was generally inspired by stories, true or fictional, of campaigns, you know, like, for example, like The Lord of the Rings, that's a campaign, right? But, but the thing that always inspired me was it could be possible for somebody who's crazy enough to go up against adversity after adversity after adversity and it succeeds. I mean, those are false, like those are fictitious, but I also spent a lot of time, I guess, reading about, I don't know, I was interested somehow in like World War II history, for whatever reason. That's a campaign which is much more brutal, but nevertheless, the idea of difficulty, strategy,

• 2:17:45Speaker 01

  fighting even when things, in that case it was really fighting, but just pushing on even when things are difficult. I guess these are, these are the kinds of general stories that made me, I guess, want to work on things that would be hard and where it could be a campaign. It could be that you work on something for a year, multiple years.

• 2:18:08Speaker 01

  Because that was the point.

• 2:18:10Lex Fridman?

  Yeah, it starts with a single person. That's the interesting thing. I've obviously been, don't shut up about it recently, about World War II, especially on the Hitler side and the Stalin side. Some of that has really affected my own family, the roots of my family very much. But it's interesting to think

• 2:18:30Lex Fridman?

  that it was just an idea and one person decided to do stuff and it just builds and builds and builds. And you can truly have an impact on the world, both horrendous and and like exceptionally positive and inspiring. So, yeah, that's a, it's like, it's an agency of us individuals. Sometimes we think we're just reacting to the world, but we have the full power to actually change the world. Is there advice you can give to young folks? We talked, we give a bunch of advice on middle school, high school mathematics. Is there more general advice you would give about how to succeed in life?

• 2:19:12Lex Fridman?

  How to learn for high school students, for college students, career or life in general?

• 2:19:17Speaker 01

  So I think the first one would be to make sure that you're learning to invent and to make sure you're not just learning how to mimic. Because a lot of times you learn how to do X by watching somebody do X and then repeating X many times with different inputs. I'm just being very generic in explaining this. But I guess this is just my own attitude towards the world. I didn't like ever following anyone's directions exactly. Even if you told me this is the way to do your homework is to write in pencil, I would say, but I think pen is nice, let's try, right? So I've been that kind of a funny person. But I do encourage that if you can learn how to invent as your core skill, then you can do a lot.

• 2:20:00Speaker 01

  But then the second piece that comes with that is something I learned from my PhD advisor, PhD advisor, which was, well, make sure that what you're working on is big enough. And so in that sense, I usually advise to people, once they have learned how to invent,

• 2:20:15Speaker 01

  ideally, don't just try to settle for something comfortable. Try to see if you can aim for something which is hard, which might involve a campaign, which might be important, which might make a difference. And it's more of, I guess, rather than worrying, what if you didn't achieve that?

• 2:20:34Speaker 01

  There's also the regret of what if I didn't try? You see, that's how I operate. I don't operate based on did I succeed or fail? It was hard anyway. If I did this novit thing and the whole thing failed, would I feel terrible? No, it's a very hard problem. But would I have had the regret of not jumping in? Yes. So it's that different mentality of don't worry about the failing part as much of the make sure you give yourself the shot at those potentially unbounded opportunities.

• 2:21:02Lex Fridman?

  You almost make it sound like there's a meaning to it all. Let me ask the big ridiculous question. What do you think is the meaning of life? Or maybe the easier version of that is what brings your life joy?

• 2:21:13Speaker 01

  So I'll just answer that one personally. For me, I'm a little bit weird. I sort of, I guess you can tell by now.

• 2:21:20Lex Fridman?

  See the pen and pencil discussion from earlier, yes?

• 2:21:23Speaker 01

  Yeah, yeah. So I mean, my thing is, I guess I personally just wanted to maximize a certain score, which was for how many person years after I'm no longer here anymore, did what I do mattered?

• 2:21:40Lex Fridman?

  Yeah.

• 2:21:41Speaker 01

  And it didn't matter if it's necessarily attributed to me. It's just like, did it, did it, did it matter? And so that's what I, that's what I wanted. I guess that is very inspired by how scientists work. It's like, why do we keep talking about Newton? It's because Newton discovered some interesting things. And so Newton's score is pretty high.

• 2:22:01Speaker 01

  It's going to be infinity, right?

• 2:22:03Lex Fridman?

  Well, let's hope it's infinity, but pretty high.

• 2:22:05Speaker 01

  Ah, yes, yes.

• 2:22:07Lex Fridman?

  So you're going for, so person, the person years, you're going for like triple digits. You're going for, so like Newton is like four digits, probably like a thousand years. Or your person lifetimes, like how do you like to think about, what do we?

• 2:22:22Speaker 01

  Sorry, I meant people times years.

• 2:22:24Lex Fridman?

  People times.

• 2:22:25Speaker 01

  So then it's like, actually his is huge. His is like going to be billions or trillions.

• 2:22:29Lex Fridman?

  Yeah.

• 2:22:29Speaker 01

  Right, trillions. But I guess for me, I actually changed the metric after a while. And the reason is because you may have seen, I found some simple way to solve quadratic equations that is easier than every textbook.

• 2:22:42Lex Fridman?

  Yes.

• 2:22:43Speaker 01

  So my score might already be not bad, which is why I decided then let's change it into the number of hours in the lifetimes as well. So the way I was doing it before.

• 2:22:55Speaker 01

  Is that if a person was sort of remembering or using or appreciating what I had done for like 10 years of their life.

• 2:23:07Lex Fridman?

  Oh, I see.

• 2:23:08Speaker 01

  That would count as 10.

• 2:23:09Lex Fridman?

  I see.

• 2:23:09Speaker 01

  So if there was one person who for 10 years.

• 2:23:11Lex Fridman?

  Yes.

• 2:23:12Speaker 01

  Remembered or appreciated something I did, that counts as a score of 10 and we add up over all people.

• 2:23:16Lex Fridman?

  Yes.

• 2:23:16Speaker 01

  And then, and that was with the hypothesis that the score would be very finite in the sense that if I didn't come up with anything that might potentially help a lot of generations in a forever way, then your score will be finite because at some point it's not. People don't remember that you made like nice bottles or something.

• 2:23:36Lex Fridman?

  Yes.

• 2:23:36Speaker 01

  Right? But then after the quadratic equation thing, it was that there's some chance that that actually might make it into textbooks.

• 2:23:44Lex Fridman?

  Yes.

• 2:23:45Speaker 01

  And if it makes it in textbooks, the chance that there will be an easier way discovered is actually quite small.

• 2:23:49Lex Fridman?

  Yes.

• 2:23:50Speaker 01

  So in that case, then the score might get bigger. I was just saying the score might actually already have been achieved in.

• 2:23:56Lex Fridman?

  Oh, I see.

• 2:23:57Speaker 01

  Non trivial way.

• 2:23:58Lex Fridman?

  I see. Because it's fun to think about because it could be different. You can achieve a high score by a small number of people using it for most of their lifetime and then generations and generations. Or you can have, if we do dissipate, if we do spread, colonize, become multi-pelletary species, you could have that little, a clever way to solve differential equations

• 2:24:24Lex Fridman?

  spread through like trillions of people as they.

• 2:24:27Speaker 01

  Yes.

• 2:24:27Lex Fridman?

  Spread throughout the galaxy. And they, it would, they would only use it each one, a few hours in their lifetime.

• 2:24:34Speaker 01

  Yes.

• 2:24:34Lex Fridman?

  Their kids will use it, the kids, the kids will use it, it will spread and you'll have that impact in that kind of way.

• 2:24:40Speaker 01

  Yes. So that's why I renormalized it because I was like, well, that's kind of dumb because what's the importance of that? That'll save people 15 minutes. But so, so, so what I meant is I didn't want to count that as the main score.

• 2:24:53Lex Fridman?

  Well, I'm gonna have to try to come up with some kind of device that everyone would want to use, maybe to make coffee, because coffee seems to be the prevalent performance-enhancing chemical that everyone uses. So I'll have to think about those kinds of metrics.

• 2:25:07Speaker 01

  Yeah, but you see, that's just giving an idea of, I guess, what I found meaningful in general, like whether or not it's like, whether or not that quadratic thing is important or not. The general idea was, I wanted to do things that would outlast me.

• 2:25:19Lex Fridman?

  Yes.

• 2:25:19Speaker 01

  And that was what inspired me, and that's just how I choose what problems to work on.

• 2:25:23Lex Fridman?

  And that's a kind of immortality, is ideas that you've invented living on long after you, in the minds of others. And humans are ultimately not, are like meat vehicles that carry ideas for a brief, for just a few years. It may not be the important thing, it might be the ideas that we carry with us and invent new ones. Like, we get a bunch of baby ideas in our head. We borrow them from others, and then maybe we invent a new one. And then you one might have a life of its own. And it's fun, it's fun to think about that idea of living for many centuries to come, unless we destroy ourselves. But maybe AI will borrow it and will remember Poe

• 2:26:06Lex Fridman?

  as like that one human that helped us out before we, of course, killed him and the rest of human civilization. On that note, Poe, this is a huge honor. You're one of the great educators I've ever gotten a chance to interact with. So it's truly an honor that you would talk with me today. It means especially a lot that you would travel out to Austin to talk to me. It really means a lot. So, thank you so much. Keep on inspiring, and I'm one of your many, many students. Thank you so much for talking today.

• 2:26:39Speaker 01

  Thank you, thank you. It's actually a real honor for me to talk to you and to get this chance to have this really intellectual conversation through all of these topics.

• 2:26:47Lex Fridman?

  Thanks, Poe. Thanks for listening to this conversation with Posh and Low, and thank you to Jordan, Harvard the Show, On It, BetterHelp, Eight Sleep, and Element. Check them out in the description to support this podcast. And now, let me leave you with the words from Isaac Newton. "I can calculate the motion of heavenly bodies, but not the madness of people." Thank you for listening and hope to see you next time.