Andy Matuschak — The reason most learning tools fail

A few weeks ago, I sat beside Andy Matuschak to record how he reads a textbook. Even though my own job is to learn things, I was shocked with how much more intense, painstaking, and effective his learning process was. So I asked if we could record a conversation about how he learns and a bunch of other topics: * How he identifies and interrogates his confusion (much harder than it seems, and requires an extremely effortful and slow pace) * Why memorization is essential to understanding and decision-making * How come some people (like Tyler Cowen) can integrate so much information without an explicit note taking or spaced repetition system. * How LLMs and video games will change education * How independent researchers and writers can make money * The balance of freedom and discipline in education * Why we produce fewer von Neumann-like prodigies nowadays * How multi-trillion dollar companies like Apple (where he was previously responsible for bedrock iOS features) manage to coordinate millions of different considerations (from the cost of different components to the needs of users, etc) into new products designed by 10s of 1000s of people. Watch on YouTube https://youtu.be/dmeRQN9z504. Listen on Apple Podcasts https://podcasts.apple.com/us/podcast/the-lunar-society/id1516093381, Spotify https://open.spotify.com/show/4JH4tybY1zX6e5hjCwU6gF, or any other podcast platform. Read the full transcript here https://www.dwarkeshpatel.com/. Followme on Twitter https://twitter.com/dwarkesh_sp for updates on future episodes. To see Andy’s process in action, check out the video https://youtu.be/OFuu4pesKf0 where we record him studying a quantum physics textbook, talking aloud about his thought process, and using his memory system prototype to internalize the material. You can check out his website https://andymatuschak.org and personal notes https://notes.andymatuschak.org, and follow him on Twitter https://twitter.com/andy_matuschak. Cometeer Visit cometeer.com/lunar https://cometeer.com/lunar for $20 off your first order on the best coffee of your life! If you want to sponsor an episode, contact me at [email protected] mailto:[email protected]. Timestamps (00:00:52) - Skillful reading (00:02:30) - Do people care about understanding? (00:06:52) - Structuring effective self-teaching (00:16:37) - Memory and forgetting (00:33:10) - Andy’s memory practice (00:40:07) - Intellectual stamina (00:44:27) - New media for learning (video, games, streaming) (00:58:51) - Schools are designed for the median student (01:05:12) - Is learning inherently miserable? (01:11:57) - How Andy would structure his kids’ education (01:30:00) - The usefulness of hypertext (01:41:22) - How computer tools enable iteration (01:50:44) - Monetizing public work (02:08:36) - Spaced repetition (02:10:16) - Andy’s personal website and notes (02:12:44) - Working at Apple (02:19:25) - Spaced repetition 2 This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com https://www.dwarkesh.com?utm_medium=podcast&utm_campaign=CTA_1

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  We underappreciate the role that memory has in our lives. If what you're trying to do is to understand something pretty difficult, your ability to understand that thing is still absolutely going to be bound on your memory of the constituent material. For the median student, the education system mostly wants to make the student do things they don't want to do. It's not about helping them achieve their goals more easily or more effectively. For the most part, it's about achieving goals that aren't theirs. The histories in educational psychology that I'm most aligned with are the most robotic authoritarian kind of histories. And also the ones that are most unschooling and Montessori-esque.

• 0:39Dwarkesh Patel?

  Do LLMs make memorization more or less valuable?

• 0:43Speaker 00

  LLMs depend on our ability to externalize things and to make them legible. Basically, everyone in the educational space are focused on really the bottom quartile, not even median.

• 0:53Dwarkesh Patel?

  Today, I have the pleasure of speaking with Andy Matuschak, who is a researcher, engineer, and designer working on Tools for Thought. In addition to this podcast on Andy's YouTube channel, we did an interesting collaboration, which I encourage you all to check out, where I just watched Andy try to learn some new material. So it was just an intro chapter of quantum mechanics. And honestly, I was expecting to see some cool techniques or be impressed. But I was way more surprised than I expected to be by the deliberateness, the effortfulness of the practice. How, what was really, I mean, it was like 15 minutes a page in this textbook.

• 1:34Dwarkesh Patel?

  And any small thing that Andy thought, like, I don't fully understand this. The author is trying to say something here. He's trying to draw an analogy or relationship. I'm not sure I totally comprehend the relationship between this classical mechanics equation and the quantum mechanics equation the author thinks is analogous. Just really delving deep in that. So I was super, I thought that was really interesting that this is a way to approach new material. Yeah, so in this conversation, I'm looking forward to talking with Andy about not only that experience, but a whole bunch of his other research and the other tools he's built. Let me ask you this. So that experience made me think. Listen, this is somebody who actually cares about understanding the material like they're going through it this deliberately.

• 2:18Dwarkesh Patel?

  Do you think people in general care about actually integrating and understanding the material they're consuming in books and textbooks? Don't you think they'd make more an effort to actually assimilate that information if they cared to get it?

• 2:30Speaker 00

  Yeah, I mean, I think the statement is just a little too general probably to comment on. I mean, so I think it's certainly the case that most students don't actually want to do this because they're they're learning stuff that they don't actually care about learning, or even if they do care about learning it, often like there isn't a clear connection between whatever reading or activity they're doing in the moment and like the thing that originally inspired them for the subject, like what they actually want to do. And so there's always something tenuous going on. I think on the other hand, like, it's amazing to look at, say, subreddits and to look at the level of nerdery and fascination that will be brought to bear on, you know, gardening

• 3:10Speaker 00

  equipment or like knots, for instance, you know, people are competing to tie some very obscure 18th century knot or whatever, and they're flipping through almanacs from the period. So like, when people are interested and it connects to something that's truly meaningful for them, I think they really do want to absorb, and we see that in their behavior. There's a second thing that I think is relevant. Well, to explain this, I will reference Mortimer Adler and Van Doren's How to Read a Book, which is a great guide on on serious reading. And they consider the case of people who often have books on their bedside table, and sometimes they're like very difficult, demanding books. These are kind of aspirational, like, oh, I wish I could ring, read King Lear.

• 3:55Speaker 00

  I want to be the kind of person who reads King Lear. You put it on your bedside table, and people will like, read it before bed. And they'll find that they like, fall asleep while they're reading it, they're not really absorbing or understanding this book. I mean, it's not just an issue of memory, it's like... They they they simply are not apprehending the words on the page. And and the authors of How to Read a Book make the case that like the the issue here with these people who are falling asleep reading King Lear is in many cases, it's not that they don't want to stay awake and to really deal with that text. In many cases, it's that they actually don't know how. They butt their heads up against this very difficult wall of material. It's it's almost like maybe a rock climber who who's not very experienced going up against a wall

• 4:40Speaker 00

  that all it has is these like really subtle notches. To an experienced rock climber, those subtle notches are like a ladder, right? Like, they can get right in there and start like making some progress and seeing what's up with this wall. But if you're an inexperienced rock climber, it just looks like a solid wall. So the claim, maybe, maybe this is an optimistic claim, and you can take me to task, is that there is such a thing as being a more skillful reader. And being a more skillful reader, well, actually, in practice, in many cases, when the reading is aligned with your actual interests, produce a more serious, more understanding forward kind of reading.

• 5:17Dwarkesh Patel?

  Hmm, right. There's like two models of why people might fail to retain the material they're consuming. One is they got it at some point, but they forgot it. And the other is they they never understood it in the first place, and they just never noticed that they never understood it. And what was really, what I found really interesting was you going paragraph by paragraph, sentence by sentence, have I got this? And by the way, this is material that I had tried to go through the week before. And there were things, when you dwelled on something, I'm like, actually, I don't understand that either. And I didn't notice I didn't understand that. How how are you, how are you able to notice your confusion while you were going through?

• 5:56Speaker 00

  This is a kind of habit. It's a skill that can be built. Adler and Venderen suggest that the first and most important rule of skillful reading, active reading, is asking questions and trying to answer them. And that really, if you just dwell on that, and dwell on, well, what kinds of questions should I be asking, and how should I go about asking them, how should I go about answering them when the author isn't present, and so on and so forth, then you'll get very far. And they also say conversely, and like, this isn't meant as a criticism, an undemanding reader asks no questions and gets no answers. And I certainly have read many, many books that way, particularly before I developed this habit. And I often found myself falling into that second category of, you know, the issue was

• 6:41Speaker 00

  not that I failed to remember things, but rather that my eyes just kind of skidded across paragraphs without even realizing.

• 6:47Dwarkesh Patel?

  You're, you know, halfway through a chapter and you're thinking, what is this chapter about? Okay, so, broader question is, now that we have all these online resources, some of which, you know, you've helped develop Khan Academy and elsewhere, it seems that the value of conscientiousness as a trait has dramatically increased. If you can motivate yourself to learn these things, the world is out there for you to absorb. What are the sort of design or UI or even content modifications that can be made to give you a conscientiousness boost? Where in the past, you have a professor, you have peers, you have in-person deadlines that motivate you. Is there something equivalent to a pen and paper, how that boosts your mathematical IQ for conscientiousness?

• 7:32Speaker 00

  Right. So, one enduring result in education psychology is that when you're doing a lot of cognition, metacognition is difficult. So what I mean by that is like, when you're thinking really hard about the stuff on the page, it's very difficult for you to like, plan, regulate yourself, figure out what the best next action to do, is reflect and evaluate, you know, whether you're understanding things, you know, all the stuff that we're talking about, about asking questions, that all that gets harder as...

• 8:02Speaker 00

  As the material gets harder and as it gets less familiar. So one common thread, at least in kind of learning science stuff, has been to outsource metacognition. So some of the ways we outsource that are actually very familiar. There are things like somebody gives you a syllabus and tells you what to read when. You reference that. So that easy user interface, like that is a design practice. If you're a self motivated student, one thing you can do and that I've done is just go appropriate a syllabus from, you know, some graduate level course that corresponds to the text that you're reading and say like, well, you know, that might be a good guide as to what's most important, how to approach this. There are also lots of things that that one can build directly into the interfaces.

• 8:44Speaker 00

  Just as one example, in Quantum Country, which was a textbook that Michael Nielsen and I developed to explore some ideas around augmented reading experiences. We embedded a bunch of review questions every, say, 1500 words or so in this text on quantum computation. And our primary intention in doing this was to help people remember what they read. And we had this theory that part of what makes it hard to learn a complex subject is that there's all these new definitions and notation and terms and things being thrown at you at once and you're being asked to combine these things which are still unfamiliar.

• 9:24Speaker 00

  And so you're constantly having to retrieve these elements and struggling to do it. Either it's taking a while or your success rate is low. So anyway, that was our motivation. But it had this other metacognitive benefit that was really important. That you read 1500 words and now you're being asked these questions. That is an opportunity for you to notice that you did not, in fact, absorb what was in that thing. Not that you don't remember, but that like, you know, there's, there's a word in the question that is apparently important that you simply didn't even notice. And so not only does that give you feedback, so it tells you, oh, maybe you need to go reread that specific section, but it may also change your behavior towards future sections.

• 10:09Speaker 00

  So in interviews, readers told us, for instance, that after they reached the first set of questions or a particularly difficult set of questions, they found themselves slowing down and reading more attentively, or realizing actually that their reading practices were ineffective in general, you're kind of in the way that you were mentioning towards the start of the conversation. There's been a bunch of research on, the key phrase here is something like adjunct questions, you know, questions that go along with a text. That's kind of like what I was just talking about. And they have all kinds of effects. So the adjunct questions have the kind of effects on forward material I was just describing, and they also have the effect of making you kind of reflect on what you've just learned.

• 10:50Speaker 00

  And in addition to the questions being asked, you might find yourself pondering, well, I'm being asked about this, but why does this matter?

• 10:57Dwarkesh Patel?

  Yeah, on the point of adopting a syllabus from somebody else, I mean, one thing, one problem you might have as a self-learner is you have some goal, this is the reason I'm learning this, and then you start thinking, well, do I really need this chapter? Do I really need this content?

• 11:12Speaker 00

  Yeah.

• 11:13Dwarkesh Patel?

  At this point, you're doing the metacognition that you were using, you're trying to use a syllabus to avoid.

• 11:18Speaker 00

  Yeah.

• 11:19Dwarkesh Patel?

  Should, if you are trying to self-learn and there is a resource that is a close approximation of the syllabus you want, should you just like, hey, I don't know why I need this chapter, I'm just gonna go through it?

• 11:29Speaker 00

  Right.

• 11:30Dwarkesh Patel?

  Or should you just, should you use your own judgment there?

• 11:32Speaker 00

  This is, I think, like a pretty classic issue for learning in general. You have this problem where you have to sort of, to bootstrap yourself in a domain, you have to outsource the question of what is necessary to know. You might know, for instance, that you really want to build, you know, a model that can generate images given descriptions or something like that, like Midjourney, but you don't even know what you need to study to do that. So, you know, you pick up some textbooks on machine learning, you're kind of outsourcing the answer to this question to the author. Like what is necessary to know to build things? Maybe you can find a book that's actually labeled what you need to know to make an image generating model, but even then you're outsourcing your answer to the author.

• 12:17Speaker 00

  So you can take that answer as a start and treat it as tentative and revise it iteratively. And as you become more skilled, you can lean less on it. And you probably should. I think like a very common mistake that people make is to feel like they need to do the thing the right way, and that is exhaustive and completionist or something. If they fail because they find themselves bored or unmotivated because the material doesn't actually seem to relate to what they want to know, but they're just kind of going on faith that like, well, if I follow what the author says, everything will be good. Anyway, they find themselves having trouble for that reason and then they just stop. So this is bad.

• 13:00Speaker 00

  And they would be better off just skipping around according to their interest and continuing. One other thing I'll say about this is that the role that these syllabi play is as a scaffold. This is sort of a term apart from learning science, but it's actually, it relates to the thing we're familiar with. If you want to get higher up a building, you may not be able to climb it yourself, but you can build some scaffolding around it. And then suddenly you can reach that top shelf or the top of that building. Where the metaphor breaks down is that although scaffolding is ubiquitous in education, we give you simpler versions of questions first. That's a kind of scaffolding. We partially work the answer first.

• 13:44Speaker 00

  That's a kind of scaffolding. We give you worked examples first where we might ask you to predict the next step of the worked example. That's also a kind of scaffolding. Where the metaphor breaks down is that once you become more capable, we try to remove the scaffolding. It's called fading. The idea is that once you have solved a lot of calculus problems, you don't need half of it worked out and you're just like filling in one of the blanks anymore. And in fact, doing that would not be as effective a learning experience. So the application with the syllabi might be something like, if I'm studying something in computer science, which is a domain that I know really well, I don't need those syllabi.

• 14:27Speaker 00

  Not in the same way for most subjects. And I think that's mostly just because the amount of cognitive demand that's placed on me by the subject is just much lower than it is for other subjects. So much of it is familiar already that I can deploy my own planning more effectively as I go. But it's also the case that because I know so many things about the subject, I can do a better job from the get go of making a plan. Because making a plan requires kind of modeling a path or predicting a path or saying like, "Well, I guess I'd need to see how this connects to that," or something like this. And if your destination and your starting point are very far away,

• 15:12Speaker 00

  then you can't necessarily see all the things in between or how to draw those lines. But if those things are only a couple of hops away, you can maybe kind of infer pretty accurately.

• 15:21Dwarkesh Patel?

  Right. I guess this maybe implies that if you do want to learn about a subject, it might just be helpful to just do an intro to X subject course or textbook, not necessarily because it is instrumentally valuable to, I don't know, whatever problem you're interested in, but because it'll give you the context on by which to proceed on, like the actual learning syllabus.

• 15:45Speaker 00

  That's true. It's also the case that like you don't even know, like you don't know all the stuff there is.

• 15:50Dwarkesh Patel?

  Right.

• 15:51Speaker 00

  And this is another key problem. There's another reason why we outsource stuff. Like there's a fundamental tension in unschooling, for instance. Like just let the kids pursue what they're interested in. And like, that's cool. And there's, there's a lot of good things about that. But also like, say that a kid's like true passion turns out to be like, I don't know, ocean geology or something, and they're in a landlocked country, and there's just no one around them that talks about ocean geology. Then they're like missing out on some great opportunity. But you know, if the school had a program where they are like bringing in guest speakers or whatever, and then there's a special lecture on ocean geology from this person, and it lights up the kid's world, even if they wouldn't have chosen that lecture, like that's a good thing.

• 16:33Dwarkesh Patel?

  Yeah, Unschooling is actually an interesting subject to talk to you about. Oh, we'll get back to that. But before that, I want to ask you about this excerpt from a Paul Graham blog post.

• 16:42Speaker 00

  Yeah.

• 16:42Dwarkesh Patel?

  How You Know is the title of the post. And it says, "Reading and Experience Train Your Model of the World. And even if you forget the experience or what you read, its effect on your model of the world persists. Your mind is like a compiled program you've lost the source of. It works, but you don't know why." So it's a compiled program. You don't need the source code. Is it okay that we're forgetting so much of what we're reading?

• 17:06Speaker 00

  Yeah, yeah. I mean, what he's saying is true to some extent. Whether or not that extent is sufficient is going to depend a great deal on the situation and on what you need. If your aspiration actually depends on having a deep, detailed understanding of the material, then the kind of imprint on your worldview or on your automatic responses or something like that made by the book may not be sufficient. On the other hand, if what you want is to absorb a lot of different ways of looking at the world, knowing the details of these isn't necessarily important. Maybe you just want to know, like, well, Confucius emphasizes community and society

• 17:50Speaker 00

  as a moral patient, in contrast, maybe to the individualism of a bunch of like humanist philosophers. And like, that's kind of the level that you feel like you need to make decisions in that domain, then I think that's fine. Very practically speaking, it's funny that he uses the word compile, because like, one of the prominent theories of cognition, that is like how we come to know and learn things, is this theory called ACTAR by John Anderson. And a key part of it is this process that he calls knowledge compilation. This is the process by which we take like individual facts

• 18:29Speaker 00

  and turn them into higher level patterns that we can generalize and apply in more contexts. And I think that's what Paul is gesturing at, that you read a book, it contains like a story, a case study, and by reading it, you learn to generalize to some extent, and you apply it in other contexts when it seems relevant. And the reason why I bring up Anderson's theory is just that like he has a bunch of specific claims about like what's necessary for knowledge compilation to happen and like what you'll be able to do as a consequence of certain degrees of knowledge compilation. I think like he'd probably respond to this by saying something like, you know, actually

• 19:10Speaker 00

  in order to effectively compile things that you've learned into schemas that will match feature scenarios effectively, then you need to be exposed repeatedly to those things. You need to like use them. You need to do a variety of things that will basically show your brain that is relevant to apply these things in combination. And simply reading probably won't do that. But if you read and you have a lot of conversations and you're in a context where it's kind of demanding and it's drawing on what you read, then you may naturally do that kind of compilation step.

• 19:42Dwarkesh Patel?

  I've actually been thinking about this in preparation to talking with you, where I look back on some of my old conversations. And, you know, I've had the pleasure to talk to a lot of interesting people across a lot of different fields. And at the time I interviewed them and had done all the prep, I actually kind of had a lot more context than I can remember now. Sometimes I'll listen back to a conversation and I won't even remember the content in the conversation. And I know, I remember thinking after the conversation, I knew so much more about this field than was compressed into this one hour interview, right? I had to prep other things that might come up. And afterwards, I'm like, I don't even remember the things that were in this one hour. But then the other part of me thinks, well, I'm getting better at doing the podcast. That might imply that, you know, I've picked up something. But it is a shame that I didn't have some sort of rigorous practice throughout

• 20:27Dwarkesh Patel?

  the time of, you know, retaining the material that I was keeping.

• 20:31Speaker 00

  Well, yeah, I mean, I expect the main way in which you're getting better is actually not really about any of the details of those materials. I think it's about your practices as an interviewer, the way that you generate questions. Like you probably have a bunch of patterns, whether you know it or not. Like you read a thing that a person has written in hopes of generating good questions about it. And even though you don't have this habit for textbooks, yet maybe, of constantly demanding things of the textbook, you have, I think, started to develop this of essays or blog posts that interesting people you're interviewing have read. And to point to this Anderson theory,

• 21:11Speaker 00

  like, in the course of repeatedly doing that, you've made it automatic, parts of it automatic, so that you don't need to do it consciously. You can focus more on the material. You can probably take on more difficult material, or actually understand material at a higher level than you could have before, because less of yourself is engaged in this kind of question of how do I make the questions from the material?

• 21:37Dwarkesh Patel?

  Yeah, I certainly hope so. Otherwise, there's a question to be asked of what I've been doing all these years. So, you know, having interviewed some of these people who are infobores and have consumed and continuously consume a lot of content, this is something you also noticed and pointed out in your notes. But, you know, Tyler Cowen, for example, I don't think he has any sort of note-taking practice.

• 21:58Speaker 00

  No.

• 21:58Dwarkesh Patel?

  He just, you know, just devours information.

• 22:02Speaker 00

  Yeah.

• 22:02Dwarkesh Patel?

  What is your theory of how these people are integrating things?

• 22:06Speaker 00

  Right.

• 22:06Dwarkesh Patel?

  That they're...

• 22:07Speaker 00

  Tyler's a good example. I think he's actually a little easier than some others we might discuss. So let's talk about Tyler for a second. One of the other things that's so interesting about Tyler is his writing obligations. So this is a man who's blogged every day since, I don't know, 2007 or something like this, and has a Bloomberg column, I think, weekly, something like 1,500 words, and also has published something like a book a year for, I don't know, a decade or more, and occasionally publishes some academic articles, plus like a bunch of other collateral. So like, that is notes. And I think it's also important to note that like, the way that Tyler writes these blog posts, and the way that Tyler does these columns, and even the books, is very different from the way that

• 22:51Speaker 00

  many other book authors work. Like Tyler, the blog posts often have this like real first draft mentality to them. Like, he's just thinking out loud.

• 23:02Dwarkesh Patel?

  Right, right.

• 23:03Speaker 00

  And he's got decades of practice thinking out loud, and like writing down a decent take the first time. And so, you know, he gets something pretty good the first time, much of the time. And that works for him. So like, that kind of is a note, right? Like, doing the thing, I guess, like your initial thoughts on the subject is kind of what you would write in a note.

• 23:22Dwarkesh Patel?

  Yeah, one of my former guests, Scott Young, was comparing Brian Kaplan into Tyler Cowen's books. And he said, you read a Brian Kaplan book and it's like a chess game.

• 23:30Speaker 00

  Right.

• 23:30Dwarkesh Patel?

  Like, the opponent is, if you try to move a pawn up on this case for education, I've got this rook that I can move here. With Tyler, it's more like, you know, he's like shooting the shit on a subject, basically.

• 23:41Speaker 00

  Yeah, yeah. Panglosshui train station.

• 23:43Dwarkesh Patel?

  Yeah, right, right, right. On a separate question, do LLMs make memorization more or less valuable? Right, so there's a case you can make for both, but on net, is it more important to have more onkey cards in your deck? Now that GP4 is out?

• 23:57Speaker 00

  Maybe this is a good time to talk about what memorization is, or like what it's for. So we could use that word to refer to the practice of like learning more trivia, for instance. So for instance, a thing that I and some people I know have done is like gone through this book, Sell Biology by the Numbers, which says all of these things, like, how big exactly is a nucleotide? Like, how much volume does it take up? It's kind of helpful occasionally to know that it's about a nanoliter and that that can help you model things. So you can just like commit all of those things to memory, right? That's one kind of memorization. And we could talk about how LLMs affect that. But I just want to make the case that so much of what you do and experience day to day is memory bound,

• 24:41Speaker 00

  or is memory influenced in important ways. So just for instance, your ability to understand a difficult argument, even in the course of a text, is memory bound. Some of that's working memory, but your ability to understand an argument that has many steps in it, more steps than you can keep in your working memory, depends on your ability to think of some of those steps in terms of some stuff that you already know, so that you can kind of reduce it or abstract it. Likewise, in creative work, there's a bunch of studies trying to catalog case studies of how it is that people have flashes of insight. You know, it's a little hard to talk about that.

• 25:24Speaker 00

  But one of the things that's a pretty consistent source for insight for people is noticing a surprising connection or a surprising contradiction. It probably feels pretty familiar, right? Like, you're reading through the newspaper and you see like, you know, people have finally figured out how to do X. And you're like, wait a minute, that means if I combine it with this other thing, like, we'd be able to do Y, or something like that, you know? Now, that's only possible if the other thing is in your memory. Like, if you have to think to look up the other thing, then the newspaper wouldn't seem so salient to you. Likewise, like, and in just really boring ways, early on in my time in Khan Academy, I just learned, in a very thorough way, using memory systems,

• 26:08Speaker 00

  just a whole lot of details about the education market.

• 26:14Speaker 00

  And this let me be in, you know, high-level executive kind of conversations where we're trying to figure out strategy stuff, and somebody would propose a particular direction, or, you know, what about this, what about that? I could say, like, "Well, you know, the total budget spending for instructional materials is this, and like, that market is growing by this percent per year, and 10% of students in the US are in this place," and, you know, and so on and so forth. And so, basically, like, on the fly, I can evaluate ideas in a way that others can't. So, anyway, this and other things are kind of just part of my rant about people. I think, in general, we underappreciate the role that memory has in our lives. So, just to come back to the question, explicit memorization, or explicit making sure that you can recall the thing reliably,

• 26:59Speaker 00

  we can test it against these things. So, for the case of the creative instinct, for instance, noticing the contradiction, noticing the connection, I imagine that we will have future notebooks that will do some of this noticing with us, and that will decrease our need to be able to rely on our own sense of salience, or something like that. But I guess I don't know how much. I'm skeptical that, like, my own experience coming up with weird ideas that feel very new is that it feels very personal, it feels very inquiet. I often haven't been able to describe textually the constituents of the thing very clearly.

• 27:45Speaker 00

  There's just kind of a feeling that something in this general direction is connected with something in that general direction, or there's a tension, and so that makes me a little hesitant. L L Ms depend on our ability to externalize things and to make them legible. Back to the learning point about the role of memory. If what you're trying to do is to understand something pretty difficult, you, your ability to understand that thing is still absolutely going to be bound on your, your memory of the constituent material.

• 28:11Dwarkesh Patel?

  Do you think there's value in, there's pedagogical value in forgetting? So I guess some sort of anecdotal or unrelated evidence is in, you know, neural networks, sometimes you can improve performance by pruning some of the weights. Obviously, we forget things, right? So clearly, we don't remember everything. When we sleep, we lose a lot of our memories. Is it possible that by forgetting the details and only getting the gist, that actually helps us better generalize the insights we're getting from text and things like that? What do you think of that way of thinking?

• 28:42Speaker 00

  Yeah, it could be. So memory is very connected to attention. And we can't attend to everything, right? So one, one of the roles of memory is to help guide us to the things that are important. You know, so like, maybe, maybe I happen to know, uh, you know, the magnitude and energy of an electron volt. That's something I can draw on because of the memory system stuff. But I also don't like, I don't want that to be like front and center in my mind all the time. I don't want it to be hyper salient the way that, I don't know, some very important design principle is to me. So yeah, there, there's, there's some role there. There's also, there's some theories that, um, the reason we have forgetting is that our environment, our ancestral environment was very traumatic. And so our episodic memory in particular, we would like to maybe not be all that faithful.

• 29:26Dwarkesh Patel?

  Hmm.

• 29:27Speaker 00

  I, I, I actually, I don't know the status of those theories.

• 29:30Dwarkesh Patel?

  Probably why you forget dreams as well, right? Like dreams are pretty traumatic. If you, you thought of them at the same as a real life experience.

• 29:36Speaker 00

  Yeah. So, I mean, another weird thing about memory is that as far as we can determine, memories aren't lost exactly, at least not completely. There's a bunch, there's a series of interesting experiments that people have used to demonstrate that, you know, decades later, like things are still there. If, if you can cue them right, uh, people can bring things back, uh, even things that they, they feel are lost. Uh, and of course, like, you can also cue people in ways that are hallucinatory. So we need to be careful about that.

• 30:06Speaker 00

  I guess the reason why I bring that up is that it flies in the face of this view that like there's a limit. Like, one of the things that I think is kind of weird about this memory system stuff, or like memory championships, or something like that, is people are like, "Oh, if you do these things, like, will you start to forget other, like, normal human stuff?" And what's weird is like, no. You know, I've been doing this memory system stuff for years. Like, I just know more stuff now. And this is aligned with the experimental literature, which seems to suggest that like, there's probably upper bounds, but we're not close to them. And some of these memory champions have memorized, I mean, truly absurd, you know, orders of magnitude, maybe two orders of magnitude more things than I have practiced.

• 30:50Speaker 00

  Certainly, people who are multilingual have really, really absurd numbers of things memorized. So there isn't like a resource management argument.

• 30:59Dwarkesh Patel?

  Huh. Well, if there isn't, why do we forget so many things? Is there some reason the brain just remembers some of the things we're bringing across? Maybe they're just not, we were training the ancestral environment to find certain things salient that just don't map onto books?

• 31:15Speaker 00

  No, it's a good question. So we're getting to, we're getting to a part of the cognitive science space that I'm less familiar with and also that I suspect we simply know less about. But let me just riff a little bit. One of the things that we sort of know is this idea of spreading activation. So when you go to try to look something up, or when you try to deal with a particular situation, there's something almost kind of like DNS exchanges, or like routing on a network or something, where, okay, like, we start from some point that is like a stimulus, and just speaking very informally, like, we kind of expand outwards from there, and there are effectively like weights on those connections.

• 31:59Speaker 00

  And by tuning those weights effectively, we like route the packets on the network effectively. Memory is encoded in these weights, at least partially. So if you didn't forget things, then you might just have this like weird cacophony on the network, and in particular, like... What's salient, what to do next, like which response seems most appropriate to this question? Like you might answer those kinds of things very ineffectively because all this stuff is coming up for you that is like much less relevant. Like one of the theories about how well we remember stuff in what circumstances is actually called predictive utility theory. And it suggests that like the probability of retrieval of a particular item in a given situation

• 32:42Speaker 00

  actually does correspond with basically a model of how like to what extent the brain predicts it will be useful.

• 32:49Dwarkesh Patel?

  Right. And then the prediction doesn't necessarily map on to?

• 32:54Speaker 00

  Doesn't necessarily, exactly. And so like, you know, when you repeatedly access something, when you practice retrieving it, the prediction of the utility of the thing goes up.

• 33:03Dwarkesh Patel?

  Yeah.

• 33:03Speaker 00

  And when you do it in a variety of situations, it goes up across a broader distribution.

• 33:08Dwarkesh Patel?

  Hmm. Okay. So that this is interesting. When did you start your memory practice? Presumably it was after after Apple?

• 33:15Speaker 00

  Yeah.

• 33:15Dwarkesh Patel?

  Okay. So let me ask you this. So at Apple, you were in charge of a bunch of important flagship features on iOS. And I'm guessing other things presumably you were, you didn't have some sort of practice, but since you were encountering these things day to day, that natural frequency and way in which problems came up. Did you have a worse understanding of those problems and the things now, knowing what you do? And knowing that having the practices you do, you're able to comprehend now? I don't know if that question made sense.

• 33:43Speaker 00

  No, that's a great question. So I mean, like, here's the fun thing. I was much better at what I was doing then than I am at what I'm doing now.

• 33:55Speaker 00

  That's pretty funny. So I mean, it was just totally different. Let's talk about this a little bit. This feels very, very juicy for me. Most of what I was doing was engineering, some of it very difficult engineering, but mostly engineering, mostly on. Like things that were fairly well understood. So I wasn't trying to decide what what should be done usually. Sometimes I was from a technical perspective, but certainly rarely from a product perspective. Was that a relevant question for me? Um, I was kind of like a somewhat design minded engineer and I did a bunch of kind of engineering and design ish things on tasks which were set out for me. At that point, I had been programming for a really long time.

• 34:39Speaker 00

  I don't know, 13 years, maybe by the time I joined Apple, longer. Um, and programming in Apple's ecosystem for probably two thirds of that time, three quarters of that time. So everything was just really familiar and like it was mostly flow all the time. Every day I was kind of like, I was just in it. Uh, I knew the stuff that I needed to know. I was very well practiced and the space didn't change that much. You know, like engineers, most engineers at Apple, most of the time are not like pushing the frontier of what is known, like trying to discover, they're like doing very difficult technical work, mostly applying things that they already know and understand quite well.

• 35:22Speaker 00

  Um, to problems which are usually not always moderately well, but pretty well understood. Let's say. Memory was essential to me doing that job well, but I had already built most of it by the time I got there. I'd already built just tons of stuff for Apple's platform and I had to learn a lot of stuff. I learned a ton of stuff about the internals of those systems, but because I already had such a rich understanding, both of Apple's platforms and of computer science and engineering in general, uh, I had this really rich network for stuff to slot into. Um, so learning stuff is easier when you you have other stuff to connect it to is is a nice principle. Metacognitive load on me was lighter, um, because others were figuring out what we should be doing. So just like my contrast now, I'm doing research, like I'm trying to discover things that are not known.

• 36:07Speaker 00

  I'm trying to make things that didn't exist. The hard questions that I answer are mostly like what should be done or like what should I do? And that question is not just a technical one of like, how should I implement this like feature that needs to get built, but like what intervention on a reader should be taken? That requires synthesizing.

• 36:28Speaker 00

  Lots of different unfamiliar literatures.

• 36:30Dwarkesh Patel?

  There's two different threads I want to go on. Maybe I'll just mention the other one. This is also related to the thing we're talking about a few minutes ago with LLMs. And there's this thing that I'm sure you've talked about yourself as well, the Swanson linking, where this guy was just...

• 36:44Speaker 00

  I don't know Swanson linking.

• 36:45Dwarkesh Patel?

  Okay. Actually, Michael Nielsen has written about it in his work.

• 36:47Speaker 00

  Okay, cool.

• 36:48Dwarkesh Patel?

  But this guy was just somebody who read the medical literature. And he was just like familiar with a lot of esoteric results. And so different things would come up and he would be able to figure out what different things are connected. For example, I think he noticed in one case that headaches are linked to some other symptom and that other symptom is linked to magnesium deficiency. And so apparently a whole bunch of people's headaches were solved once they were given a magnesium supplement. And he noticed that connection. Again, this is the kind of sort of combinatorial thing that you wouldn't have noticed otherwise. But on this, on this subject itself, so listen, there's this natural way in which you're able to get up to speed in all of the things that are happening at Apple.

• 37:32Dwarkesh Patel?

  And is it possible and maybe advantageous to do similar kinds of things in other fields? For example, instead of doing an explicit spaced repetition system when you're trying to absorb material from books, you just read a cluster of books and hopefully things will just come up that are relevant to get them again? Or is there value in having explicit practice of setting up cards and so on?

• 37:52Speaker 00

  Yeah, right. So again, the answer is going to be it depends. I think that maybe the most familiar example of what you're talking about is immersion learning a new language. Right? So immersion learning is like a great thing. And it's going to be more interesting and more effective than doing spaced repetition practice. It's going to be integrative. It's going to be socially based. So there's a bunch of stuff about social learning that's relevant. A problem is though, is that like, say that you decide you want to learn Swahili today and you like go down to like the local Swahili community center and you're like, cool, I'm going to immerse myself. Like, good luck. You know, you can't even get started.

• 38:31Dwarkesh Patel?

  Yeah.

• 38:31Speaker 00

  So through this lens, explicit practice is a way to bootstrap yourself.

• 38:35Dwarkesh Patel?

  Right.

• 38:35Speaker 00

  Likewise, a great way to become a better musician. All of the best pianists at sight reading that I knew in university played with churches. They were so good at sight reading because, you know, they had to show up every Sunday and they're playing a different thing. Uh, knew him every Sunday, right? So this is immersion also. And, you know, over time, they're learning all these cadences and these things that are really common and whatever. But like, you can't show up and be the church pianist every Sunday in the first place if you don't already have like some decent foundation. So this is sort of a bootstrapping argument that like one role for explicit practice of this kind is to get yourself into a position where you can more naturalistically reinforce.

• 39:17Dwarkesh Patel?

  I see. Okay, got it.

• 39:18Speaker 00

  But there are still going to be instances where the naturalistic reinforcement isn't going to work. So, so for example, the, the linking that you brought up, one issue for doctors is rare diagnoses.

• 39:29Dwarkesh Patel?

  Yep.

• 39:30Speaker 00

  So if it's only going to be once every couple of years that you see a patient that's going to present with these symptoms, that's not going to be frequent enough to naturally reinforce your, your memory of that. And you're going to need some out of band mechanism. And unfortunately, I think for, for many kind of creative leaps and creative insights, that may be closer to the regime that we're in.

• 39:50Dwarkesh Patel?

  Yeah, that makes a lot of sense where in many fields, you just have the things you're regularly doing is a thing you need to reinforce. It makes a lot of sense that if you're a researcher, the long tail of events that might come up is a thing you're, you know, it might happen once every few months, but the regularity is not a thing that matters, right? It's sort of effect on your work. Here, here's a question I actually have. So when we were doing the practice or when we were doing the, the quantum mechanics textbook, it was like three hours and afterwards I was just exhausted. And I was actually surprised that you went the entire three hours without interruption. And so afterwards I was packing up and you were like, hey, I'm about to actually go to my piano lesson. I was so confused at how you had the stamina to keep going.

• 40:33Dwarkesh Patel?

  Is, is the stamina just inherent in you or is that something you did to develop?

• 40:37Speaker 00

  So one of the things that I, I think is funny about stamina is first off, there's some kind of weird grass is always greener kind of situation where like, I often feel struck by other people's stamina and feel like I have very little of it. I struggle with energy. Um, I've actually written extensively about all my struggles with energy and like ways of managing energy. I, I spent a lot of time thinking about it and like managing my energy levels, structuring my day around it. So I think there is something where like one often feels maybe lower stamina than one actually is because one misapprehends other's stamina. Okay. So, so in that particular situation, how do I explain why three hours of studying, et cetera? First off, social. So, uh, if I were alone and studying that book for three hours, um, and I weren't.

• 41:21Speaker 00

  Yeah, effectively trying to perform for you, Dwarkesh. It wouldn't have been nearly as energizing for me, and I definitely would have taken breaks. I still would have been able to go for three hours, I think. Part of the reason for that is that it's simply way less hard than things I normally do. In some sense, learning quantum mechanics should be like much harder, and it kind of is cognitively demanding in a lot of ways.

• 41:46Dwarkesh Patel?

  Right.

• 41:47Speaker 00

  It's much more cognitively demanding in kind of a direct way than what I actually do day to day. But it's much less, it's much less demanding on what William James calls the energies of men, which is something like, like a life force that permits you to act according to your will or something like that. Maybe it's gumption, maybe it's willpower, maybe some people call it spoons, I don't know. These aren't all the same thing exactly. But sitting and staring at a page and deciding what you should do next on a research project is incredibly draining on that resource. The not knowing, sitting and not knowing is like the hardest thing that I do in my work.

• 42:29Speaker 00

  And so there's something, it's like a wonderful vacation to be presented with, oh, great, somebody else is going to tell me what to do.

• 42:37Dwarkesh Patel?

  Right.

• 42:38Speaker 00

  This is great.

• 42:39Dwarkesh Patel?

  So although it might be less demanding than your usual work, it is definitely more demanding than the way in which I or most people approach textbooks or other material in the sense that, you know, I would just like read through and then once I get to the exercise, I'm like, let's see what I didn't understand. Whereas just the attention and the intensity to go through sentence by sentence and constantly paying attention seems like way more exhausting than.

• 43:07Speaker 00

  Yeah. I mean, so this is sort of true, like, and it's definitely the case that I will occasionally do some of this like before bed reading where I think like, oh, like, you know, let me just do a little bit more, you know, and it's like, it's basically useless.

• 43:18Dwarkesh Patel?

  Yeah.

• 43:19Speaker 00

  But I want to make the case that there is a kind of pocket that you can fall into, maybe you call it flow, I don't know, where like the demandingness that you're bringing to bear is matched to your ability, the book is not overwhelming, like you feel like you can make your way through it. And this is actually more engaging.

• 43:37Speaker 00

  So I occasionally will find myself reading as an undemanding reader and finding my attention kind of slipping because I feel like I'm just not that attached to the text emotionally. I'm kind of reading dutifully. I'm like trying to get through it. That produces sometimes like an adversarial aspect where the text is in my way. Or it's kind of something to be, I don't know, like accomplished.

• 44:09Dwarkesh Patel?

  Yep.

• 44:10Speaker 00

  And often then I will find that I need to bring more gumption to bear to kind of power through and like, you know, make myself sit there and keep flipping the pages than I need if I actually just like open my curiosity and open my my attention and really start engaging with the book.

• 44:26Dwarkesh Patel?

  There are sorts of ideas that people have come up with for different pedagogical tools, which are mediums that give closer connection to the reader. One is, you know, you have some sort of fiction account where a concept is introduced and reinforced. Or you have a video game with characters you care about. And as far as I know, there isn't something that has really taken off using these sorts of new mediums. Why do you think that is? Is it just an inherent limitation of everything but text and lectures? Or people just haven't given it the right content and?

• 45:00Speaker 00

  Yeah.

• 45:00Dwarkesh Patel?

  Design?

• 45:01Speaker 00

  Yeah, I'm fascinated by this question. Let's see, I can say a few things about it. One is that I would argue that one medium has taken off in an absolutely enormous way, and that's video. People love video. People will watch Grant Sanderson spend, you know, an hour going through some explanation of an esoteric math problem. People who would never crack like a Springer graduate textbook in mathematics or something like that. The issue is that like they will in general not walk away from that interaction with much understanding.

• 45:31Dwarkesh Patel?

  Yeah.

• 45:32Speaker 00

  But they're much more engaged.

• 45:33Dwarkesh Patel?

  Yes.

• 45:34Speaker 00

  So that's cool. That's suggestive, you know, and suggests the question like, well, but like, is there a version of that that actually produced detailed understanding? Maybe one approach to producing that might be like a game. My, my favorite example of this is The Witness by Jonathan Blow. Have you played The Witness? No. Okay, I, I think The Witness is an absolutely extraordinary work of art. So it, it's a game that, that has no text, at least no text that's relevant to the game elements. In, in kind of classic Mist style, you, you wake up on an island and figure out what's going on. And the game proceeds to explain to you without using words,

• 46:14Speaker 00

  but just by shaping your environment, a series of extremely complex mechanics, basically, of a system that exists in this, in this world. So you, you learn a bunch of stuff and it gets to the point where it feels like you're in conversation with the game's designers. It's like, ah, he's asking me to do this here. No one's asking you, right? There's no text, but you can feel that you are being asked. And you perform some interaction in the environment and you feel that you have answered. And the game responds in kind. This is very, very interesting. It's, it's like, it's a medium of action. Some people have tried to make educational games, games which are explicitly about arithmetic or something.

• 46:59Speaker 00

  Jonathan Blow's game is not about that. It's, the mechanics that you learn are, they're like about the environment. I don't think anybody has yet really succeeded in doing this about explicit subjects. There are, for instance, things like Kerbal Space Program. Maybe people learn some things about project management or orbital mechanics from that. Zactronics has a bunch of games that are sort of about assembly language, roughly speaking. Maybe you can learn some things about that. The issue seems to be that games are ultimately, they're, they're an aesthetic form. Like, the, the purpose of the game is to have an experience that feels a particular way. And so they're, they're sort of serving a different purpose than Grant's videos or a text.

• 47:45Speaker 00

  Grant's videos are also serving a different purpose from the text. Like the text you might pick up because you're like, I want to be able to build a robot. Like, so you pick up a textbook on robotics or something. And so is there something that you can pick up that's sort of like a game insofar as it's an active environment that you'd use in a similar situation to, I want to learn to build a robot? Maybe, kind of. We don't, we don't quite have those yet. We have some things that are kind of like that. I don't know if you've seen like From NAND to Tetris?

• 48:14Dwarkesh Patel?

  Oh, yeah, yeah, yeah.

• 48:16Speaker 00

  This is a very interesting project that's kind of along these lines. And what characterizes it, like games, is doing. It's active. So when I was asking all those questions of the book, that was active learning, active reading. NAND to Tetris is naturally active. So this is a course in which you're, you kind of start with basically nothing. You start with memory and, and you, you build a, a virtual computer and you build Tetris. You build a processor and stuff. And so the whole thing's active, the whole time, you're, you're, you're making the computer grow. This is doing a similar job to the question asking that I was doing, except that you don't have to regulate all of that yourself.

• 48:56Speaker 00

  The regulation, the choice of what activity to do is in the course, is in the structure of, of the material. I think there is waiting to be created some kind of mass medium that is like that, but that can be applied in many, many circumstances. We have the non-mass medium version of it already, and it's apprenticeship. Like, if you want to be a good yoga teacher, you like, go hang out in yoga studios. If you want to be a good surfer, like, you go to the beach when the other surfers are there and you like, participate peripherally, and you talk to them, and you learn about their tactics. They might give you some feedback eventually. And you'll start to participate less and less peripherally over time.

• 49:41Speaker 00

  And eventually you'll be part of the community. This isn't a mass medium. We can't print a billion copies of it. We can with a book.

• 49:47Dwarkesh Patel?

  What is the experience of watching George Hotson on a stream, a quote of TinyGrad? How does that compare to just being in an office with him?

• 49:54Speaker 00

  Yeah.

• 49:55Dwarkesh Patel?

  Um, cause even if you're in an office with him, there's, you know, there's, there'd be constraints on his time and how much engagement there would be. So why, why isn't video a scalable way to increase apprenticeship?

• 50:06Speaker 00

  I'm incredibly excited about streaming, actually, as, as a medium for this. So, so we're kind of gesturing at a particular kind of, of learning that needs to happen. It's often called tacit knowledge. So like one of the things that you have to learn to do as an engineer is to learn to deal with a hundred thousand different, like weird situations where something is not behaving the right way. And eventually you learn a kind of pattern recognition. You learn ways of dealing with this. And, uh, much of this is like not described in any book. It's not explicitly taught. You just kind of learn it by doing it over a long period of time. So by watching George do it, I, I think that people do absorb stuff. They can absorb some of that knowledge. That's part of how apprentices absorb that knowledge.

• 50:51Speaker 00

  There's a few things that are missing. You know, you, you're not getting feedback. There's a whole lot of chaff there. Like there's a whole lot of stuff that probably isn't all that meaningful. It's also true for apprentices. Um, so I'm pretty, actually I'm pretty excited about streaming videos. I, I have, um, I've complained loudly that there aren't more designer streamers. So one, one of the things that, that I think is really interesting is that we have some disciplines like programming, where there are like a million books on courses, uh, about how to learn to program. Uh, and they don't, they don't give you everything you need. There's this tacit knowledge stuff that you, you need to develop, but like, you know, like if you work through these courses, if you go through the MIT OpenCourseWare for computer science, like you'll be able to build some stuff.

• 51:36Speaker 00

  Uh, and, and you'll be able to leave yourself up. Um, this is not true in all domains. You know, if you're looking at, uh, well, in particular design, but lots of other domains that are sort of like that, like, uh, musical composition, architecture, something like this. Nope. Like it's normally done in studio classes, uh, lots and lots of hands-on feedback. Um, stuff has, like the, the feedback is highly contingent. It's highly contextual. Uh, we just haven't figured out how to communicate this. And so it's good to see lots of programmer streamers, but I really want to see the streamers in these other domains.

• 52:09Dwarkesh Patel?

  Right. On the point about the more programming books, ironically, the reason why there's so more resources on programming is that it's just so legible, but it already makes it easier to understand in the first place. So you just have this reinforcement. But also, you know, the nantitestress is, let's say it's like a video game analog to learning, maybe not just programming, but how things in the internals of a computer work.

• 52:33Speaker 00

  Yeah.

• 52:33Dwarkesh Patel?

  But programming has an element where it already feels like a video game. I have a friend who has a sort of intense sort of manic energy and he used to be addicted to video games when he was a teenager. And now he just stays up all night in these coding binges, just the same part of the brain. Are you optimistic about things like video games and fiction being able to work in fields that are not already kind of like a video game, like programming?

• 52:57Speaker 00

  I think what makes programming feel like a video game is this, this sense of instantaneousness, this sense of direct contact with the environment. You're learning about a kind of a conceptual world, but that world is, is right underneath your hands. And as you manipulate it, you're constantly getting this feedback, the red squiggly, you're pressing command R regularly. And you're, you're seeing it fail and that feels great. Like there's this feeling that's, it's very common for programmers and, and it's, it's laden with doom. The feeling is, uh, it's like nine PM and you've been working on a thing all day and it's almost working.

• 53:42Speaker 00

  It's almost working. And you know, like, if you just debug this one thing, then like your project will be like done and you'll be able to like go to. So you're like, well, I'll just stay up and like, I'll, I'll debug this one last thing. And then you start debugging it and you get it and you solve it and that feels great. And then immediately you like run into like one more thing, like, oh, it's almost running all the way through. It's almost going end to end. And you're like, well, I'll just stay up a little bit longer. And you know, before you know it, it's like 2 AM and you keep going, cause it, it feels so good. Like you, you feel the sense of forward progress. You're not just staring at a wall. I think like for the programming problems where you are at a brick wall, it doesn't feel like this. It feels bad. So can every field be transformed into something where you can feel the sense of forward progress?

• 54:26Speaker 00

  You can get this rapid feedback cycle. I think that's really hard. I think some fields, like it's not clear to me that they can be transformed in that way. I think we can like get a lot closer in most cases than than we're at right now. So what's hard about designing a user interface is that often there's this feeling of exploring kind of a combinatorial search space. Programming often feels like a search problem, too. You have a sense that there's like, there's some right way to solve the problem. There might be some set of right ways to solve the problem and you're kind of looking for it. And you have some heuristics that guide you to like, oh, this might be a dynamic programming

• 55:10Speaker 00

  problem, or this might be something that is solved well by, you know, separating concerns or something like that. Design often feels less like that. You have those heuristics, too. You have those patterns, too. Often, it just feels like, nope, I just need to try like 300 things. And so there's this like characteristic where you'll look at a designer's Figma file and like what you do in Figma is you press Command D. This is like the core action of Figma is to duplicate. So you have an artboard, you tried something, and you're like, that didn't work. So you select it and you press Command D. And what you end up with, and you look at design Twitter, and it's just all these screenshots of people's Figma's with like a million artboards. They're just trying stuff.

• 55:52Speaker 00

  And you don't have this feeling, or at least I don't, and I think many designers don't have this feeling of like progress. It's like you're just kind of exploring parts of the search space and like you're learning that parts of the search space don't work. And, you know, eventually you stumble on one that does, but you don't have this feeling of like getting closer often. Often, there will be like weeks that go by without feeling like you're getting closer because what you're doing is just kind of like narrowing the search space.

• 56:22Dwarkesh Patel?

  Interesting. Although there are people who are obsessed with design.

• 56:25Speaker 00

  Oh, yeah.

• 56:26Dwarkesh Patel?

  And what is the sort of loop that keeps them obsessed with a process that doesn't feel intrinsically forward feeding?

• 56:35Speaker 00

  So to some extent, I think they are skillful. So the people that I know who are like this, it's a combination of their skillful often and like the nature of the problems that they're solving are highly tractable. So like a kind of an example of a kind of thing that designers will often rabbit hole into is like designing a poster. It actually often used to be kind of a cliche that at Facebook, there were all these posters up on the wall of the office. Very, very elaborate, beautifully designed posters for like a talk that someone was coming to give at Facebook. And it's like, why, why did somebody put all this effort into it? Like, well, it feels really good.

• 57:18Speaker 00

  Because a poster is this like really constrained, it's like finite, it's ephemeral. You can start it and within a few hours, like, yeah, there's a search space, but you can find a decent part of the search space pretty rapidly. And once you're there, there's this beautiful and very enticing feeling of turning the crank and like making it better and polishing it and like trying this or that. But when you're trying this or that, like all of the options are kind of okay and you're kind of trying them out of curiosity. Or like, maybe it can be even better. And that's very different from the kind of design where you're just like, I simply don't know how to do this. And I think it's part of why

• 58:01Speaker 00

  those designers loved making those posters. It's like, it's a snack, it's a treat. It's also something they get to control, whereas ordinarily they don't.

• 58:11Dwarkesh Patel?

  Yeah, just, just don't tell the manager how many software engineering hours were used up in the poster designing at Facebook.

• 58:18Speaker 00

  Well, yes, it's not no software engineering.

• 58:19Dwarkesh Patel?

  Oh, okay.

• 58:20Speaker 00

  It's only designers.

• 58:21Dwarkesh Patel?

  Got it.

• 58:21Speaker 00

  But for the software engineers, I mean, Code golf is the equivalent, right?

• 58:26Dwarkesh Patel?

  What is code golf?

• 58:26Speaker 00

  You know, in golf, like you try to get the lowest score.

• 58:28Dwarkesh Patel?

  Uh huh.

• 58:29Speaker 00

  Um, so code golf, you try to solve the problem uh as minimally as possible, right? Like, ah, ah, you know, I don't need this. I can, I can like combine this, I can do it in three lines. If I, if I use Haskell, I can do it in one line. That's a, that's a kind of thing programmers do that's, that's like this. But just endless refactoring is, is another thing that's kind of like this. You have the thing working, but it could be more beautiful.

• 58:50Dwarkesh Patel?

  Right. Well, so it seems like uh the tools and the ideas you're developing seem especially geared towards very intelligent and very motivated students. Uh, if, if they would be different, how, what would the tools um that you would develop for a median student, the education system look like? Both in motivation and in other, you know, other traits.

• 59:14Speaker 00

  Yeah, they'd be super different. Uh, I, I kind of got out of the educational space in part because I don't like the framing of this problem. Like, for the median student, the education system mostly wants to make the student do things they don't want to do. It's not about helping them achieve their goals more easily or more effectively. For the most part, it's about like achieving goals that aren't theirs. Obviously, like, that's not always true, but for the median student, I think it kind of is true. I become very interested when I was at Khan Academy, I was kind of thinking about this problem. One of the angles that I found really interesting was, so at Khan Academy, we were mostly thinking

• 59:55Speaker 00

  about not just the median learner, but like maybe the 25th percentile learner. One of the angles that felt most relevant, maybe not from an efficacy perspective, but for me, from like a breaking out of this, getting them to follow goals that aren't their own perspective, uh, was to focus on inquiry learning and to focus on transforming the learning experience into something that actually is related to their goals. That is, we're asking questions that uh are authentically interesting, uh, that they authentically want to answer and that they can participate in, uh, in a way that feels natural. We did a lot of experiments with Dynamic media representations of things.

• 1:00:36Speaker 00

  The idea being that like, you've probably seen, you know, maybe these like, plastic blocks or things that people can play with when they're kids to get an idea of numbers and number systems. Kids will play with these things unprompted because they're fun. Like, it's just, it's a pleasure to handle them. It's a pleasure to manipulate them. When you have them in hand, it's very natural to suggest like, ah, can you make a pattern like this? Like, why? You can't seem to make patterns like that. Uh, why is that? So, you know, for instance, you can start to point out things like,

• 1:01:16Speaker 00

  so Quezner rods is the name for um, a, a set of uh, ten rods that have basically unit length one to ten and they're all different colors. Um, and so you can do things like, take eight and like, the rod that represents eight and um, put two of the rods that represent four up next to it and show that like, this one, you can, you can like divide into two rods effectively. But then if you take seven, like, there aren't, there is no other pair of rods that, for the same color, you can put it next to it. So, so you get these different patterns. So things kind of naturally suggest themselves um, by experimenting with these materials and having conversations with people around these materials. And so, one of the things we were interested in was,

• 1:02:00Speaker 00

  well, are there things that are like that? That are for more advanced topics? Like, can we, can we create something that's kind of like those rods? But that is about, you know, like a, a more advanced topic in math? Or um, are about uh, debates in history or something like that? One of our tactics was to lean heavily on social interaction. People like talking about stuff with people if it's like a real conversation. You know, and for the same reason that I had to use less willpower to study that quantum mechanics text because you were there with me. A student who's engaged in like a real activity with a peer will need less willpower as well.

• 1:02:44Speaker 00

  They'll also learn from their peer if you structure things right. So, social learning becomes interesting. But I think at a high level, I mostly have abandoned this question to others. Basically, everyone in the educational space, this isn't totally true, but like, you know, ninety plus percent of people in the educational space are focused on really like the bottom quartile, like not even median. And there's a good reason for this, like many, not quite as many people who are in education. They are motivated by arguments of equity and opportunity. And they want everybody to have the opportunities they had.

• 1:03:25Speaker 00

  They, they're very motivated by the injustices that they see in the differing access and the differing support that different people have. And they're very motivated by the, the very real disadvantages that accrue to the bottom quartile performing students. It's also true that the, the marginal impact that you'll have on that student's life will be much greater probably than the marginal impact on say an eightieth percentile performing student. Or so the argument goes, like that student will be fine, which is like probably true.

• 1:03:57Dwarkesh Patel?

  But there's a, there's a big marginal difference between fine and, you know, supercharged.

• 1:04:03Speaker 00

  Yeah, yeah, I mean, that's true, but anyway, I mean, like, I, I say all this to say like, I, I understand why the vast majority of people in education are, are focused on what they're focused on. And I think it is good and I'm glad they're doing it and I'm, I'm mostly have decided to let them do that and to focus elsewhere.

• 1:04:21Dwarkesh Patel?

  Yeah, no, I, I see tremendous value in focusing on the, realistically, like the cool new shit that's coming out, you know, where's that coming from? And what's the way to increase that?

• 1:04:32Speaker 00

  Yeah.

• 1:04:32Dwarkesh Patel?

  That, okay, so but it's interesting to know that the same tools might not just work across the spectrum.

• 1:04:36Speaker 00

  Yeah, I also say like part of the trouble here is that like.

• 1:04:41Speaker 00

  The cool shit is very likely to come from students who are like performing at the 20th percentile in school because they're like disaffected and bored and none of this stuff matters to them. Right? And so like, part of the trouble here is that by like opting out of helping these people learn, there are all kinds of interesting inventions that could probably occur that aren't occurring. So I don't quite know how to contend with that. I guess basically I'm like trying to bite off a piece of a problem that feels maybe tractable.

• 1:05:12Dwarkesh Patel?

  Once, you know, all the tools are built, you know, when you're at the end of your career, would the, would the learning process, is it supposed to feel fun? Or does it have to feel fun? Is there an element of even when all the tools are there, that there's just like a level of David Goggins? You know, this is gonna be miserable, but I've decided to learn this in this way and I just had to go through it.

• 1:05:34Speaker 00

  Where does misery come from?

• 1:05:37Dwarkesh Patel?

  Hmm, where does it come from?

• 1:05:39Speaker 00

  I guess I'm asking this honestly, not really rhetorically. Let me try to answer my own question. I mean, so let me say first off, like, I think I am broadly speaking very opposed to what I understand to be David Goggins-esque attitudes towards almost anything. In this particular instance, I think what I think is something like, if I ask, why is it miserable to learn a particular subject? The answers that come to mind are things like, first off, I don't care about this subject. And I think that's not what we're talking about. Like, you're asking about a world in which these great tools exist and someone's using one of these tools to try to do something they really care about. So another reason why it could be miserable, that I think is pretty common, is that you have some idea about like,

• 1:06:21Speaker 00

  you're not going fast enough, or like you're failing, or you're struggling. And the misery comes from resisting that. It comes from feeling like you're, you're doing poorly and like you shouldn't be doing poorly, like it's bad that you're doing poorly.

• 1:06:35Speaker 00

  And maybe you're feeling fearful, like others are gonna judge you, or like you don't have enough time, or something like that. And I think that's basically like an emotional problem that needs to get healed, rather than like a practical problem with learning. In the case of something like organic chemistry. Where like you truly do just need to learn like, you know, 200 names or something. One answer is that like, okay, that can be done very cheaply using modern memory systems, actually. So like organic chemistry students suffer through this and they don't need to. But even with modern memory systems, like you're probably going to spend a total of, I don't know, call it 100 minutes across some weeks, uh, start studying all of these, um, formula.

• 1:07:19Speaker 00

  And that still is unpleasant. So can that be resolved? And I think the answer is yes, actually. So, um, I was thinking about this in the context of the cell biology by the numbers book I was telling you about, where there's all of these, um, uh, things like the, the volume of a nucleotide, I said, is a nanoliter. So to, to like study the flash card, what's the volume of a nucleotide is like not terribly pleasant. I'm not sure it constitutes suffering exactly. It's fine. Uh, you know, I'll do it while like waiting in line. Um, but I think there is a better version of that, which is like, uh, solving an interesting Fermi problem, which involves that term. So something like, if I have a vial of the COVID vaccine,

• 1:08:02Speaker 00

  like how many copies of the COVID RNA are likely to actually be in it? If the vial is a milliliter large or something, that's like kind of a fun little question. Uh, and I can enjoy sitting and noodling on that. And in doing so, I will need to retrieve the, the volume of the nucleotide to help me make that approximation. So I think there's moves like that, that you can use to kind of paper over any remaining stuff that feels kind of necessarily unpleasant or rote.

• 1:08:30Dwarkesh Patel?

  I'm actually surprised to hear you say that because one way in which I read your stuff is, really some of your stuff is that this is actually a way of endorsing the traditional way of thinking about education, but using new tools to get the traditional ends. To give you an example of what I'm talking about, you know, you go back to like a headmaster from the 1900s and you say, is, is it important to have the taxonomy of a subject memorized? You say, of course it is. That's why you're going to spend a year memorizing taxonomy. And then you would say, you know, memorizing shit is actually important so that you have a dictionary by which to proceed on the subject. So in those ways, but you have a sort of, you know, new systems for doing that same kind of thing. So I'm not just surprised to hear you say on this particular, and the reason in this particular case,

• 1:09:12Dwarkesh Patel?

  I was expecting you to say that, no, that you have to be disciplined if you decide to learn something. Is I I I expected that, you know, in the case of the three hours of intense learning, followed by an intense piano session, you were just like really tired at the end and you're like, but no, I've this is something I have to do this evening. So, um, yes, I'm actually kind of surprised to hear you say that.

• 1:09:33Speaker 00

  Yeah, yeah, no, I I really enjoy this tension. Um, and I I'm probably overstating my, I'm probably like reacting to the Goggins reference.

• 1:09:42Dwarkesh Patel?

  Right.

• 1:09:42Speaker 00

  Like, like a bit of an over extreme over correction or something. Um, but but this really is how I feel. And and I feel this tension all the time. Like, the the histories in educational psychology that I'm most aligned with are like the most robotic, like authoritarian kind of histories. And also, the ones that are most like kind of unschooling and and Montessori-esque, like, I really have a ton of sympathy for for elements of of both of these directions. And there's kind of a weird synthesis of this in my head that I can't, I guess, fully externalize. I guess part of what I'm saying is, is aspirational. Like, I mean, it certainly is the case that I do in practice, uh, uh, use willpower to

• 1:10:24Speaker 00

  make things happen. So, I mean, just as an example of of something that's totally contrary to everything I was saying, I use a tool called BeeMinder, which charges me if I don't do certain things. Right? This sounds, I don't know if it's, you know, kind of military, but it it's certainly more authoritarian than this kind of freewheeling butterflies kind of gesture I was making a moment ago. And I use it to make sure that I do my memory practice. Shouldn't my memory practice be so joyful? It's at the center of my research, right? It should be like the most interesting, exciting part of my day. But often it's not. And so I use this to do it anyway. So there's some tension here. I think I I do want to say, you know, the the reason why I'm willing to endorse this headmaster's view about the

• 1:11:06Speaker 00

  taxonomy has to do with the price. I did a bunch of memorization in high school, and it was very inefficient, and it was very uncertain. So it was like it was emotionally difficult, because I like I wouldn't even feel confident that I had learned this stuff. I didn't know what it was to learn something reliably, like to be confident that I'd be able to recall it. And it also like, it was hugely time consuming. Because I didn't have techniques or tools. And now, you know, part of why I respond so favorably to like, yeah, just learn the taxonomy is that like, for me, it's just trivial. Like, yeah, sure, whatever, throw it in the deck, you know, like, it'll, it'll consume a total of 15 minutes over the next few weeks and then I'll know it. Uh, you know, it just doesn't cost anything. So like, yeah, okay, fine.

• 1:11:48Speaker 00

  Um, other things in, in learning do still have real costs and, um, those, those are maybe more difficult to negotiate.

• 1:11:57Dwarkesh Patel?

  Actually, this is a good place maybe to ask you about unschooling and your attitude towards, like, just let me ask, I think somebody on Twitter had this question, which is, your kids, as they're growing up, how are you, how are you structuring their education?

• 1:12:10Speaker 00

  Well, okay, so to, to be clear, I don't, I don't have kids.

• 1:12:13Dwarkesh Patel?

  Right. Hypothetical kids.

• 1:12:16Speaker 00

  And so, yeah, so you, you're gonna hear the, the foolish response of, you know, like a, a person talking about what one would do hypothetically. This is very difficult. So, the school, of course, has many purposes other than instructional, right? Like, it, it has a social purpose, it has a societal purpose, it has a behavioral purpose, and it also has like a pragmatic purpose of basically babysitting. Those things can be unbundled. I, I think it's pretty interesting to consider that. If I actually did have a kid, I would probably consider that project pretty thoroughly. I think it's like, pretty likely that some kind of, um, homeschooling situation would occur. It probably wouldn't be me being the teacher, but it would probably be the people I would hire.

• 1:12:59Speaker 00

  I have some resources, like, I'm not wealthy, but I have some resources, like, that is, uh, it may be a difference. But during the pandemic, I was struck by, um, Brian Tobal started a, a company, which is now defunct, and so this is a fun example to bring up, but it's called Schoolhouse. And, uh, the idea was that he noticed that, um, people were getting together in pods, right? That was the thing we did during the pandemic. And in particular, they got together in pods with like, their classmates from school, maybe five or six kids, and some of these pods started hiring elementary school teachers who were not working because of the pandemic. And the elementary school teachers would like, come to the backyard of one of these people's houses, and the five or six kids would get together with the elementary school teacher and like, they do stuff all day.

• 1:13:42Speaker 00

  Buying this one teacher's time split five or six ways was actually really very tractable. You know, say you want to pay the person fifty dollars an hour, maybe that seems reasonable for a teacher. This is not that hard to do and actually cost less, substantially less than a private school. I think Schoolhouse costs something like a fifth or whatever, the cost of an elementary school. Once you got to older grades, you need maybe specialists. It's actually not clear if you do. My friend Alec Resnick is working on a very interesting school called Powderhouse in Somerville, Massachusetts, that does something like the model I just described, where you have adults who are in more of like a coaching role. And they aren't necessarily domain specialists, but

• 1:14:26Speaker 00

  they'll connect people with domain specialists. So anyway, I would explore something like that model. I'm sorry, this is a little bit vague. If you want to ask about specifics.

• 1:14:34Dwarkesh Patel?

  Yeah, sure. Let me ask, so you got a twelve, this child grows up and is twelve. So at this point, you know, it's not just like, of course, you've taught them arithmetic and reading and everything. Are there, do you proceed and you have to learn your biology, you have to learn chemistry, or do you just say, what are you interested in? Are you interested in Roman history? Oh, let's learn about the aqueducts. Or is there an actual curriculum that proceeds until they get to college?

• 1:14:59Speaker 00

  Yeah, this is really challenging. So one of the sort of the heroes of the reform school movement is this philosopher named John Dewey. And he has a lovely book called Experience in Education, sort of written near at the end of his time, looking back on all of his efforts to reform schooling in a kind of unschooling-ish direction. He was never as extreme as that, but broadly looking for freedom on the child's part. And he makes this wonderful argument that because these kids, a twelve-year-old, doesn't have a fully developed prefrontal cortex, certainly doesn't have a fully developed

• 1:15:40Speaker 00

  kind of sense of self, to let them do whatever it is that their whim

• 1:15:49Speaker 00

  Commands them to do in any given moment is actually not freedom, but rather is chaining them to whatever that impulse is. It makes them the the subject of these tides of impulse. And I think that's a pretty compelling argument. It doesn't authorize tyranny. But it also suggests that, you know, you gotta be a little bit skeptical about the the the planning or the plans of 12 year olds, I guess. How skeptical should bone be? I don't know. I think I would probably have stronger opinions on that if I had a 12 year old. But my instinct as a foolish non parent would be something like a kind of mix. I would be interested in exposing the 12 year old to lots of topics and possibilities.

• 1:16:33Speaker 00

  I would be voluble in expressing the consequences of any particular actions. Like if they just want to compose music all day, we could talk about like, well, what does that mean? What kind of life does that look like? I would try to be non coercive in this as much as is possible. And I think to some extent, the the student should or the the child should be allowed to feel the consequences of of their choices. This is complicated by the fact that like, you know, I again, like, I'm not wealthy, but like, any any child of mine would like have chances, I guess, you know? Like if they made some weird choice about a career path when they're 13 and so they didn't get into Harvard

• 1:17:17Speaker 00

  or whatever, like, that would be OK. You know, like, they could do. They could be 24 and finally figure it out then, or 32 and finally figure it out then. Like, it would probably turn out fine. And so this this doesn't seem like reliable guidance. You should notice, I'm feeling very confused about.

• 1:17:36Dwarkesh Patel?

  Yeah, no worries. Yeah, OK. So one question I have is historically, and maybe even to the modern day, it seems like improving education has been a very intractable problem. And you did reference this earlier when we were talking about gearing towards the median student versus the whatever percentile you're working with. But I don't know. But do you feel like there's been progress even in the percentile you're gearing your stuff towards? And if not, what is the explanation for the relative stasis? I mean, this is something you've talked about. We have so many new tools with IT. Where is the well, what explains the like, the broader sort of stagnation here?

• 1:18:10Speaker 00

  Well, the fun answer to your question is actually there's been a ton of progress. Uh, like, actually, things are like pretty good. One thing is, I think the stat is, uh, in 1900, um, 6% of teenagers graduated high school in the US. Now, that doesn't mean that 94% like didn't have, you know, an education that we would regard as a high school education. But it like, kinda means that, it roughly means that. Now, these people are homeschooled. It's also the case that a high school education, uh, meant something lesser than. Uh, a substantial fraction of, of high schoolers now study AP courses and, uh, complete them in, in high school. And that's at the high end.

• 1:18:49Speaker 00

  On the low end, illiteracy, uh, was a, a very live situation 100 years ago in, in the US. Uh, and is emphatically not now. Um, now, it is the case that something like, uh, 10 to 15% of adults, depending on which, which polls you use, are maybe, would, would struggle to perform like simple kinds of, um, number manipulation or, um, reading or writing kind of tasks. But, um, our bar has basically moved. It used to be like, you know, can you read at all? Uh, and these tasks are like maybe a little artificial, like they're maybe not relevant to their day to day. And that's actually why they're experiencing this.

• 1:19:30Speaker 00

  So, the, the, the number of people, the, the fraction of the population who, who graduates, uh, you know, at 17 or something, uh, knowing a, a particular amount of stuff, has basically moved up monotonically. And this is mostly about the bottom, like, portion of, of, of the, the population. It used to be the majority, uh, were effectively uneducated past, you know, age 10 or something, other than informally, and in their trade. And, um, really, the story of the 20th century has, has been in part one of mass education. Uh, where, part of why we have a service economy, IT economy, is that, uh, basically all of our population is educated at a particular level.

• 1:20:14Speaker 00

  If you look at, there are these, you know, national tests of 4th, 8th and 12th grade math and, um, language proficiency, you'll see really like pretty slow movement in, in the 75th percentile and like...

• 1:20:30Speaker 00

  Practically none at all in recent decades. But you'll see like absolutely enormous movement in the bottom quartile. And so in some sense, the story, especially the last 20, 30 years, has been closing what's often called like the performance of the achievement gap, where certain groups, part of underfunded schools, or who might have households that are unsupportive or difficult. You know, we're just not having anything like the educational attainment of their peers. And that story has changed.

• 1:21:02Dwarkesh Patel?

  One thing I'm curious about is every other part of the distribution has been moved upwards. Has the, has the ceiling been raised significantly?

• 1:21:09Speaker 00

  Yeah, well, it depends on what we mean by the ceiling.

• 1:21:14Dwarkesh Patel?

  Because you can go back like hundreds of years and the most learned people around, it's just incredible. You look back on how many books Thomas Jefferson read.

• 1:21:23Speaker 00

  Yeah.

• 1:21:23Dwarkesh Patel?

  There's some story where Kennedy had a bunch of Nobel laureates, who was hosting a bunch of Nobel laureates in the White House in 1963 or something. And he says, "This is the greatest collection of genius and insight and wisdom that has been collected into this room ever since the time that Thomas Jefferson dined alone."

• 1:21:46Speaker 00

  Right. I think it's very hard to raise the ceiling. So the ceiling has aristocratic tutors. The ceiling has whatever family dynamics and heritable propensities produce tremendous intellectual greatness. Early 20th century schools produced von Neumann.

• 1:22:08Dwarkesh Patel?

  Right.

• 1:22:09Speaker 00

  And it's certainly not at all clear that they are now producing more von Neumanns or something like that. In fact, von Neumann's production seemed to have probably very little to do with any kind of mass schooling that we would recognize. As far as the very top, I think that's, that's difficult. We're talking about An institution that was created for, I guess, the masses. I guess that there have always been people who have been using resources outside of those kinds of systems. So the mass system doesn't seem to help those people, I guess. That doesn't seem surprising.

• 1:22:46Dwarkesh Patel?

  By the way, on the Von Neumann thing, okay, mass system doesn't help him. What is the production function for Von Neumann?

• 1:22:53Speaker 00

  Yeah, so lots of people have studied this. I actually am not a student of Von Neumann's history. I know that many of his peers, the 20th century greats, got something like aristocratic tutoring or came from small Eastern European incredible schools, that there's stories about these things. I actually don't know them, I'm sorry.

• 1:23:14Dwarkesh Patel?

  I mean, you actually referred about that one high school.

• 1:23:16Speaker 00

  Yes, yes.

• 1:23:18Dwarkesh Patel?

  Okay, interesting. Are we getting worse at the Von Neumann production, or is it just static?

• 1:23:26Speaker 00

  Well, maybe. I mean, so I don't know. So let's see. Here's a theory that seems kind of plausible. If someone was going to have aristocratic tutors in the late 19th century, would they now go to a fancy private school? And would that experience now actually be less good for them? I don't know. I think it's probably more likely that they'd go to the fancy private school and also still have fancy tutors and then go to a very exclusive university where they're going to get a bunch of highly hands-on kind of interaction with professors.

• 1:23:58Dwarkesh Patel?

  Although the reason that might not be the case is the opportunity cost for people who might become teachers or aristocratic tutors is much higher now, whereas the kind of person who would be, you know, your tutor can now directly be making lots of money on Silicon Valley or Wall Street.

• 1:24:13Speaker 00

  That's interesting. Okay, so that would be an argument that maybe it's not so much about the 20th century that we've gotten worse about this, but more like over history, you know, maybe Aristotle was a tutor to Alexander the Great, and now Aristotle would be like a full professor and would maybe take that job. That might be so. I mean, I think it's, it may be the case that some tutors have been priced out of the market, but it's not clear to me that the most expensive tutors. Actually would be the best. There is a bunch of empirical research on tutoring. And like, one of the questions they ask is like, what kind of experience level do the tutors need to have? And it's interesting, like, how far you get in tutoring efficacy when the tutor doesn't necessarily know anything.

• 1:24:58Speaker 00

  So like, just having like, another warm body there actually contributes a very large effect. No, I mean, things get better as you get an expert. And I also, I have a kind of healthy skepticism of these studies. Like, I think part of the role of having Aristotle as a tutor is communicating a worldview. It's not something that would show up on a test or something that these studies would be measuring. So having an extremely inspiring individual might actually be the important component. And inspiring is going to be highly correlated with expensive, I think. Not necessarily, I don't know. That feels complicated.

• 1:25:32Dwarkesh Patel?

  I mean, especially today, the material is available. What the tutor is bringing is the inspiration and the motivation. Not the exclusively, but one of the large parts of their.

• 1:25:41Speaker 00

  That's right. They're not really responsible for instruction. I mean, I'll say also, like, I know lots of people who have postdoc tutors right now. You know, these people, as graduate students, they're very pleased often to have, you know, a $60 an hour, say, tutoring kind of commission. And that's a little sad, but, you know, the pool of available postdocs to hire as tutors, I think, is very large now compared to how it would have been 100 years ago. The pool being bigger doesn't mean that the top, you know, 1% are getting more, though. So I think that's undecided. There is a question of, like, have teachers gotten better at their jobs, like, over the

• 1:26:26Speaker 00

  last 50 years, say? And like, there are some ways in which maybe they have. There's been a bunch of projects of trying to disseminate, you know, certain research results, like, ways of instruction that are more effective than other ways. Like, it's good to interleave stuff, for instance, like, rather than doing blocked units where it's like, okay, like, we're going to talk about the Civil War, and then we're going to talk about women's suffrage. You know, it's better, that this is somewhat far apart, but it's better to kind of weave these things into each other, and not just in history, but in general. So that kind of dissemination has been happening more systematically in the last few decades. I actually, I'm unaware of any kind of studies or results trying to establish anything about. The efficacy of teachers now versus long ago.

• 1:27:10Dwarkesh Patel?

  Well, I'm sure you've seen the the claim that one of the consequences of the very unfair circumstance of the 20th century or the mid 20th century was that one of the very few occupations an intelligent woman could pursue was teaching. And now that other options are available, which is obviously on that hugely hugely good, you know, there's there's there's other competition for those same very intelligent women.

• 1:27:34Speaker 00

  Oh, that's interesting. I I haven't heard that claim. Yeah, I think I'd have to I'd have to think about it.

• 1:27:40Speaker 00

  I guess I I it's not clear to me how much intelligence matters, like if you want to think of that as like some some kind of separable quantity.

• 1:27:48Dwarkesh Patel?

  Or whatever trade is relevant to just that you you just had a population that was hostage to either housework or teaching.

• 1:27:55Speaker 00

  I guess what I'm saying is something like if that were true and there are like a bunch of people who are now, you know, astrophysicists or something, you know, it's not clear to me actually that they would have been the good teachers, like being a good teacher is is often about empathy and effective communication and care.

• 1:28:15Speaker 00

  It's it's very personal. It's very intimate. Like, you need to understand the subject, but to teach a 15 year old or something, you actually don't need to understand it at a like a postgraduate level necessarily. It's it's very interesting to see that there's a bunch of studies of kind of the impact of domain knowledge on teaching efficacy. I've read some in math. I'm sure they exist in in all fields. And one of the things that comes up is like, if you aren't very familiar or comfortable in math, then you will struggle specifically to do like inquiry oriented classes, classes that are more about like creative ways of thinking with math or open ended problems

• 1:28:58Speaker 00

  as opposed to like, here's how to do this algorithm. Because to conduct those kinds of classes, you have to be able to think on your feet. Like, you pose a difficult question to which there may not be just one appropriate answer. And your students will throw all kinds of stuff at you. And you have to be able to take that stuff and integrate it and show how one student's answer relates to another student's answer and show how those conceptions can be built upon in order to, you know, produce some useful understanding for what you had in mind. Anyway, this this kind of improvisation requires a mathematical familiarity and ability, but I don't think it requires anything like. An extraordinary ability.

• 1:29:38Dwarkesh Patel?

  Yeah, yeah, yeah. But more than the extraordinary have been pulled out of teaching as a consequence.

• 1:29:43Speaker 00

  Yeah, I guess I'm just wondering what the correlation is. Like, if it's the case that actually effective teaching is mostly about empathy, then maybe it's actually correlated. Like, the people who are going to be good anti uh particle physicists are actually like, they wouldn't have made good teachers anyway.

• 1:29:57Dwarkesh Patel?

  Interesting.

• 1:29:58Speaker 00

  Maybe.

• 1:30:00Dwarkesh Patel?

  Why hasn't hypertext changed how people write more? You know, so often, you know, I write a blog post and I actually do wonder how much different it is with the knowledge that I can add footnotes and I can link to things. But you just hope that, you know, I'm actually kind of a fan of how Wikipedia organizes content. It is genuinely surprising how often the best explanation of a subject is just this like resource that is trying to explain every single subject. Because I think there's just this practice of you don't need to do exposition on every single topic, you can just hide it behind links and things like that. Anyway, so why hasn't hypertext changed writing more, online writing at least?

• 1:30:34Speaker 00

  This is a really good question. So I think the reason why Wikipedia works as well as it does is that encyclopedia entries are already forced to stand on their own. And that was true before hypertext existed. In fact, encyclopedias were already hypertextish before there was hypertext. There are some other interesting kinds of hypertext that existed pre-computers. There is this very interesting book called the Syntopicon from Adler that

• 1:31:10Speaker 00

  if you want to understand what classical authors had to say about a topic like the father's responsibility to a daughter, you can look that up in the Syntopicon and you will get references across Rousseau through the Bible and so on and so forth. And those are kind of hyperlinks. I mean, they were printed on on dead trees. But you're expected to, you know, get the books down, look up the appropriate pages. The Syntopicon wasn't that successful. I think it's in part because those concepts, unlike the Wikipedia entries, they don't quite stand on their own so cleanly. You kind of need sinews, you need linkages. And actually, I want to make the case that while Wikipedia is an astounding resource,

• 1:31:51Speaker 00

  I find it rarely to be the best available introduction or explanation of a topic. I find it often to be like a good jumping off point. Like it'll help me know the right thing to ask about. It's good as a reference. So hypertext is a very effective navigational aid. It can help you get to a spot that you're looking for very quickly because it's about automating flipping through pages. And so for a reference, it's very effective. If what you have is like a table of chemical compounds and their properties, like a book starring of chemical compounds and their properties, hypertext is going to let you navigate that book very effectively. Likewise, dictionaries have been revolutionized by hypertext. So navigating around thesauruses by clicking on links to say like, oh, shaded a little

• 1:32:36Speaker 00

  bit more like that, it's like a much better thesaurus. So I guess I'm making the case that like, there are certain kinds of text that are more amenable to hypertext because they are more amenable to having the reader dropped in the middle of them. Encyclopedias are like that, dictionaries are like that. Most text is not like that. And I guess like most concepts are not like that. I guess most ideas are embedded in something kind of holistic or richer. They require a narrative arc. They're difficult excerpts. Not everything, but things that are not so raw and atomically informational. So there were all these dreams of hypertext novels, for instance, and some people wrote them.

• 1:33:19Speaker 00

  And one of the problems that a hypertext novel has is actually could be seen in a choose your own adventure book that existed before there was digital hypertext. And that's that the author is forced to write something like a lowest common denominator story. The page that is the destination of a hyperlink, it has to work as the endpoint of all of its reference. And so it can't establish.

• 1:33:45Speaker 00

  Any kind of coherent or consistent arc unless there's a kind of sameness to all of the reference. And the more that there's sameness to the reference, like the less useful hypertext is. So a lot of people have been disappointed by this conclusion, I among them. I'll say that I do find hypertext very useful in my own notes, not really for reading. I actually don't think it makes for a very good reading experience for others.

• 1:34:14Dwarkesh Patel?

  But having, being a reader, you have a separate webpage where you have your working notes for the audience, by the way. It actually is like a very cool UI and a format to explore your thoughts.

• 1:34:24Speaker 00

  Thanks. It does an interesting thing for me as a writer. It lets me build stuff up over time. So today I was working on, I was reading this, this very old cognitive psychology paper on the topic of adjunct questions, which we discussed earlier, the effects of asking questions while you read. Not on remembering the information covered in the questions, but actually just on kind of the general effect that it has on, on stuff that isn't touched by the questions. I have some notes on the design decisions of the mnemonic medium, this quantum country thing that I was talking about earlier, interleaving the, interleaving the questions into the text. And those notes are kind of partial, they evolve over time.

• 1:35:08Speaker 00

  What was the impact of doing this? My, my notes about that, they've come from interviews with readers. They expand when I read a paper like this that's relevant to them. And it means that when I go to design the next system and I'm thinking about the, the role of questions in text, I'll have sort of a place to look. The role of hypertext in this is roughly as a kind of navigational aid.

• 1:35:37Speaker 00

  It's possible to do this without hypertext. You just end up with like what Luman had, you know, a giant, something like a dresser, but made of card files rather than drawers for clothing.

• 1:35:50Dwarkesh Patel?

  This actually goes back nicely to the original conversation we had about why people like Tyler are able to integrate so much information without an explicit note-taking system. And in fact, I just remembered, another person who comes to my mind immediately when I think about a person like this is Bernhard Hohberg. And again, you have an example of somebody who is extremely prolific, a daily finance newsletter with, just like a tremendously detailed and insightful daily financial newsletter. So it's like, it is like a daily note-taking practice in some sense.

• 1:36:19Speaker 00

  Nothing quite accumulates for either of them, at least not in the same way.

• 1:36:23Dwarkesh Patel?

  Right.

• 1:36:24Speaker 00

  It's very interesting, like they're, they're doing the whole thing over again every day. One thing I find kind of interesting about Matt Levine's newsletter is that when he's talking about a topic repeatedly, like something that comes up, like the, the recent bank collapse or something, he will have to explain some concept like interest rate risk over and over and over again for days. Like every day he has to explain it, but every day he explains it anew. And every day the explanation is like colored a little bit by that day. And this is an argument against the kind of note-taking that I do. It's an argument for ephemerality, for, for like recreating the thing every day because it will change and it will become kind of inflected by, by what you're thinking about now and your experiences.

• 1:37:09Speaker 00

  I think it's pretty interesting. I, I find myself these days doing kind of mix. Like I have, you know, kind of like a journal that's about today and I'll do a bunch of writing and often I'm recapitulating stuff I've written before. And I have these other things that are trying to be more durable, be like a, a useful reference that can stand outside of time.

• 1:37:32Speaker 00

  The combination feels useful. I don't yet have like a clear model of like when one is better than the other.

• 1:37:37Dwarkesh Patel?

  Well, actually, an interesting way to tie in what you just said with the hypertext is. You, I, I, it was at Burns newsletter is, I, doesn't give that much context on, you know, often you'll find yourself lost about what, you know, what is the concept being talked about here if you're not familiar with the, um, the topic. And in fact, I asked him at some point, uh, have you considered doing narrations of your blog posts? Like Scott Alexander has somebody who has a podcast where they narrate his blog posts. And he said, I don't think it would work out as well for mine because I heavily rely on the old blogosphere's norms around, uh, hypertext. Where you can add jokes and sarcasm based on, um, one example of this is he was talking, he had a write-up about SPF and his collapse.

• 1:38:22Dwarkesh Patel?

  And he said, you know, he had a bunch of links, like, if you want to learn more about margin calls, read this, you want to learn more about this. And he goes, and if you want to learn more about the psychology of utilitarian bets, read this, and there's a link to the Amazon page of Crime and Punishment. So just that kind of stuff is harder to do.

• 1:38:39Speaker 00

  Yeah, you're right. So he's leaning more on his past explanations, which is interesting because he can't update them. Like that format of writing a newsletter and then linking to past newsletters, or as you say, the, um, this is sort of a former blogosphere thing to do. You have a series of six words and like each word is a link to a previous post from the topic. I certainly have written stuff like that. It's kind of funny. I mean, it's, it's approximating the durable note thing I was writing about, um, but without the ability to revise it over time. And, and maybe for many topics, like you don't need that ability. It's certainly the case that, well, I wonder now what fraction of, of my notes are basically in the state they were. The, they're, when I did my first major revision of them. It's probably at least a third. It might be more than a half.

• 1:39:24Dwarkesh Patel?

  What person's journal notes have you published?

• 1:39:27Speaker 00

  I don't know. You know, by word count, by note. So like, for instance, my journal notes are not published and there's one of those every day. So there's a lot of them. So if we're looking by note, we're excluding all of those. I also have like a note about all of the people in my life. And those are for the most part not public.

• 1:39:46Dwarkesh Patel?

  Right.

• 1:39:46Speaker 00

  Unless they're public individuals, you know? And so there's a lot of notes that are not public, but they're mostly not durable. They're like, they wouldn't be all that meaningful to others. The journals might be, but they're also intimate.

• 1:40:01Dwarkesh Patel?

  Are they written in a way that would be intelligible to if you were to give that to somebody else?

• 1:40:05Speaker 00

  It depends. But usually, actually, my journals are like complete sentences, complete paragraphs, usually. Sometimes bullets, sometimes kind of veering and breaking and changing to new subjects suddenly. But they tend to be filled with links to the things that I'm talking about, in part because I'm trying to accumulate context in those things.

• 1:40:27Dwarkesh Patel?

  How come they're, why not just shorthand?

• 1:40:31Speaker 00

  It's partially because past me is another person. It's kind of a cliche, you know? But like, I am routinely looking at journal entries from a year ago.

• 1:40:39Dwarkesh Patel?

  Yeah.

• 1:40:40Speaker 00

  This is partially like, you could view that as a failure of this note writing system. Like, maybe in some ideal sense, I shouldn't be looking at these journal entries because if something's important and it's going to be something I refer to a year later, it should be in some durable evergreen note. I don't know. Like, you don't, you don't always want to do that. It feels like, feels like prepping. Maybe there's an amount of prepping that's good. We live in California and like, maybe everybody should have like an earthquake kick. Right? Like, maybe that's good. But maybe you don't need to like hoard 300 cans of beans. So there's like an amount of prepping that feels like a reasonable amount to do. There's an amount that feels kind of dutiful and unpleasant.

• 1:41:22Dwarkesh Patel?

  As a researcher who is in these Silicon Valley circles, what, what is your opinion on the startup advice? Of, you know, do things fast, fail fast, get to users immediately with an MVP? As, as somebody who is making products, but is also in a sort of, in a different mode than a typical startup kind of making products? How do you think about advice like that?

• 1:41:43Speaker 00

  I have complicated feelings about this. I need different advice on different days. And of course, different people need different advice on different days. But when I was getting into this kind of work, what that kind of advice led me to do, practically speaking, is to not think all that deeply about the ideas I was exploring. And to look to, basically, like, an idea would come up, and I'd think like, oh, I can try that. And then I would, I would try that. And then I'd learn something, and then I'd repeat. And there wasn't this sense of building a theory of like, what the problem is and what it would mean to solve it. Instead, it was just a theory of action. Like, a theory of action as opposed to a theory of change is,

• 1:42:26Speaker 00

  if you imagine, like, you're at some point, your, your current position, and eventually, you want to get to some goal state. A theory of action is you look around you and you say like, well, what can I do? What can I build? What do I see is possible? And a theory of change is to look at the endpoint and try to work backwards. Now, the metaphor is imperfect because in research, you actually usually don't exactly know what the endpoint is. And you certainly don't know how to work backwards. But I guess what I'm saying is that following that advice, historically, often has led me to try things that were straightforward. I think the most powerful design work has ideas in it.

• 1:43:09Speaker 00

  What makes a nonlinear text editor, that is, like, the text editors that we all know and love, so powerful is this observation that writing is a nonlinear process, but writing with a pen linearizes it. And many, many other observations like that, and on the nature of what it means to have a thinking environment, is, is how we got that particular interface. Likewise, the way that we got powerful programming environments, insofar as we have them, is by people thinking very hard about what it means to specify a system

• 1:43:52Speaker 00

  and coming up with new primitives that express those ideas.

• 1:43:58Speaker 00

  I think the most powerful interfaces are often the expression of new ideas or new primitives

• 1:44:06Speaker 00

  that capture new ways of doing, new kinds of objects that can be manipulated. In Photoshop, for instance, you can manipulate a photo by means of a construct called a layer. This is a very strange idea. It has some precedent in darkrooms where you could potentially have kind of like sheets of film. I don't mean like the negatives, sheets of like gels that you could potentially put over the lights to affect the exposure and to make there be more exposure here and less there. But in Photoshop, they're non-destructive and they're continuously manipulatable.

• 1:44:45Speaker 00

  And the layer is like, it's a new primitive that is introduced into the activity of photo editing. And it utterly changed what you could do in photo editing. So I guess what I'm saying in a very long-winded and confused way is that I think it's difficult to have ideas by means of building an MVP very rapidly. Now, if you have an idea that you think is interesting, it is good to test it rapidly. And so part of why I'm confused is, in my response here, is that it's good advice once you have something worth testing. It's just that, for me, adopting that mindset,

• 1:45:29Speaker 00

  and I've lived in it for so long that it's very ingrained in me, it makes me not sit in stillness and in confusion and in contemplation with the ideas long enough for them to be good. I mean, very concretely, so Michael Nielsen and I made this Quantum Country thing. When I was trying to think about what to do next, the most obvious or natural idea was like, well, what if we just tried that with lots of other things? And that idea occurred to me, and the pandemic had just struck, so I was feeling a little timid, I guess, creatively or emotionally. I wanted something that felt kind of safe. And I knew that I could do that. Like, okay, I can build a platform that like generalizes this thing that we did for this textbook. So I did.

• 1:46:12Speaker 00

  And I did it relatively quickly. I did it in a few months. And that wasn't the right thing to do. It wasn't really the right question to be asking. The idea... Wasn't that strong, it wasn't the right way to test it. Like building this highly general version of it, I would have been better building more one offs. Rather than like a self serve thing that anyone could use. And this comes down to like the difference in aim. Like, I'm not trying to build some kind of scalable thing for the world at this moment. I'm trying to build the idea. The prototype is an expression of the idea. And once it arrives at a good place, then maybe there can be some scalable solution.

• 1:46:50Speaker 00

  But it's not necessarily at that place. And until it's at that place, there's like a lot of thinking and sketching that goes along with the building and prototyping. I think part of my confusion here is that often I still need to hear this advice. Like often I will just tie myself in knots in theory land. And like what I really need to do is to have a friend sit me down and say like, you know, is there a piece of this that you can carve off and like build next week? And so you're hearing a lot of tension.

• 1:47:15Dwarkesh Patel?

  Interesting. And then so what was the consequence of shipping Orbit out? I guess before it felt ready to scale.

• 1:47:22Speaker 00

  I mean, I learned some things. It was fine. Like it taught me a lot about where that particular format succeeds and fails in other venues. It was just not a very effective way to find those things out. I built this like very general. It was an MVP in the sense that it has very few features and it's very simple. But it was highly general. I mean, it's like, it's a deployed thing that has infrastructure and has accounts. It has like all this stuff that you do when you're building like a real thing. And that's very different from like, well, let me like work with this one author and like see if I can make it work with this one other book. That's very different from Quantum Country to form like a specific question or a specific theory about like, well, it worked for this text.

• 1:48:06Speaker 00

  Like what's the next kind of text that would be good to test with? And then to do that, I could potentially do it. I certainly could have done it much more rapidly.

• 1:48:16Dwarkesh Patel?

  Why do you think this idea of tools of thought has nerdsniped so many people in Silicon Valley?

• 1:48:24Speaker 00

  Well, it contains this message for technologists that they can potentially be very powerful. And that's always tantalizing for people, I guess. I think it also feels very actionable for people. In a way, that's actually super misleading. I mean, so I meet tons and tons of people who tell me that they're interested in Tools for Thought, and 95-plus percent of them are engineers. And the problem with this is that, like, building an interesting tool for thought is basically entirely a design problem. And their design ideas are usually not very good or are troubled in a variety of ways. And yet, they can make a thing. They can make a thing that solves a problem, maybe, for them in their lives.

• 1:49:09Speaker 00

  And that feels very tantalizing or encouraging. It feels like something to get their hands around, I think. I think we in Silicon Valley are very, very interested in thought. We are like a thinky people. And people are very interested and engaged also with anything that could potentially expand our capacity there. And so that, too, is tantalizing. Like, what if I could think better? It's also tantalizing because it's meta. So there's all these cliches about people tinkering with their dot files endlessly or tinkering with their blog website, which has two posts on it, but they have to rewrite it because they want to do something else. And now the new one will have three posts on it before they rewrite it again. Tools for Thought also scratch that itch.

• 1:49:52Speaker 00

  It's work about the work. This sounds very cynical, by the way. I don't mean for it to be. I'm just trying to earnestly answer the question. But, okay, here's a more optimistic and generous response. I think many of us got into computing because computers portray a sense of personal empowerment and possibility. We maybe remember growing up and being locked in our bedrooms at midnight or whatever, fooling around. And we have this very powerful tool at our disposal, and it's opening up these worlds for us. And I think for many people here, that was a formative part of their personal development.

• 1:50:35Speaker 00

  And so anybody pointing to that and saying, "We can do more stuff like that," is going to be pretty compelling, I think.

• 1:50:44Dwarkesh Patel?

  Okay, this was an interesting question from Matt Clancy on Twitter. What are the characteristics of a good crowdfunded research project?

• 1:50:54Speaker 00

  One of the maybe unfortunate things that I've learned in my crowdfunding experience is that there are some dynamics that seem hard to change. So, one of them is churn rate. You know, like any subscription kind of revenue business model, I guess that's what I have. You lose subscribers every month. In my case, it's about 2%. And it's not that large, but it does mean that I need a certain number of new subscribers all the time. And one thing I've learned that's kind of interesting is that the churn rate is surprisingly insensitive to anything that I do. I, you know, I've experimented with a variety of things and it really hasn't meaningfully changed the churn rate. What does change things is getting more people into the top of the funnel. In other words, marketing.

• 1:51:39Speaker 00

  And there are some things that have maybe affected like the fraction of those people in the top of the funnel who convert or whatever. I really hate this way of thinking about it. In summary, the thing that I've discovered that's kind of sad is that I ended up having to think about this a little bit. And in particular, I realized that this project only even slightly works, the one that I'm doing, for crowdfunding, because it's understandable to others and it's interesting to others. And it's already in a place where, you know, there's some results that maybe look kind of promising, or people are like, "Oh, more like that." But it's very easy to imagine other projects that like are not broadly applicable.

• 1:52:19Speaker 00

  You know, if I were doing marine geology stuff, I probably wouldn't have a big crowd of internet people, not nearly as large anyway, who are excited. So, that's one property. This work is very general. It applies to many, many people. It applies to people who have disposable income. So, if I were doing a research project on, I don't know, like writing practices of disadvantaged artists, like, I don't know, like, I think my audience might not have as much disposable income. I have already made some progress. I think that's probably important, unfortunately. So, it's probably very difficult to use crowdfunding in the very early days of a research project. I've already sort of chosen a research agenda or direction like and I can kind of

• 1:53:01Speaker 00

  express it. So I think this says that like crowdfunding applies probably after the first few stages of research have been completed. There's probably something like the standard grant advice where like at some point here, I'm going to be using this crowdfunding like to figure out the next thing and I won't be able to explain it to anybody. And there certainly are, you know, seedlings like that, but you have to like have something in flight. I probably need to be able to say something about my progress with some kind of regularity. So for instance, my wife is working on this study of biological markers of age in association

• 1:53:41Speaker 00

  with delirium and traumatic brain injury. And to do this, she is basically signing up patients who show up to the hospital who have traumatic brain injury. And once they agree to participate in the study, taking, you know, various blood samples and things like that from them and recruiting enough patients to get like the significance that she requires will take like, you know, two years or something like this. She can report a little bit of intermediate stuff, but like certainly not a monthly update or something like that, right?

• 1:54:13Dwarkesh Patel?

  Be a weird Patreon post.

• 1:54:15Speaker 00

  Yeah, I mean, I can't quite report monthly updates either, but but I think like there's there's a cadence that's necessary.

• 1:54:21Dwarkesh Patel?

  Why bother with it at all? I'm sure there's many wealthy individuals would be happy to single handedly fund your research. Is there a reason you chose crowdfunding?

• 1:54:31Speaker 00

  Well, those wealthy individuals are very welcome to to reach out and offer to do so. You know, I will say I've been I've been fortunate to have many high net worth individuals sponsors, but I guess each of them is is, you know, providing.

• 1:54:48Speaker 00

  I guess on my Patreon, the sponsorship is a hundred dollars a month. So like, that is, you know, what I, what I get from these people. And, you know, I'm certainly not getting, you know, wild offers for more.

• 1:54:59Dwarkesh Patel?

  I think you're using the wrong tool going the distribution of your, given the wealth distribution of your.

• 1:55:04Speaker 00

  Maybe. There's a couple of ways to interpret your question. One question is like, why crowdfund when I could appeal to high net worth individuals? And another version is like, why crowdfund at all? Like, as opposed to raising grants, or talking to philanthropies, or whatever? Are you mostly focused on the first of those?

• 1:55:19Dwarkesh Patel?

  Um, yes.

• 1:55:21Speaker 00

  If I'm gonna be honest, it's, it's because it has worked. The, the history of, of the crowdfunding of this project is, like many things in my life, the result of goading from Michael Nielsen. Early on, when we were working on this Quantum Country project, he suggested we set this up. And I kind of hemmed and hawed. And I said, like, ah, you know, it's gonna be a distraction. Like, we don't, we don't really need this right now. Like, let's, let's deal with it later when we have something to show. And he's like, no, no, no. Like, let's just get it started. You know, it's gonna be a long time to get enough subscribers and so on. And it turned out, he was right. You know, the, the process of crowdfunding a project is, it takes like, maybe a couple of years, at least in my experience, to, you know, build up a subscriber base. And starting earlier was better. And if that hadn't worked, or if we hadn't started early, I think I probably

• 1:56:06Speaker 00

  would have just reached out and asked for individual help. And I probably will if it fails on me. I'll say also, like, when there have been specific projects that I've wanted to do, that require, say, hiring people, I have reached out to high net worth friends and they've helped. But, you know, in kind of like, the low five-figure, four-figure kind of range. And that's great. And I'm very grateful. So I guess the answer may be a mix. One of the big limitations to the crowdfunding thing is. It seems pretty clear to me, it can't sustain a team or institution or anything like that. It can barely sustain me. I'm somewhere between like a grad student and a junior faculty member, you know.

• 1:56:51Speaker 00

  And like, that's kind of okay, I guess. And there's like a variety of reasons why that's okay for me that are like pretty particular to my circumstance, but you know, it certainly wouldn't be okay for everybody. And even for me, it doesn't allow me to support others.

• 1:57:06Dwarkesh Patel?

  Right. And it's even more, I guess, striking because in terms of the success of what a public intellectual is basically, in some sense, you're like a public researcher. In the sense of like some of your research is public, or you publish your research in a public facing way. Even in a context in which you're like pretty well known, especially amongst the, you know, the kind of audience would be happy to fund this kind of thing. If the LeBron James of, you know, independent public research is like between a grad student and yeah, it's not a great sign in general for that.

• 1:57:38Speaker 00

  I think it's worth considering that I'm also maybe not very good at this. Like, I mean, so first off, like, I'm not that successful as a researcher. Like, I guess I kind of object to the LeBron James characterization. It's true that I'm maybe the most successful crowdfunded researcher in tech stuff, and that's kind of weird. But like, the last couple of years, you know, I've like figured some stuff out, but I guess I wouldn't say I've had any like spectacular hit kind of publication kind of things. One thing that is true of this is that, you know, when I have big publications, I get a lot of subscribers. So I think like, there is some kind of market force that could be higher if I were, you know, like having a more spectacular success or whatever with my research. I think it's also true that I pretty systematically avoid marketing it.

• 1:58:23Speaker 00

  That's kind of a self-protection thing. Like, I am really worried about the corrosive influence of audience and marketing on inquiry, honest inquiry.

• 1:58:35Dwarkesh Patel?

  Okay.

• 1:58:37Speaker 00

  It is very easy, very, very easy to distort my work. It's almost a default to try to make it be something that people would be more likely to like, rather than the thing that I actually want to investigate. Or to do the boring, simple version of it, rather than the interesting, deep version, so that I can publish more stuff more often. Like, one thing that I've chosen not to do, that is a choice that's definitely cost me financially, is to publish what academics would call like, minimum viable units of paper, or something like that. They have a pithier phrase than that. Minimum viable

• 1:59:16Speaker 00

  papers. It's very common to take, you know, any new marginal insight that is kind of above a particular bar, and publish that as a little thing. And I just haven't done that. Like, you know, I've written little informal letters to my patrons, like, "Hey, I figured this thing out this month." And like, if I were an academic, I probably would have published that as a paper. And if I were a marketing-oriented, crowdfunded researcher, I probably would have, like, done some glossy thing, and like, promoted it, whatever, like, "Look at this thing I figured out." But like, actually, I just don't think it's that big a deal, and I'd rather get on to the next thing. I have that choice, I guess, of waiting to publish. But, and that's not really what I'm worried about. Really what I'm worried

• 1:59:58Speaker 00

  about is like, marketing, man, marketing. It makes it so hard to be honest with oneself, at least in my experience. Not only, to be honest, as I said, with like, what I think is interesting and what I think is important, but even to be honest about the results. Like, every paper is, in some sense, a little marketing piece, trying to make the case that it's significant, that its results are really exciting, are really important. And that is really corrosive to discovery. I mean, it's true that you need to, you need a really strong emotional connection to the work, I think, in order to do good work.

• 2:00:35Speaker 00

  And part of that emotional connection comes from a sense of excitement, of maybe being hot on the tail of something really good. But there's a temptation to kind of portray what you've found in the best possible light, and to kind of downplay its limitations, and to take up space, and to totalize. And all of this is just...

• 2:00:57Speaker 00

  I think, I think it's just death for discovery.

• 2:00:59Dwarkesh Patel?

  It is interesting to hear that from somebody who inadvertently and without intentionally trying to do so, has done a good job of spreading your material. You know, I've, I've known about you for a long time, so. But I do wonder if there's an element of, I think if you get to a certain level of quality, after that, trying to market your stuff not only doesn't help, but probably hurts you. If you can try to think of somebody like, I don't know, Goren trying to like post YouTube shorts of his blog posts or something like that. It would just be like, what are you doing, man? Right? Where's like, it's just so good that he doesn't need to promote it.

• 2:01:31Speaker 00

  Goren is an interesting example because there's a simpler failure mode. And that's that I still routinely run into people who will tell me like, oh, I've really liked your work for a while. I didn't know you had a Patreon. And that's kind of like a simple failure of a certain kind of marketing on my part. And I think Goren actually has this even worse. I adore Goren, like I have learned so much from him. And it is the case, like you can go to his Patreon page and he actually makes public his revenue. He makes on Patreon like a tiny fraction of what I do. I think this is inappropriate, like Goren is a much more impactful researcher than I am. And he has a much bigger audience than I do.

• 2:02:13Speaker 00

  So the fact that they aren't converting into patrons, I think, is mostly a matter of like the way that he talks about it and the way that he presents it. And like, it's not that he needs to market more people to his webpage. I expect he has plenty of traffic and plenty of large audience. I think it's much larger than mine. I think it's more just like, there are a bunch of variables about the way that you talk about this membership offering. And none of us really want to think about them. And I've ended up at a slightly more effective part of the space, but I'm pretty sure that there's like much more effective ways to do whatever it is I'm doing.

• 2:02:48Dwarkesh Patel?

  Yeah, this is a really interesting problem because I have a sub stack where if people choose, they can help contribute to the podcast. And while it's broad enough revenue to help pay for certain episodes and traveling, in comparison to now that I'm going to be doing ads, in comparison to while we're making an ads, it's like a small fraction. And which is, some people might say it's unfortunate that you have to do ads and I

• 2:03:15Dwarkesh Patel?

  maybe listeners will just be finding out for the first time that there was an option on Substack. But also, you don't want to be in the position where you're asking listeners for money every episode, right?

• 2:03:25Speaker 00

  Yeah, I hate asking people for money. I think this is a common issue for creative people. I hate it. I really hate it. I probably need to get over this. I do want to make one point, though. I had much more success with my Patreon when I recast it as like, oh, please like support me, like subscribe to support my work. Like, like the thing you were describing to something that I guess feels slightly more like, I don't know, an offering, like become a member. And like, when you become a member, like, these things will happen. We're like, these things are not terribly substantial necessarily. But I guess what I'm saying is like, there's a difference between a tip jar and a membership in people's minds.

• 2:04:06Speaker 00

  And like, becoming a member means something. And if you could offer something small that feels membership-ish, you might get very different results. And Guren has the kind of tip jar vibe. And I, these days have kind of like a member vibe. And my instinct is that if he were to move to like a, you know, become like a member of Guren's lab kind of thing, he would have better results.

• 2:04:30Dwarkesh Patel?

  Well, he has a thing on Patreon where it's like, if you donate five bucks or eight bucks, he'll read an entire book and review it?

• 2:04:37Speaker 00

  Yeah, this is crazy. I don't know if anybody's ever taken him up on this.

• 2:04:40Dwarkesh Patel?

  Yeah, but I mean, that's like valuing his time at a dollar.

• 2:04:43Speaker 00

  Yeah, I don't, I don't quite understand this. I mean, I think it's also the case, like, you'd probably have an easier time asking for subscriptions if you had a larger audience first, like, you can build the audience for free and then kind of have some bonus offering or something that's behind a wall, maybe. I feel very conflicted about this, actually. Maybe you can help me think about it. Well, I just, I have all these patron essays. It's like where most of my writing is these days, because I'm kind of waiting until I can collect enough things for the next big public piece. I have a couple of like big public pieces in various stages of flight. And so anyway, I mean, I'm writing a lot for patrons.

• 2:05:26Speaker 00

  And I think probably much of my audience or people out there like don't even know that's there. So like one challenge of member only content is even making clear that it's there to others. And often people will try to achieve this by like tweeting about or sending newsletters out about this subscriber only content. And I just can't bring myself to do it. It feels terrible to say like, oh, here's a link, but you can't view it. Yeah, I can't, I can't do it. I don't know how you think about this, or if you think about like subscriber only material for Lunar Society.

• 2:05:59Dwarkesh Patel?

  I was actually just about to mention this to you, which was, you know, I'm a patron and I got a chance to read all your patron only, patron only essays. And they're great. And I was actually thinking while I was reading them, it's like really unfortunate that a person might not know they exist, or if they're not familiar enough with their work to go ahead and sign up, just like behind the Patreon. So it's a shame that the way to fund public work is, or one of the ways to fund public work is to make some of that work less public. I think it would be, yeah, if there was some way to make this.

• 2:06:31Speaker 00

  I think there are better ways to do this. I think there are like design solutions. So for instance, like if it were the case that my work was kind of mostly all in one place rather than in these separate places, and the subset of the work that's public was kind of visually and structurally adjacent to the subset of the work that's private, it would be clear that there's like this additional stuff that's available. And perhaps you can see the first bit of it, you know, Substack has this kind of stuff as to get some sense of what it is that you'd be seeing. I just, I've invested like zero effort into figuring out an appropriate presentation of this stuff.

• 2:07:07Dwarkesh Patel?

  Right. And then also another thing to consider is to the extent that this may not be the largest thing you care about, but it is a factor, is that a big part of the impact of at least your writing work, well, one of the things in that equation is how many people will actually consume it.

• 2:07:21Speaker 00

  Yeah.

• 2:07:21Dwarkesh Patel?

  And the expected value on that is dominated by the probability it goes viral.

• 2:07:26Speaker 00

  Sure.

• 2:07:27Dwarkesh Patel?

  Uh, and it just can't go viral if it's on. Like, for example, I think your recent post on, you had this really insightful post on, based on your experience in industry at Apple, about the possibilities of a Vision Pro, and in what ways it's living up to it not. And I think like, that would have just-

• 2:07:43Speaker 00

  Oh, thanks. I mean, I, I, I did make it public. I put it on Twitter and it, it was on the front page of Hacker News.

• 2:07:48Dwarkesh Patel?

  Alright, okay.

• 2:07:49Speaker 00

  I, I, I think you're, you're right. And like, usually I don't want this stuff to go viral. Like, it's in media's race. It's, uh, I, I think that the primary value that most of it has for people is kind of opening up a window into a, a particular, very unusual kind of creative work that they don't normally get to see, the, the behind the scenes of. And most of it is kind of context laden. It's, it's not really freestanding, and I don't really wanna write it, uh, as if it could be freestanding. I've occasionally had the experience of one of these kinds of things getting widely distributed and then getting all these comments of people like, just like, being kind of angrily confused about what I'm even talking about. That's kind of discouraging. I gue- I guess all, all of this to say, when I wanna write something for broad public

• 2:08:34Speaker 00

  consumption, I, I write something for broad public consumption, you know?

• 2:08:37Dwarkesh Patel?

  Okay, I've got some questions for Twitter, from Twitter.

• 2:08:39Speaker 00

  Okay, bring it on Twitter.

• 2:08:40Dwarkesh Patel?

  This is actually another question from Matt Clancy. Are there other examples of beneficial knowledge work practices that perhaps mostly work because they are a form of spaced repetition practice, where the participants don't realize it?

• 2:08:52Speaker 00

  Yeah, I mean, I, I think this is like, uh, this is embedded in our, in our, in our working world.

• 2:09:00Speaker 00

  So, for a researcher, when you need to write papers regularly and you're writing those background sections, and you're, you're repeatedly explaining, uh, the, the history of a particular line of research and citing the appropriate sources, like, that is a kind of spaced repetition. When you have students and you're mentoring them in conversation about like, oh, in this kind of situation, you really need to remember to do X, um, that is a kind of spaced repetition. And all this stuff is, it's kind of accidental. Doctors have grand rounds when, you know, even when they're, they're not seeing patients regularly, they're still exposed to, uh, to other patients. And there's often a structure in this where, like, while the- The patient is being presented, you're supposed to be kind of trying to think like,

• 2:09:44Speaker 00

  you know, what would I think to ask? Like, what would my differential be? Before you hear it, there's like covert retrieval happening. So I think it's like, it's everywhere in our world. And it's spaced and it's, it's repeated. The thing that differentiates the kind of formal practice that I've been exploring, I think is mostly, it focuses on material that you wouldn't otherwise normally have repeated, either because you're too early with it to have a consistent practice, or because it's just not firmly tethered enough in anything in your life.

• 2:10:16Dwarkesh Patel?

  This is a question from Ian Banagast. What is the optimal amount of effort that should go into a personal website?

• 2:10:25Dwarkesh Patel?

  And I think you might have noticed the amount of CSS that exists on Tusha.org, which is very beautiful, but.

• 2:10:32Speaker 00

  I don't like it. This is what everybody says about their website, right? You know, it's three years old, that means I want to redesign it, but I will not allow myself because it feels like a distraction. What's the right amount of effort? You know, I mean, there's no general answer to that question. Of course, that's going to be my answer, but what can I say about it? What's the job of the website? What's it trying to do? I think many, especially engineers, do themselves a disservice by fretting over their websites unnecessarily, building vast technical infrastructure when really what they want is like a place to post some markdown files. And they're better off just like getting a ghost installation and going to. I think the main thing to think about is like, what is it that you want to put out in the world?

• 2:11:17Speaker 00

  What is the ideal form of that thing? And to try to find some way of organizing and expressing that. We have these common patterns, like a blog or a portfolio. And often people end up kind of forcing themselves into these patterns. People will end up using blogging software to make something that's kind of durable. And I think like very interesting, personal websites often come from people who are thinking about that question, kind of the shape of the thing that they want to put out into the world and making something that speaks to it. Often, once you understand the shape, making the thing is not that effortful. My website was not an enormous project for me, probably.

• 2:12:00Speaker 00

  Should have been a slightly larger one, given that my income depends on people coming through it.

• 2:12:04Dwarkesh Patel?

  The working notes with the, I mean.

• 2:12:06Speaker 00

  That was a weekend.

• 2:12:07Dwarkesh Patel?

  Really?

• 2:12:08Speaker 00

  Yeah, I feel kind of bad about it because it's made its way into tons of commercial projects now. And people are like, oh, this is like, this is the way to present network notes. And like, I was actually, I think it's not very good in a variety of ways. I spent like a couple of days on it.

• 2:12:20Dwarkesh Patel?

  Wow, because I thought this is where the question was alluding to is you must have spent months on this.

• 2:12:25Speaker 00

  Nope.

• 2:12:28Speaker 00

  I mean, it is a little bit of like the, you know, the thing about the mechanic, like, you know, hitting one thing and kind of knowing the thing like, okay, I have design intuitions that led me in a particular direction. But there's lots of things I don't like about it. I just haven't allowed myself to spend any more time on it because I just don't think it's important enough.

• 2:12:44Dwarkesh Patel?

  I have a question about actually your time at Apple before I ask the final Twitter question. You know, like, like everybody have iPhone and from the outside, there just must be so many different trade offs and constraints. When a thing like this is being designed, you know, what is the supply of certain components and the cost? What do different consumers want? What, what features is the R and D team ready to put forward? And then at your time at Apple, you were like responsible for a lot of these cornerstone design features. What like, how is all that information integrated where a guy is like, all right, taking all of these constraints into account? This is the design, like, how does that happen?

• 2:13:22Speaker 00

  I mean, one thing that's very interesting is that it's very compartmentalized and like basically none of what you just said was relevant to me. Like, it was all pre-specified. So like the thing at Apple is, is like, you have a little domain that's like your own. And like the boundaries of that domain are determined by like everybody else's little domain. And so like, there's a person who's responsible for thermals. Actually, there's a team that's responsible for thermals and they kind of figure out like, you know, okay, like, how can? I guess like, what is our thermal budget? Like, what, how much can we have the CPU on during what kinds of working situations? And I basically can't argue with that. Like, those are just my constraints.

• 2:14:01Dwarkesh Patel?

  But, but, but aren't those constraints informed by

• 2:14:04Speaker 00

  Oh, yeah.

• 2:14:04Dwarkesh Patel?

  differential problem?

• 2:14:05Speaker 00

  It is iterative. So we'll run into stuff where like, oh, there's a thing we really want to do. We can't pull it off because like it drains power too much.

• 2:14:13Dwarkesh Patel?

  Right.

• 2:14:13Speaker 00

  So hay Siri is an interesting example. To be able to activate a voice command at any time without interacting with the device is great. And people prototype that just like having a thing listening in the background and like watching for it. But that requires having the, the main CPU on all the time, like processing audio buffers. And like, you simply can't do that. It drains the battery. And so that attempt led to eventually having this dedicated co-processor that runs at a lower power. And it's very limited and restricted. And it can be on when the main CPU is not on and it can listen for that sound.

• 2:14:51Dwarkesh Patel?

  Hmm. So is there a person whose job it is to take all things into account? And like, I decided, given, given the memos from everybody that thermals, you guys need to work on this. You know, you guys work on this.

• 2:15:03Speaker 00

  Um, not exactly. It's a little more push and pulls. So like a given team, usually some of their priorities will be internally determined. Like the thermals team has its hobby horses and it knows what it thinks is important. And some of them will be externally determined. There is an executive team that makes ultimate decisions about like, you know, the main priorities for next year's devices or whatever. Like, ah, next year, we're, we're going to do this like face ID thing to unlock the phone. And we're not going to have a home button. Like, okay, as soon as you like, if you want to not have the home button and you want to have the screen go edge to edge, like this has all of these impacts, like top to bottom on the device. So that decision creates like lots of necessary work for lots of teams. But some stuff is kind of handled at a, I guess, a more local level.

• 2:15:48Speaker 00

  So for instance,

• 2:15:51Speaker 00

  More locally to the iOS team rather than at a top level executive team, the director of iOS apps might decide like, we have this problem that because the apps were built at the same time as the system frameworks, we end up building our apps using this like weird Frankenstein, like partially internal framework, partially the public one that our developers use. And the internal one is always like a little bit different and like it's not always maintained reliably. And so we have all these problems about the skew between the two. So like a big priority for us is going to be to like, you know, rewrite all the pieces of our apps to only use the public bits and like, you know, so that they could be distributed on the App Store. And that's kind of like a more local decision.

• 2:16:36Dwarkesh Patel?

  What I find really interesting about this is that it's possible for a $2 trillion company to integrate all this information to have some a cohesive hierarchy where so many different products, so many different trade offs are being made. Does that make you think that over time, these very well functioning tech firms will get bigger and bigger? That they can actually handle the cost of having this much overhead?

• 2:17:02Speaker 00

  Let me first just respond to this observation about the enormity of the company and then maybe we'll talk about the other firms. I think the reason Apple is able to do this is because of the way that they delegate. So while there is a very strong command and control structure and for important decisions that they really are made by a small group of people at the top,

• 2:17:24Speaker 00

  the individual leaders in the various areas at all levels of the hierarchy have an enormous amount of latitude and that's the only way that any of this can work. So individual people are given very, very strong responsibility and authority within domains to make decisions.

• 2:17:41Speaker 00

  And that's how you can have all of these disparate products. Like Craig Federighi is head of software at Apple. What does that mean? How can you be head of software? Like, they have, how many platforms do they have? You know, iOS, iPadOS, watchOS, VisionOS, macOS? Also, there's like an operating system running in a bunch of the cables, like on little chips in the cables, right? And like, all of that is under Craig. What does that mean? You know, like in practice, what it means is there is a set of software concerns that he's actually super concerned with and he's thinking about day to day. Like, when I was at Apple, I had Craig Federighi in my office, like, talking about gesture recognizer heuristics with me, because like, that was something that was hyper salient to him. At the same time, he was basically completely ignoring,

• 2:18:24Speaker 00

  you know, 95% of software-related decisions. And he just fully delegated those things to others. There's a really interesting Harvard Business Review piece from a few years back about Apple's management structure and about how they have a couple of different concentric rings of kinds of responsibility for any given leader. There will be, I don't remember exactly the breakdown, you know, call it 5% of things that you're responsible for that you have your hands on at all times and you are like directly manipulating, controlling. And then there's, you know, a ring outside of that that's a little bit bigger. Those are the things that you're keeping an eye on, right? So like, they are salient to you. You're getting reports on them. You are checking in on them. You are thinking about them. You're coming up with ideas and sending them down the chain, but you're not directly controlling them.

• 2:19:08Speaker 00

  And then there's a bunch of stuff that you've figured out how to delegate and you want to hear if there's problems. And they talk about how that structure's evolved over time. It's been now eight years since I've been at Apple and so I'm sure it's practically unrecognizable to me.

• 2:19:25Dwarkesh Patel?

  This is a question from Basil Halperin on Twitter. Is the lack of spaced repetition adoption a market failure? Or is the lack of adoption efficient?

• 2:19:35Speaker 00

  I think it's probably mostly efficient. So in places where spaced repetition, as it stands, without substantial, like, novel cultural knowledge that's difficult to transmit and isolate. Where all of that is valuable, we see a lot of spaced repetition usage. So among medical students who are highly motivated, have lots of reason to study, and the material is shaped in a way that's highly amenable to spaced repetition usage, there's tons of spaced repetition usage. In fact, the med student Anki subreddit is like bigger than the Anki subreddit. Likewise, among language learners, spaced repetition in various forms is extremely common. Duolingo has spaced repetition integrated into it.

• 2:20:17Speaker 00

  And spaced repetition is kind of naturally present in the process of immersion learning. In fact, modern spaced repetition tools, between the Lightner Box and Wasniak SuperMemo, they were both originally motivated by language learning. So in language learning, there's like a substantial market for spaced repetition. It could probably be used in a variety of more creative ways. For instance, Russell Simmons has pointed out to me that studying individual vocabulary words on flashcards often misses kind of integrative opportunities. What you really want is to kind of study lots of sentences or something like that, or possibly to build up towards that. Duolingo does something kind of like that. People in spaced repetition for language learning subreddits mostly don't.

• 2:21:01Speaker 00

  Some of them do, it's kind of complicated. So there's edges of the market, right? Where you need early adopters to kind of try things that have rough edges. And the early adopters, sometimes they get cut and they bleed a little bit. And so that's why people aren't rushing into it. As to why spaced repetition isn't widely used, for instance, to like learn quantum physics, I think it's basically correctly priced. You know, I can use spaced repetition to learn quantum physics a bit faster. It doesn't make it a fait accompli or anything like that. It's not like learning anatomy, where basically if you study the deck, you'll be done. You need some more stuff, and I'm working on some of that stuff. And also, you need like an incredible amount of very unusual knowledge that's largely tacit at the moment in order to use it in that way.

• 2:21:45Speaker 00

  That's part of what motivated recording this other video is, is to kind of show some of that in action. So the fact that the market isn't acting on this thing that it kind of can't really act on seems pretty appropriate.

• 2:21:56Dwarkesh Patel?

  That's, I think, a good place to tie off that other collaboration and taggo on this project. This is really interesting.

• 2:22:04Speaker 00

  Thank you so much.

• 2:22:05Dwarkesh Patel?

  This was many hours of just insights and lots of food for thought.

• 2:22:10Speaker 00

  Wonderful, thank you.

• 2:22:11Dwarkesh Patel?

  Yeah, things are coming on. Hey everybody. I hope you enjoyed that episode. As always, the most helpful thing you can do is to share the podcast. Send it to people you think might enjoy it, put it in Twitter, your group chats, et cetera. Just blitz the world. Appreciate your listening. I'll see you next time. Cheers.