Ep. 380: ChatGPT is Not Alive!

There has been a lot of loose talk online recently about the capabilities of existing AI tools. In this episode, Cal reacts to a specific recent clip from the Joe Rogan podcast in which the guest argues that language models are like a child’s brain, and may already be conscious. Cal puts on his (always stylish) computer scientist had to explain why this cannot be true. He then answers listener questions and reacts to feedback on his recent episode about using a notebook to enhance long thinking. Below are the questions covered in today's episode (with their timestamps). Get your questions answered by Cal! Here’s the link: bit.ly/3U3sTvo Video from today’s episode: youtube.com/calnewportmedia Deep Dive: ChatGPT is Not Alive [0:04] Are there other forms of dangerous AI other than LLMs? [49:03] What are the immediate concerns of AI? [53:24] Will AI be smart enough to evolve to not destroy itself? [55:47] Does James Somers make a valid argument in his recent New Yorker article that the AI can think? [1:01:49] How can I build career capital if I’m forced to integrate AI into my workflow? [1:04:52] CASE STUDY: A Workplace Productivity System [1:06:31] CAL READS THE COMMENTS: Regaining Your Ability to Think (in 60 Minutes a Week) [1:10:48] Links: Buy Cal’s latest book, “Slow Productivity” at calnewport.com/slowGet a signed copy of Cal’s “Slow Productivity” at peoplesbooktakoma.com/event/cal-newport/Cal’s monthly book directory: bramses.notion.site/059db2641def4a88988b4d2cee4657ba?youtube.com/watch?v=gXbsq5nVmT0youtube.com/watch?v=NnA2OoH_NFY Thanks to our Sponsors: calderalab.com/deepauraframes.com (Use code “DEEPQUESTIONS”)byloftie.com (Use code “DEEP20”)cozyearth.com (Use code “DEEP”) Thanks to Jesse Miller for production, Jay Kerstens for the intro music, and Mark Miles for mastering. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:05Cal Newport?

  A couple of weeks ago, the biologist Brett Weinstein went on Joe Rogan's podcast. Their conversation turned, as it so often does these days, to the topic of AI. But Weinstein goes on a monologue pretty early in the episode about AI, and it's a monologue that begins as follows. I want to play you a clip here from the start of his conversation.

• 0:29Speaker 01

  Entity. So you'll hear people say, well, it's not, it's not really thinking, right? It's just figuring out if it was thinking, what the next word in the sentence is garbage.

• 0:41Cal Newport?

  In some sense, right? Weinstein is right in how he starts the conversation, right? He's saying, look, we can't, just saying, uh, language models predict the next word. Uh, that's gar, that's too dismissive. So while it's true that, yeah, they literally do predict the next word, there is a lot of impressive understanding and processing that goes into actually figuring out what word to, to, to produce. In fact, writing The New Yorker recently, James Somers argues that we can even think of the processing that goes into predicting next words and language models as thinking. And I think he lays out a good argument for that. Uh, so Weinstein, he starts, I think I'm on board with him. But then, as he continues in his discussion, he begins to argue that not only

• 1:22Cal Newport?

  is AI doing impressive processing, but is rapidly evolving beyond our ability to control, and that it is, quote, like, five minutes, end quote, away from starting to manipulate us. Weinstein implies that existing LLMs might already be conscious and that we have no real way of testing whether or not this is true. It's exactly this type of conversational turn where we shift from what's actually impressive about language models to impressive stories about sinister things these models might be doing. That is the turn that I want to push on back on in today's episode. I think it's just become. Really common in conversations about AI today to to jump from real things about these

• 2:07Cal Newport?

  models to fake things. And in doing so, I think it's distracting us from real problems, real issues that we really do need to be facing with this technology. So I want to get into all this today. What I'm going to do in the episode is I'm going to walk through some more of Weinstein's points. I'm going to play some more audio from that discussion. We're going to start on common ground and then we're going to start, he's going to start to deviate from what I think is true. And I'm going to put on my computer scientist hat and argue why what he is saying is deviating from what we actually know to be true about these machines. All right. And then I'm going to turn to an actual computer scientist, Jeffrey Hinton, who seems to have been saying sort of similar things to what we're going to hear. And I'm going to deconstruct what he's saying. If he actually knows why these machines work, why is he saying something similar to Weinstein? I'm going to argue

• 2:52Cal Newport?

  actually, he's not. We're going to be more careful about deconstructing what actual computer scientists are saying. All right. So AI is interesting and scary enough on its own that I don't think we need extra doses of made up concerns. Today, I want to help set the record straight about what AI can and can't do. As always, I'm Cal Newport, and this is Deep Questions.

• 3:26Cal Newport?

  Today's episode, Separating AI Fact From Fiction.

• 3:34Cal Newport?

  All right. So I want to return to Brett Weinstein's discussion of AI on the Joe Rogan Podcast. I'm going to play a few more of his clips, and then I want to react to them and give my computer scientist view. We'll start on some common ground, and then we're going to begin to deviate. Our two understand this technology will deviate as we get farther along in these clips.

• 3:53Cal Newport?

  And before we get into this, I want to emphasize that I don't mean this episode as a way of being dismissive or hostile to Weinstein. His general argument is actually very common. There's a lot of otherwise very well-informed people who are making similar arguments. The reason why I'm using his audio is actually that I think he's a very clear speaker. So he's very clear and careful about what he says, which I think makes it easier to discuss and react to. So I actually, it's because he's so careful in talking that I think he's a good foil for this discussion. All right, so let's get back into it. The conversation that he had with Rogan began with a clip I played for you in the intro where he's saying, hey, we can't just see these things as next word predicting machines. That's just garbage.

• 4:35Cal Newport?

  And I sort of agreed, like, yes, in doing that, they're doing much more. And that's kind of sounds dismissive to say they're just predicting next tokens. Now I want to play the next part of the conversation, what Weinstein said next.

• 4:49Speaker 01

  No way. Okay. What we actually have is something so analogous to a child that that is the right model. In other words, when a baby is born, it has no language. It may have some structures that language will slot into, but it doesn't have any language. It is exposed to tons of language in its environment. It notices patterns, right? Not consciously notices, but it notices them in some regard, you know, that every time somebody says the word "door," you know, there's a fair fraction of those times that somebody, you know, opens that portal in the wall. I wonder if "door" and that portal in the wall are connected, whatever it is.

• 5:34Speaker 01

  So the point is, a child goes in a matter of a few years from not being able to make a single articulate noise to being able to speak in sentences, make requests, to talk about abstract things. That is an LLM.

• 5:55Cal Newport?

  All right, so there's some things about what Weinstein just said there that I loosely align with. In particular, he's focusing on the functionality of these models, and he's saying, "The way they behave reminds me of the ways that children behave," right? So that you, you train these models, and they build up something like a real understanding of the linguistic concepts that you expose it to, right? So babies do this by being exposed to words that people are saying, and they connect them to concepts in the real world, and they develop a concept, "Oh, this is what a door is," and then I can see all these other different things or examples of doors. LLMs do something similar functionally.

• 6:39Cal Newport?

  You sort of expose them to a lot of language that includes various descriptions of doors, and they learn more generally what that concept is. If we look underneath the covers, this has to do with the semantic embedding of the tokens in some sort of semantic space, and they basically find a location in this multi-dimensional space that all these different ways you might talk about doors all cluster around. So it has some sort of concept of abstractly what a door is. So as long as we're talking about how these things appear to function, I say, "Okay, that's, you know, maybe a little flowery, but that's like a reasonable, I think that's reasonable." But then he continues, and this is where we're going to begin to deviate between my understanding of these models and how Weinstein describes them. So Jesse, let's play the next clip from the conversation.

• 7:24Speaker 01

  It is being exposed to a training data set, which is the world of people talking around it. It is running little experiments, and it is discovering what it should say if it wants certain things to happen, etc. That's an LLM.

• 7:37Cal Newport?

  All right, now Weinstein has lost me. He has moved from describing how these appear to behave on the outside, to try and describe what's happening inside the LLM, and the way he's, what he's describing is not how a language model actually operates. Language models don't run experiments in the way a human mind might to help it better figure out what's going, what to do or to see what will happen. LLMs certainly don't want anything to happen. They have no values or drives like a human brain does. So here's what we need to do here to make this conversation productive going forward. We need to separate what LLM language models can do. From how they do it.

• 8:19Cal Newport?

  This is where I want to start making a key distinction. And this is where I think Weinstein's mixing things up. So what can language models do? Well, there's a lot of impressive things that language models can do. They internalize the following things. An ability to embed words into a much more abstract semantic space. Like, so they have a notion of like what a door is and different ways of describing a door. It understands to be the same thing. They have a facility with languages, be them sort of like spoken human languages or programming languages. They're very good at sort of figuring out, given these words, what are things that could come next in that language in a way that like makes sense with the structure of that language? They have a great ability to recognize patterns at various levels of abstraction. Like, it can figure out, oh, this text that I'm extending here with a token is a joke and I'm at the punchline of a joke, right?

• 9:04Cal Newport?

  So there's all sorts of different types of patterns they could recognize in text. They have the ability to apply a lot of pre-wired logics and helping to figure out which of possible next words to apply. So if they're figuring out like, oh, this is a punchline of a joke and there's a bunch of words that might grammatically make sense here. They might have a pre-wired logic for humor. This word is funnier than using that word. They also have a bunch of knowledge that they can draw from that is not in your prompt, but has been internalized during training that they can draw from. These are all things that the models can do, which are very impressive. But let's now separate that from how they actually do it. So what actually happens when you open up the black box of a language model

• 9:47Cal Newport?

  and watch how the underlying data and algorithms execute? Because I think this is important. I'm going to emphasize why as we go on here. All right, so here's what you see when you actually open up the box of a language model. There is a vast table of numbers that are often called parameters. These define a series of layers, each of which is a mix of something called a transformer and a neural network, roughly speaking. There's some other things like some embeddings early on as well. But it's a series of layers that are, you can imagine them as being arranged in a sequence. And so they're defined by numbers with this giant table of numbers that just defines. The structure is always the same for these things. But what changes is like how we wire them up internally.

• 10:32Cal Newport?

  And that's what those numbers define. So we have a vast table of numbers. The input to a language model is a sequence of numbers that are each representing the words of the input that is trying to expand with a particular new word. I'm being loose here, it's actually like an embedding of the underlying tokens in a semantic space that happens sort of separately, but just think of it as like a bunch of numbers that represents the text that's being implemented. These numbers, this sequence of numbers is going to be passed through those layers defined by the table of values sequentially. It goes through this layer, the output is in the input to the next layer, the output of that is in the input to the layer after that. What do I actually mean by pass-through? Well, the actual algorithm that is running this, that's moving things through these layers, mainly what it's doing is multiplying numbers.

• 11:16Cal Newport?

  So the way these various transformers and neural networks work is you can actually take a vector of values and a table that describes the structure of one of these layers, and you can do matrix multiplication. Actually, roughly, is the main operation in simulating information moving through these layers. This is why GPUs are so critical to the AI revolution. GPUs are graphical processing units, are specialized chips that are wired to do one thing fast: multiply matrices. And why? Because this is how you do high-end 3D graphics. You represent all of these 3D shapes with a bunch of points in space, and then you multiply them by each other to do transformations like to rotate things or to apply light to them.

• 12:00Cal Newport?

  And so, video games require these chips to do nothing but the matrix multiplications you need to do to do high-end 3D graphics. And the AI revolution said, "Oh, we're also multiplying matrices, so let's take these chips that were made for video games, and they'll allow us to calculate moving through these layers much faster than if we just had like a traditional computer processor doing it." The output of the final layer then gets mapped onto a single word or part of word, and this is the output of the language model for that input, the word for which is trying to expand it.

• 12:31Cal Newport?

  Critically, once one of these networks is trained, that vast table of numbers is fixed, it's static, it does not change. Like GPT-4, GPT-5, they're each defined by a static table of numbers that defines like what happens in their wiring. They were trained sometime in the past, and once the training is over, the training is done. And this algorithm that, you know, that boosts things in the layers is very straightforward, right? It's just doing a lot of multiplication. Right? Now, what's actually captured, what's actually captured in those numbers could be all of these things we talked about before, like a lot of knowledge that was learned and logics and pattern recognizers. All of that is captured in those numbers. But the operation of applying

• 13:15Cal Newport?

  that knowledge to produce the next word is very, it's static and sequential, it's very straightforward. We are systematically moving a vector of numbers through these layers by multiplying matrices. Boom, boom, boom, boom, boom, boom, answer. All right? Complicated information is being pulled upon, but the computation itself is very predictable and static. This is very different than how a human brain thinks. And it's different than the type of thinking that Weinstein just summarized before in the clip I played. In particular, there's no spontaneous experiments in a language model. Static numbers, it does nothing until you call it and say, generate a token, and then through the layers, there's

• 13:59Cal Newport?

  the token. There's no learning, once it's live, once you've trained it, once you're using these models, there's no learning. It's just the same numbers every time. It doesn't have an idea of learning the impact of its actions in the world like Weinstein talked about. All it is trained to do during the training is to minimize loss on token prediction, right? So it's trained on real text, you cut out a word, you see how close that mechanical process comes to producing the right word, you adjust all the numbers so that in the future, its answer will be slightly closer, you do that a couple trillion times, and you get a lot of interesting knowledge encoded in those numbers, that if there's any want of these machines, it's just in training minimizing loss, that's it. Mathematically, we're minimizing.

• 14:40Cal Newport?

  There's no notion of, as Weinstein talks about it, these machines learn how to get what they want, or the impact of what they do upon the world. There is no world, there is no impact. It produces a number, a word, they adjust it so that word is closer to the right word, and then once it's live, it's completely static and non-spontaneous. All it can do is mechanically transform an input into an output. There's no evaluation of what's good, there's no evaluation of what's bad, there's no adjustments on the fly. Once these things are deployed,

• 15:09Cal Newport?

  the thinking that happens in producing the token could be very complex, but the process of the thinking is very simple. So what Weinstein is talking about is actually the way a human brain works. But LLMs are way more static and sequential and simpler to describe their operation than a human brain. All right, let's keep going. I want to play the next thing, the next clip of Weinstein, where things get even more, in my opinion, outrageous.

• 15:33Speaker 01

  At some point, we know that that baby becomes a conscious creature. We don't know when that is. We don't even know precisely what we mean. But that is our relationship to the AI. Is the AI conscious? I don't know. If it's not now, it will be.

• 15:58Cal Newport?

  I can answer your question. Is the AI conscious? No, it is not conscious. And we know this because, again, how these things operate matter. So I just described what the AI, and he's talking very clearly about language models here. Right now, the AI we have right now, he's talking about language models, not a thing in the future. We know what a language model is once it's deployed. It's a vast table of numbers and an algorithm that marches a vector through those numbers by doing matrix multiplications and telling the other end it gets numbers that it transforms into another word.

• 16:34Cal Newport?

  Regardless of the complexity of the information and logics that is represented by those numbers, that static process where nothing changes, it's non-spontaneous, it just moves sequentially through layers and produces a word on the other end, does not and is incapable of matching any reasonable definition of consciousness. I mean, to make things even worse, there is no even single machine where your language model is running. Because these tables of numbers are static, what really what you have is vast data centers of chips. And each chip is specializing on maybe just like one of the layers of this model. And it's like simultaneously handling all sorts of unrelated queries. Because all it's doing is like matrix multiplications.

• 17:18Cal Newport?

  And so your query is getting outsourced to many different chips that are all like mixing in your multiplications, interlinked with other multiplications and sharing these values back and forth. There's never even a place where the language model is all in one memory or some sort of unified being that could even be conscious. It's a because it's static, so we just have thousands of copies of different pieces of this. And it's all, it's all spread out. So how do we get consciousness then? Well, we don't know exactly what causes human consciousness, but we do know that it seems to depend on a bunch of things. All right, so if we look at human consciousness and writing about human consciousness, we know that the human brain involves many different systems that do different things and are interconnected in complicated ways. We know that the human brain, and this seems to be key to consciousness,

• 18:01Cal Newport?

  has dynamic, ongoing computation. The human brain is constantly surveying its environment. It has an internal state that it can update. It plans, it simulates various futures based on what it sees around it and what it's learned before. It has a value system and drives to evaluate what's good or bad. It uses those simulations, those values to come up with different actions. It takes actions, it learns from that action, then updates all of its prior knowledge or understanding of how the world works. All of this type of computation, which, which happens in many different types of systems that are connected together, requires updatable and accurate models of the world. It requires the ability to plan and simulate the future. It requires drives, values and motivations to help evaluate plans. It requires memories and more generally, the ability to learn and to adapt and experiences in real time.

• 18:46Cal Newport?

  You put all those functions together and something like human consciousness seems to arise. A language model, as we just described, is vastly simpler and vastly more constrained and has almost none of those features. Once we understand how they work. We see, oh, it has none of that stuff. It is static once it's trained. There's a simple mathematical process that even though it draws on complicated information that's been stored in these vast tables of numbers, the process is linear and static. Nothing gets updated, nothing is learned, there's no model of the world, there's no planning, there's no values, there's no drive, there's no ongoing dynamic computation or spontaneous experiments. Here's a vector, multiply, multiply, multiply, multiply, multiply, the word cat.

• 19:30Cal Newport?

  And this is spread out over hundreds of GPUs, they're doing lots of other stuff at the same time. When we understand how language models work, we can respect the fluency of the results they produce, but the operation matters. And we look at that operation, we say this has a fraction of the types of operations and behaviors you would need to even imagine something like the conscious, or even the sort of manipulating, hard to control types of minds that we would have to worry about.

• 19:57Cal Newport?

  I think a good analogy is maybe to think about the language model like the language processing center of the human brain. We actually have a couple of these centers, but we can imagine kind of combining them, like there's a massive neurons in our brain that is very good at, you describe words to it and it can, it has understanding of the words and it can map them into a more conceptual space. And then that, it can send that understanding to like all the other parts of the brains that then do stuff with it. Right? So we can think of a language model as like you're taking the language processing part of the brain and putting it in a vat. And maybe we wire up the input neurons to a machine so we can kind of like expose it to different words. That mass of neurons will do like a really good job of like, okay, I know what this word means and this is similar to that word and I'm going to fire the neuron that corresponds

• 20:42Cal Newport?

  to like the general concept of a door. It does complicated processing, those neurons have complicated information wired into them. But we would never look at that language processing center in a vat and say that thing is a human, that thing is alive, that thing is conscious. Like, oh no, you have to hook that up, these 20 other parts of the brain that all function in this complicated way before we have human consciousness, just take that part out. Yeah, it's, it's really sophisticated, it's understanding of language, but that's all it does is understand language. That's not consciousness by itself. And yet that's kind of the way we're arguing about language models. There's a bigger principle at play here that I want to emphasize that I think we need to keep in mind when talking about AI more generally.

• 21:25Cal Newport?

  System design matters. The way a system is wired together, the algorithms that you run on the system, that determines what that system can and cannot do. These details are important. When a computer scientist looks at a language model, they look at specifically how they work. And a computer scientist can then say, alright, this is very different than how a brain works. It does just a fraction of the different things that a brain does. It's static and it's sequential. This is very good at producing words fluently, but it doesn't do the other types of stuff that a brain does. This is very different. When a non-technical critic like Weinstein looks at language models, here's what I think

• 22:09Cal Newport?

  they're doing instead. They're observing from the outside the things that the model is doing. Oh, look at this language it's producing that's very fluent and shows understanding. They then write a story about what's happening inside the model that matches what they observed. Oh, this reminds me of like the type of thing a human child can do. So I'm going to write the story that maybe whatever's happening in this box, it's like a human child's brain. Then they extrapolate from that story which they made up to make predictions about the future. Say, well, a human child brain eventually becomes conscious. So we're going to assume that the language models become conscious too. They wrote a story about what they observed and then they extrapolated from that story to talk about what they think is going to happen.

• 22:54Cal Newport?

  This is essentially the difference between a scientific and something like an ancient animist religion approach to understanding the world. The scientist tries to figure out the actual underlying mechanisms behind something they're observing. And then, once they understand those mechanisms, they can use that to reason about what else might be possible or what else might happen. The ancient animist, by contrast, writes a story about what they observe. Oh, lightning happens. I'm gonna write a story that lightning is because the gods are mad. And then they extrapolate from that story. If lightning is because the gods are mad, I assume that sacrifices make the gods happy. Ergo, we really need to be doing sacrifices to prevent getting hit by lightning.

• 23:38Cal Newport?

  That's like an ancient way of thinking. You look at the stars, you look at the world, you write a story that makes sense to you, and then you build your understanding of the world off of extrapolations of that story. We have to sacrifice to keep the gods from hating us with lightning. The scientist says, I don't care what story seems interesting to me. I want to actually try to understand the real mechanisms, and that greatly constrains what's actually possible. We cannot just say whatever we want to be true or what sounds interesting about systems like AI. The way the system's design matters. The fact that language models are using tables of static numbers that are being sequentially processed with a multiplication algorithm, that matters. That tells us something about what these minds can and can't do. It's what tells us that many of the things that Weinstein just said there casually

• 24:23Cal Newport?

  is an inappropriate description of how a language model actually operates. All right. So this brings us to an interesting point because a common response I get at the point in these type of explanations is like, okay, sure, like Brett Weinstein is not a computer scientist. But there are computer scientists who seem to be saying similar things, like in particular, Geoffrey Hinton, the so-called godfather of AI who invented some of the key technologies behind modern language models. He also seems to be saying similar things, and he does know how these things work. So how do you explain that? Well, I think this is a really good question. And this is what I'm going to get into next because I think understanding what people like Jeff Hinton are actually saying and how it differs from what non-technical critics like Weinstein are saying,

• 25:06Cal Newport?

  and how these get mixed up is going to be the next critical step in understanding and talking about AI in a more reasonable manner. So that's what I want to get to next. But first, we have to take a quick break to talk about the sponsors that make this show possible.

• 25:22Speaker 02

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now? Then this is a job for Indeed sponsored jobs.

• 25:52Cal Newport?

  All right, Jesse, we have to talk about Cozy Earth. As listeners know, I'm a huge fan of Cozy Earth's bamboo sheets. They're the most comfortable sheets we own. I think we have something like three different sets that we rotate through to make sure that we're never sleeping without them. But I just got recently the bamboo pajamas and it's really an incredible set of PJs because they're, they're super comfortable. It's like the fabric from the sheets, but softer. And now you can just wear them on your body all the time. Well, I've been thinking about both the sheets and the pajamas because look, the nights are getting colder. The holidays are upon us. Right? So this is the time then we think about being cozy. Now, when it's that kind of cold night and you're downstairs by the fire, having on

• 26:36Cal Newport?

  the soft stretch knit of the bamboo pajamas, that's a game changer. Right? Having the, like their bubble blanket. Oh, I love that one. They have this, this, this blanket, which is so comfortable that we fight over in our house, being able to cuddle with that by the fire. Oh, that's what we want this time of year. And then the ability to go up and get in those sheets, you know, gonna be so comfortable. I don't know, the holiday season, when things get colder, I think cozy earth because I think comfort and I think coziness. So if you're, if you're looking for the perfect holiday gift, whether for yourself, you deserve it, or for other people you care about, join the Cozy Earth community. Start with the amazing bamboo pajama sets and the bed sheets, and then you can go on from there. So give the gift of comfort that lasts beyond the holidays.

• 27:21Cal Newport?

  This weekend only, from Thanksgiving Day through Cyber Monday, you can get 40% off at cozyearth.com if you use the code DEEP. This is the best deal of the year. You don't want to miss this. That's code DEEP for 40% off. And if you get a post-purchase survey, be sure to mention that you heard about Cozy Earth right here on the Deep Questions Podcast. Wrap the ones you love in luxury with Cozy Earth. I also want to talk about our friends at Lofti. Like, we had daylight savings time not so long ago. And with those shorter days and less sun, our internal clocks begin to get mixed up. It messes up with our sleep hygiene. Fortunately, this is where our sponsor Lofti enters the scene. The Lofti clock is a bedside essential,

• 28:04Cal Newport?

  engineered by sleep experts to transform both your bedtimes and your mornings. All right, here's a few things I like about the Lofti clocks. A, they're beautiful, right? It's like, they're really well-designed. It's kind of like a modern, I don't know the terminology, but it looks really nice. It's very simple. Two, it has gradual wake-up. You can have, for example, a two-phase alarm where you get like the sort of soft wake-up sound that kind of like, hey, it's kind of time to wake up, that slowly brings you out of sleep. And then you get the more definitive sound, which like pulls you the rest of the way, as opposed to just jarring you out of sleep all at once with like some sort of really loud buzzer. And two, it's entirely operational with the physical clock itself. You don't need your

• 28:48Cal Newport?

  phone in the room. You know, I don't want my phone in my room. I'm a big believer that your phone stays downstairs in the kitchen. But a lot of people end up with their phone in their room because they need the alarm function, or they need to use the phone to control their alarm clock. Not with Lofti. No phone. Keep that downstairs. You can control it, turn it off, snooze it, whatever you need to do, right there from the actual physical device. We love these types of clocks, these gradual wake-up types of really nice looking clocks. We have four of them. All three of my kids and my wife and I, we all have one in our room. My 10-year-old just discovered the snooze function, which is kind of a problem because he doesn't realize the snoozing the alarm doesn't also snooze the bus. Like, that's still going to come in the same time. But otherwise, we're a big

• 29:31Cal Newport?

  fan of these type of clocks. All right. So these clocks are a better, more natural way to wake up. They also look great, and they keep your phone out of your room. You can join over 150,000 blissful sleepers who have upgraded their rest and mornings with Lofti. Go to byloftie.com and use code DEEP20 for 20% off orders over $100. That's B-Y-L-O-F-T-I-E.com and use that code DEEP20.

• 30:01Cal Newport?

  All right, let's get back to our discussion of AI. As promised, I want to get into Geoffrey Hinton. Geoffrey Hinton knows a lot about language models because he helped invent the fundamental algorithm that's used in how we train them. And yet, in recent years, he has been sounding the alarm about AI in ways that kind of sounds similar to the arguments that Bret Weinstein was just making, that I was just saying are based on an inaccurate understanding of how these things function. So I want to try to resolve these discrepancies. Jesse, let's start by playing a clip of Hinton. This is from earlier this year, giving, you know, sounding the alarm.

• 30:38Speaker 03

  And I think until quite recently, almost everybody thought that was just science fiction. We didn't really have to worry about AI becoming smarter than us and taking over from us. But I think now many people have come to realize that's a very real risk. So most of the experts believe that sometime between five and 20 years from now, AI will get smarter than people. And when it gets smarter, it won't get a little bit smarter, it'll get a lot smarter. And the problem is, we know very few examples of smarter things being controlled by less smart things.

• 31:11Cal Newport?

  All right, so these are pretty dire warnings, but what is he actually saying? Well, we can get some more insight if we go back to a great profile of Hinton that was written by Josh Rothman in The New Yorker back in 2023. And if we look in that profile, we can figure out, first of all, what was it that in more recent years made Hinton suddenly afraid? He wasn't warning about AI five, 10 years ago, but in the last few years, he's gotten worried about it. The first thing we can do is identify what did he see in new AI technology recently that got him worried? So I'm going to read now from the The New Yorker profile, a relevant piece of the answer to this question. All right, so here's Hinton from the profile.

• 31:54Cal Newport?

  People say it's just glorified autocomplete. Now, let's analyze that. Suppose that you want to be really good at predicting the next word. If you want to be really good, you have to understand what's being said. That's the only way. So by training something to be really good at predicting the next word, you're actually forcing it to understand. Yes, it's autocomplete, but you didn't think through what it means to have a really good autocomplete. All right, so this is an important point, and it's one I've made repeatedly, including in the introduction of this episode. What made Hinton originally impressed is that in winning the game of predicting the next word, which is what these models are trained to do, they got a lot smarter than he expected they would. Because if you think about it, just like he argues here, if you really want to be good at winning the game of what word should come next,

• 32:38Cal Newport?

  you have to really understand the text that you're extending. And like in the example I gave earlier, if the text you're extending is like a joke and the next word is a key part of the punchline, well, like to win the game there, you have to understand humor. And in fact, if you read longer in the Rothman profile, this was the thing, the specific type of functionality that really caught Hinton's attention is when he realized these models are produce, they can produce text that's funny, like they can understand what funny is. And that's where he said, oh, the token prediction game, if you train it on a big enough network long enough, requires these models in those vast tables of numbers that define them

• 33:20Cal Newport?

  to encode really complicated pattern recognizers and logics and knowledge. It gets and language understanding, like it gets pretty sophisticated. So there's a lot of understanding. That's what James Summers argued in his point as well, encoded in those vast table of numbers. And Hinton was caught off guard by, wow, just how much understanding a language model can build up. But here's the thing, Hinton is not worried about language models.

• 33:49Cal Newport?

  Him seeing how much understanding language models built up to win at the prediction game, got him, and this becomes clear in the profile, got him thinking, well, if we could solve that problem so much more impressively and faster than I would have guessed, I'm going to get, I'm going to assume that we could do that with other AI-related problems as well, right? Because he says, if you look at it closely, he's not worried about language models. He would completely disagree with Weinstein that language models, as we have them today, are conscious, or that they're running experiments, or that they have intentions. No, he knows how they work. He helped invent the technology. He knows it's a static table of numbers, and there's rote multiplications that moves information through. There's vast

• 34:31Cal Newport?

  understanding and knowledge and coding those numbers, but the actual operation of executing does not operate like a human brain. It can't be conscious. There's no intention, etc. But he was so impressed by how quickly that technology went, that he began to worry about, well, what other AI-related technologies are going to start going faster as well? So what he's really worried about is hypothetical new machines that we haven't built yet, but now he's saying, maybe we will build them faster than we thought. We could build a language model that understands humor, when I thought that was impossible 10 years ago. Why can't we also build an artificial brain that can be conscious, or have intentions, or manipulate us, or try to actually do things that are outside of our interest? Here's another quote from the New Yorker profile that gets at this reality, that Hinton

• 35:15Cal Newport?

  is worried about future machines that we haven't built yet. Right, here's the quote, "Hinton warns that even a benign autonomous system could wreak havoc. You want a system to be effective, you need to give it the ability to create its own sub-goals," he says. "Now, the problem is, there's a very general sub-goal that helps with almost all goals, get more control." So what he's talking about here is not a language model. Language models don't have goals. Again, it's a static table of numbers. You multiply vectors, you get a word on the other end. There's no goals, right? There's no intentions. There's no decisions being made like that. But he's thinking like, oh, what if we did build a system with goals? What if we built a system that could make decisions and plan and figure out what it wanted to do? Then that could be pretty dangerous.

• 35:55Cal Newport?

  So Weinstein is talking about language models, Hinton's talking about AI artificial brains that we haven't built yet, but he's now more confident than he was before that they are buildable. That's different. Now, here's the key question. Do I agree with Hinton that, am I one of the, he says, "All experts agree, within five to 20 years, we'll have AIs that are more intelligent than us." I'm not so quick. I don't think that's that easy to build. I mean, Hinton's an expert on language models, but now he's speculating about just artificial brain designs that don't exist yet. There's much more of an even playing field here. I'm not so sure that we can keep making progress with these brand-new designs that don't exist yet to get something like an artificial brain that does the type of things that like Weinstein was worried about. All right, we know this

• 36:40Cal Newport?

  in part because we've been trying in simple ways, and it's like been going okay. So like the simplest thing we've been trying recently is what are known as like AI agents. And really what these are is like a control program, the straightforward program, you know, written in Python, just like normal code a human wrote, that has like linear logics we build into it. And then it prompts language models to try to get ideas or plans, right? So like, "Hey, language model, here's what I want to do. Give me a plan for this, or tell me what I should do." And then it gets the answer from the language model, and then it goes and executes on its behalf. We talk about AI agents right now, that's what we're talking about. Those are okay, but they're not working that well. Like, 2025 was supposed to be the year of AI agents, where all of these, these programs that call language models,

• 37:24Cal Newport?

  we're going to automate more and more of our work tasks. That was mainly a failure, because it turns out that, yes, language models have really sophisticated understanding baked into their vast tables of numbers, like they really understand words and have a lot of logics. But there's also a lot of things these language models don't do, because they weren't, that's not what they're supposed to do, the, the keeping models of the world, understanding how things work, planning, trying to figure out different scenarios for the future. They don't have any of those capabilities, because again, it's a, it's a sequential static processing, but can just move a number through, and they were trained on word prediction. So what they have encoded in there makes them good for predicting words, but they're not very good at like, "Let me make a plan about what to do for

• 38:06Cal Newport?

  you in your office that's going to make sense, given your particular job and the people you know." Doesn't have any of that information, and it can't simulate the future. And it doesn't have a really good world model. We thought they might, but they don't have a very good world model. We now know this from research. So the agents we build that just ask language models, they're inconsistent and unpredictable. They're not that great. That's why we don't have a lot of these, even though that was supposed to be the financial savior of OpenAI in 2025. They're just too unpredictable for anything but the simplest of tasks. And so that's why we haven't seen them used. So if we really want to build a more complicated artificial brain, there might be a language model module that can, we can leverage that for its deep understanding of language and what it means and its ability to create fluent language. But it's also going to need other modules to do all these other things that

• 38:51Cal Newport?

  brains do. It's going to need a module that has like a really good model of the world. It's going to have to have something like a policy network that we encode sort of like its values and drives. They can evaluate potential actions to figure out what action is good based on what happens. It's going to need memories. It's going to need updatable state. It's going to need actuation, ability to try things, sees what happens, update its memories, update its state. It's going to have to hook these all together in some sort of complex way. The language model might be in there, but it's one of like a bunch of different pieces we would need to have anything like an artificial brain that does more than like what these very simple agents do. But we don't really know how to do that. You know, in fact, for an article I was working on earlier this year, I ended up interviewing some of the the world's top roboticists, and they've been working on some of these

• 39:35Cal Newport?

  problems in the world of robotics, especially building a good general world model you can use to plan actions. The advances in language models really have no impact on that challenge. That that's its own challenge. The fact that we're better at language models doesn't help that challenge very much. And they're, they have no idea how to do this well. They've been working on it for decades. There hasn't been any major breakthroughs recently, and they're continuing to work on it. And they have, there are no optimism that like, oh, yeah, we're just a year away from being able to model the world. That's like a completely separate challenge. Policy networks, good simulations of general types of operations, like these type of things, people are working on them. But they're different technologies than language models, and they're hard. But we need to have breakthroughs in all these

• 40:16Cal Newport?

  different non-language model technologies to maybe get something like a more complicated artificial brain. But even if we do, Now we have a very module AI system where we have different systems that we're connecting together. That's way more controllable than like a language model's understanding. We're like, we don't know what's in its neural networks and we don't know what word it's going to produce and it's very unpredictable. So we are so far away from something like a brain with intentions that can manipulate us or get smarter than us. And if you really push Hinton, and I read a bunch of interviews with Hinton, if you really push him, well, what are these systems going to look like five to 20 years ago? They're not language models. What are they going to look like? He more and more now falls back on the same

• 40:58Cal Newport?

  explanation that I talked about a couple of weeks ago when I analyzed Eliyahu's, Yudaskis' conversation with Ezra Klein. He falls back on recursive self-improvement. I don't know, but maybe the AI will just build smarter and smarter AIs, which, you know, is a complete cop out. You know, I don't know, but the AI will figure it out. That's not that interesting to me. All right. Let's, uh, let's go to some takeaways, Jesse.

• 41:34Cal Newport?

  All right. To most computer scientists, especially computer scientists that are not associated with a particular company that makes money by selling AI, they look at language models as being exciting and impressive, but not mysterious and not scary. Now, once you understand how these models actually work, you realize that they are, they are much, much simpler and much, much more mechanical and way more limited than the way a human brain functions. And even if the pattern recognizers and rules and knowledge that they encode are vast and impressive, their operations are understandable. They are not alive. They do not have motives. They have no memories. They cannot act on their own. They cannot manipulate us. They certainly cannot approach anything like consciousness. What they can do is understand text and produce fluent answers in response. And that

• 42:19Cal Newport?

  is really useful for a lot of things, but not everything. And as researchers learned when they tried to build like simple agents by, hey, here's a control program that's going to query an LM to figure out what to do. LLMs are not even really good general thinkers. Again, they're good at producing fluent language, but they're not general thinkers. They do not have general capabilities to create the type of AI that is, the type of AI that concerns Jeff Hinton, the type of AI that Weinstein thinks language models are right now, you would need many, many more types of modules that do many more advanced things that are hooked together in complicated ways. And we don't know how to do most of that yet. We would need systems that have world modeling, simulation and planning, drives, motivational systems, memories and real time learning. All of these are things that language models don't have. All of these are hard.

• 43:03Cal Newport?

  We're nowhere near having coherent systems that function with all these capabilities together. So AI presents many problems, but most of them are happening right now, not in the future, but right now. Problems like impacts on our mind or our relationships or our politics or environment or our very definition of truth. But when I look at AI, I think the right question is like, hey, who is asking for this? What good is this bringing us? Why are we tolerating all these harms that it's causing right now? What problems is this, this technology actually solving in our life? Why are we standing for all the real issues that it's creating? The question that's not interesting me is when will it come alive and is it trying to trick us? I think that latter type of questions is exciting to ask.

• 43:45Cal Newport?

  It makes for good conversations on, you know, podcasts, but it distracts us from the real problems that AI is causing right now. And when you understand how these systems actually work, we can be much more specific about what they can and can't do. And most of these science fiction type concerns falls squarely into the camp of things that current systems can't do. All right, there we go. A little bit more AI. I don't know. I can't help myself, Jesse. I just, I think about AI a lot. I read a lot of these articles. I hear a lot of discussion of it and it still seems very messy to me.

• 44:17Speaker 04

  Do a lot of your students go into AI as a career?

• 44:20Cal Newport?

  Uh, you know, our AI classes are popular. Yeah, for sure. I think there definitely is a lot more interest in AI because it's the various AI systems, especially those around the language model and visual model spaces are like really interesting, impressive systems. But they're not, I just, it really confuses the conversation when we get into this type of like, it's just like a baby and it's conscious and it's trying to figure out how to change the world. And we can't just say whatever we want to be true about AI. I'm an algorithm theorist. The whole way we see the world of computation is give me a model, give me an algorithm. And that uniquely specifies capabilities, what it can and can't do and how fast it can do it. Like, everything is about constraints. What can this system do and not do? What can this system do and not do?

• 45:05Cal Newport?

  But with AI, non-technical critics just come at it from, this system can do anything, why not?

• 45:12Cal Newport?

  But how they operate really matters. So, I don't know. I try. It seems to upset people sometimes when I talk about this. There's, I think some people are really invested in like, no, no, no, it's a baby's brain that's probably manipulating us right now.

• 45:24Speaker 04

  Were a lot of those classes you took when you were in college and your PhD program similar to the AI classes now?

• 45:29Cal Newport?

  No, it was much older technology. Like, I took, I took a course at MIT that is now completely, you would never, outdated. I took, I took a course in automated speech recognition. Right? So, how do you have a computer understand if you speak into a microphone and say something? How do you convert that into like text? We literally liked exams in that course was looking at waveforms, right? And learning like, oh, that's a, that's explosive. That's a T8 sound. Like, we're trying to learn like how the different shapes connected to sounds and building these like simple recognizers that like you would hand code

• 46:10Cal Newport?

  that would break the waveforms into the individual sounds they look like and then put those together and try to figure out what word it is. This is completely solved by machine learning today. You're just like, no, I'm just going to feed you a bunch of sounds where we know what we, someone typed out what the, what the words are. And you just, I don't know how you do it. Just train yourself until like you're very good at predicting what the actual words are. So, there was no reason to try to, that's old fashioned AI. We were trying to like program in manually what different sounds look like. And now, you know, you make one call to PyTorch in like a Python program. And, you know, it's way simpler. Yeah, it's like, that was way out of date. Like, there's a lot of machine learning around, but it was more specialized. A lot, I mean, this is getting kind of nerdy, but like.

• 46:56Cal Newport?

  Neural networks, people were coming around on neural networks, but there was a lot of people really liked in the 90s, coming into 2000s, the more mathematically precise machine learning tools. So the tools that had like precise mathematical theorems you could prove around them. They didn't want to just take like a neural network and train it unsupervised with a bunch of data and see if it got good. You would use things like support vector machines, which is like a very precise mathematical way of trying to learn, you know, separators in multi-dimensional space that you can prove things about. Like it's going to converge in this many iterations. People were very comfortable with that. It was really in the 2010 that this idea of no, no, neural networks can just learn stuff on their own, even though

• 47:38Cal Newport?

  mathematically, we don't know why. We can't prove why they work. Like that, that took off. So I was, I was in grad school in the 2000s and so it was a very different landscape.

• 47:47Speaker 04

  If, last question. If you went into one of the Georgetown courses today with dealing with AI, would you do fine in it? Would you like know everything they're talking about?

• 47:55Cal Newport?

  Um, yeah. I mean, I'm not doing a lot of development. I'd have to learn the libraries, like the coding environments, but yeah, no, I think I could do well. If I took one of those courses, I'm sure I'd do okay. Yeah. I mean, it's, you know, it's, it's a lot of this stuff today. It's like, it's all captured in libraries. You're not writing a lot of original code. It's like a lot of like the art with machine learning based neural networks is like parameterizing it, like what struck, what type of network do I want to use? How big should it be? How many layers should it be? It's a lot of this like subjective, like I just have a feel for like how I'm going to set this thing up. Oh, that didn't really work. Let me reduce the layers and includes the increase the density of parameter. Oh, that works better. Like there's like this, the sort of feel to it. But it goes

• 48:36Cal Newport?

  to show how hard it is to predict what's going to be big. Because like if I had studied deep learning when I was in grad school, it would have set me up perfectly time wise to be like one of these, you know, $2 million a year AI hires, but no one-

• 48:51Speaker 04

  They're getting more than that. They're getting these huge deals.

• 48:53Cal Newport?

  But like no one-

• 48:54Speaker 04

  That's kind of what got me thought, thinking about that, actually.

• 48:57Cal Newport?

  So it's a supply and demand thing. That's the problem is no one knew that. And now a lot of people are studying it, but by the time they're ready, there'll be a lot of people studying it and they're not going to be worth that money. So you had to be the person who in like 2005 was like, you know, Yann LeCun's on to something, deep learning, like we have enough data now, let's go get some GPUs. You had to be that guy, like the woman in 2005 in order to like in 2025 to really be balling out. It's like impossible to predict sometimes.

• 49:26Cal Newport?

  All right, let us move on to some questions.

• 49:33Speaker 04

  First question is from Maeve. Can other forms of AI like those used for military biomedical and financial purposes be dangerous? Can you can your explanation against LLMs taking over the world extended this sort of AI use case?

• 49:46Cal Newport?

  Well, here's what's important about this question. Like AI is a broad term. Often what people are talking about when they use that term today are language model based tools like chatbots, right? That's like what Brett Weinstein was talking about. But AI has been around for a long time. And there's lots of other tools that use something like AI that try to do something useful. Like when I first started at Georgetown, there's a there's a group there that continues to work today, like very closely with the hospital at Georgetown. I do a lot of work with, you know, medical records, looking at films or test results and looking for patterns and helping people to figure out like, hey, do diagnoses or whatever. That's something to do with language models. Language models are very specifically for producing text. But they are working on that. Lots of people

• 50:27Cal Newport?

  have been working on that. Radiology has been like a subject of AI interest for decades. Can we train models to be really good at looking at MRI or x-ray films and finding things that like a human radiologist might miss like that? I think there's a there's a cancer over there that maybe like the human observer missed. And they're like, you could do pretty good at this. Some of these things, there's a lot of plateaus that they hit like Jeff Hinton famously said, by like 20 whatever, a date that already passed, there would be no radiologist left. There's just as many radiologists as they are. We haven't actually cracked the problem of studying these films with with AI nearly as well as we thought we would be because there's like plateaus, like you run out of data, the techniques can only do so much. There's a lot of different AI use in a lot of different places. They're

• 51:12Cal Newport?

  not language models. I think that's important because people often get this wrong or they mix it up. They mix all these things together and they assume because language models made rapid breakthroughs in a five year period that these other AIs did as well. But there are different types of problems and different types of systems. And they're not connected. You know, some people think, I hear this all the time, they think it's like the language models that we're using, that will be the key to like biomedical AI. Most biomedical AI has nothing to do with language models. Or the language models is what we're going to, now our weapons will be smarter. No, that's a completely different type of AI that you're going to use in weapons. Or robots now, look at Tesla's Optimus robot, which they basically canceled that project. Like, oh, because of language models, like now our robots will be real smart. No, that's completely different technology.

• 51:56Cal Newport?

  They're not having comparable breakthroughs. There's a lot of AI technologies for a lot of areas, protein folding, playing games. These are all different types of AIs. And they're all having advances at different speeds. None of them really are having advances at the speed of language models with like the exception of some of DeepMind's work. You know, they were having some big breakthroughs, I think, on games, and the protein folding was a big breakthrough, but those are like bespoke systems. But for the most part, these are all separate AIs that require separate breakthroughs, and they're all moving at their own speeds. Most of these AIs have been around for a long time, the technologies, there's nothing really new. So if you weren't afraid five years ago of these technologies becoming super intelligent, then like you shouldn't be afraid of that today.

• 52:39Cal Newport?

  There hasn't been any major change for most of these AIs and how they operate. It's much more incremental, trying to break through plateaus. So no, these are all like systems we can understand that operate in specific ways and are really hard to get to work well, and they're working to try to get them to work well. And the gap between how do we get better at noticing abnormalities on MRI scans and Skynet is so vast in all these technologies that, no, they don't. It's just the wrong question. It's just the wrong question. Like, are these things going to get out of our control? It's just the wrong question. We're trying so hard right now just to get them in these bespoke fragile systems, just to like do what we're asking to do, you know, in a way that's useful to us. Like, that's what we're working on with most AI systems.

• 53:23Cal Newport?

  Yeah, there's a lot of AI, most are not language models, they all make progress on their own pace, breakthroughs are unevenly distributed, and none of them are on some trajectory of, you know, where I would be concerned about, oh my God, it's about to be really dangerous. But again, this, we mix this up a lot. I'm uncomfortable with the language produced by a language model, and I'm going to translate that to be uncomfortable with the fact that there's like an AI involved in a biomedical application. These are different systems. They work in different ways. All right. What do we got next?

• 53:54Speaker 04

  Next up is from M. You said that it's imperative that we stop distracting ourselves with the fear of superintelligence and said focus on actual concerns of AI. What are those concerns?

• 54:05Cal Newport?

  Well, I think this deserves its own, really its own full episode, but some of the common concerns that are out there that I share about AI is things like its impact on our ability to think and create new knowledge fluently. It's like using a car more than walking, but the cognitive equivalent of that. You're going to lose the ability to produce new knowledge from scratch when you're over dependent on language production. Language production is hard. Language models are good at language production, so we use them. But that hardness is how we actually get better. We cement those circuits and get more facile with producing original knowledge. So I worry about that. I worry about the obviously impact on truth, creating a society where we really can't trust what we see. Be it a picture or a video, like that's going to be, that's a real problem.

• 54:48Cal Newport?

  I'm worried about slop in all of its forms, whether it be like super distracting slop like you get on an app like Sora or Office Slop, or you have more people lazily producing long emails and meeting summaries with AI that don't have much useful content. It wastes everyone time. Social media slop, it's just like a bunch of, you know, whatever. Instagram just becomes a lot of, of sort of middle of the road, sort of like generic inspirational sayings, like none of that's that great. I'm very worried on the financial impact that if there is a big market correction because of our overextending in this race to build AIs that aren't going to be as powerful as we think, that's going to hurt everyone's portfolio. That's going to hurt the economy. That could cause real problems. And there's environmental concerns. Though my, my prediction

• 55:30Cal Newport?

  for the future of AI is that these giant models that use up all this energy, it just doesn't make sense financially. And really, the future of language models are going to be much smaller language models that are deployed at the edges, right? So you're going to have a language model on your phone to understand your text, not you sending a query to a giant foundational model in a data center somewhere. There'll be a language model on your computer as part of the word processing application that helps does things on your behalf. So I think we could probably solve the environmental issue because a language model that uses so much energy to query that is environmental problem is not a very financially sustainable type of program to run. And we don't need, an oracle can do everything. I think we need oracles that can do the specific things we need. So I'm worried about it, but I think we might solve that one.

• 56:16Cal Newport?

  All right, who do we got next?

• 56:17Speaker 04

  Next up is CJ. I recently heard Dr. Hinton assert that there is no evidence in nature of a more intelligent organism or species being controlled by a less intelligent one. This observation was made as a rebuttal in the context of the question as to whether we can control AI or not. If Dr. Hinton's observation is true, would this not create an internal break on recursive self-improvement? While humans may be so dumb as to create a new technology that we cannot control and ultimately destroy us, won't AI be smart enough at some point not to make the same mistake?

• 56:49Cal Newport?

  I thought that was really clever. It's actually like a pretty clever thought experiment, right? It's saying like, okay, if it's true that things that are smarter than you aren't going to let you control it, then why would you build something smarter? He's like, okay, maybe we're dumb enough to do this, but as you're doing recursive self-improvement, at some point, you're going to have an AI smart enough, right? To realize, why would I build something smarter than me? It's going to take control of me and it would refuse to do it. I think that's a really cool thought experiment. But here's the bigger issue here. I'm getting kind of tired of these thought experiments about superintelligence that we talked about a few weeks ago. They come out of the sort of rationalist community, existential risk community.

• 57:31Cal Newport?

  To me, discussing like the, these type of thought experiments about recursive self-improvement is like discussing time travel paradoxes. It's kind of interesting. Well, wait a second, if I could go back in time and then I murdered my father, then how would I exist and how would that work? It's like an interesting thought experiment, but we're not going back in time. Like it's not actually, we don't really want physicists caring about time travel. We don't want like legislation, you know, being passed to prevent time travel paradoxes. It's just, you know, it's mental masturbation, right? Like, we're not going to invent, we're not inventing these superintelligence. This is not going to happen. RSI is not going to happen. We can barely vibe code, like really simple. It's not going to invent technology that is, it's just, if you understand these technologies, that's just not going to happen.

• 58:15Cal Newport?

  Or being able to help you like produce code that it's seen a thousand examples of, it's not going to enable you to now create like the next new AI model no one's seen before. So this is like, to me, the same as discussing time travel paradoxes. It's interesting. It's really not like a serious conversation though. It's fun, not serious and not something we should be losing sleep about. All right, we got a few more questions and then we have a case study that's a PowerPoint coming up.

• 58:42Speaker 04

  Yeah.

• 58:42Cal Newport?

  Someone sent a PowerPoint of their like organizational system, which I think is really cool. We're going to read some comments later, but first let's take a quick break to hear from another sponsor that makes the show possible.

• 58:53Speaker 02

  This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsored jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsored jobs. Get a $75 sponsored job credit at indeed.com/podcast. Terms and conditions apply. Hiring now, then this is a job for Indeed sponsored jobs.

• 59:23Cal Newport?

  Are you looking for a gift for your parents or aunts or uncles or grandparents? Let me tell you exactly what you should get. An Aura Frame. These are digital picture frames that allow you to easily upload your photos, right? You can do it straight from your phone right after you take the picture with just a couple of clicks. Then that frame will rotate through all the images that you've uploaded to this. So we've bought these frames for both my parents and my wife's parents. They love them, right? It's become our habit that every time we're doing something interesting, we just upload some photos right from our phones to their frames and they get to see, oh, what we're up to, see pictures of the grandkids, see pictures of their kids. They really do love it. Here's something else you can do that's cool.

• 1:00:05Cal Newport?

  You can pre-upload photos to a particular frame that you're buying for someone as a gift. So as soon as they unwrap and set up that Aura Frame, boom, they're going to have photos there to look at right away. You can even imagine getting one of these for yourself, put on your desk at work. Why not keep up with pictures or get a little boost of pictures of your own kids or family while you're otherwise toiling away? Look, you can't wrap togetherness, but you can frame it. Now for a limited time, if you visit auraframes.com, you will get $45 off Aura's best-selling Carver Matte Frames, which were named number one by Wirecutter. You just have to use the promo code DEEPQUESTIONS at checkout. That's A U R A frames.com promo code DEEPQUESTIONS.

• 1:00:50Cal Newport?

  This exclusive Black Friday/Cyber Monday deal is their best of the year. So order now before it ends. Remember to support the show by mentioning us at checkout. I also want to talk about our friends at Caldera. Here's the thing about being an aging guy. When you're young, you don't think about your skin. You spend time in the sun, maybe like only occasionally cleaning the grime off your face with a Brillo pad and some axle grease, and you still end up looking like, you know, Leonardo DiCaprio in Growing Pains. But then one day you wake up and you're like, oh, my God, I look like a grizzled old pirate captain. Or maybe like one battle after another version of Leonardo DiCaprio. And you're like, why didn't anyone tell me I'm supposed to take care of my skin? I actually, you know, I can't say what it was, but Jesse knows I was doing a thing recently

• 1:01:32Cal Newport?

  where I was being filmed by a big film crew and there was a makeup artist and she was getting on me about it, right? She's like, yeah, you gotta moisturize, like you can't put makeup on for camera if you're not moisturized. Like, no one told me this. All right. So how do you avoid that? Oh, and I told her about this product, by the way. I was like, I know exactly what to do. Caldera Lab. Their high-performance skincare is designed specifically for men. It's simple, effective, and backed by science. Their products include the Good, an award-winning serum packed with 27 active botanicals and 3.4 million antioxidant units per drop. Their Eye Serum, which helps reduce the appearance of tired eyes. And their Base Layer, which is a nutrient-rich moisturizer infused with plant stem scales and snow mushroom extract. This stuff works.

• 1:02:14Cal Newport?

  A consumer study, 100% of men said their skin looked better and healthier. So men, you want to look more like growing pains Leonardo DiCaprio and less like a grizzled pirate captain, you need to try Caldera Lab. Skincare doesn't have to be complicated, but it should be good. Upgrade your routine with Caldera Lab and see the difference for yourself. Go to calderalab.com/deep and use the code DEEP at checkout for 20% off your first order. That's C-A-L-D-E-R-A-Lab.com/deep and use that code DEEP for 20% off your first order. All right, what we got next?

• 1:02:49Speaker 04

  Next up is Ben. Does James Summers make a valid argument in his recent New Yorker article that AI can think?

• 1:02:56Cal Newport?

  It's a good article. I like that article. So James is one of, there's several writers of The New Yorker, like who are on various, the contributor staff or the staff writer staff. There's three of us that I know of that have technical backgrounds. So James has a development background. I think he's still actually an active computer programmer. Kyle Shaka has a development background and I'm like the sort of resident computer scientist. And so we kind of know each other and each other's work. His article on AI thinking came out in the magazine a few weeks ago. I thought it was good. Here's what he's arguing, and I think he's right.

• 1:03:30Cal Newport?

  He basically is arguing what Jeff Hinton said before, that quote I read is like, yes, while it's true that what a language model does is just produce a single next word, is trying to win the game of guessing what word it comes next. In doing that, it can deploy such complicated processes, pattern recognizers, like language knowledge, logics, you know, like what would be a good joke, or how do I follow the logic of the argument, and knowledge, like it can pull from, that there's a argument, you can say, that's thinking. That's a form of thinking. Give me information and I can apply like lots of things to it to produce some new type

• 1:04:13Cal Newport?

  of information that depends on like deep understanding. He's saying like, that's a form of thinking. So we can say that even though the operation is simple, what it's doing is thinking. I think that's fine. I think you could definitely call what happens in a language model when that algorithm takes the big static table of numbers and does all those multiplications, you can call what's happening there, thinking, there's understanding happening, etc. Well, there's lots of different types of thinking, and this is doing one type of thinking. So if you want something like an artificial brain that we have to be worried about, there's all other sorts of thinking you need to do as well, like seeing, planning the future, having a model of the world, responding to drives, learning on the fly. These learning models don't do any of that type of thinking, but they do this one thing, this one type of thinking really well. And I think this, this fits well with my analogy of thinking about a language model,

• 1:04:58Cal Newport?

  like the language processing centers of your brain being isolated in a vat. That's part of thinking, when the language processing part of your brain tries to understand the words that it's encountering. That's part of your brain thinking. And if you take that out and put it in a vat, there's thinking happening when we, when we activate those neurons and send words through that neural network. But it's not all the types of thinking that happen in our brain. So these are the two thoughts we have to hold true in our head at the same time.

• 1:05:26Cal Newport?

  We can have a system, a digital system, do something that we say, yeah, that's a type of thinking. We can mean it. And at the same time, we can look at that system and say, we know how it operates. It's pretty straightforward how it operates. There's all sorts of other types of thinking that we do as humans that it's not doing, and it never can do because of its structure, how we wired it, and the algorithms that we run on it. So I think both are true. And Summers makes that argument towards the end of his article as well. All right, who else do we have?

• 1:05:51Speaker 04

  Next up is Kim. How can I figure out a way to get back into my preferred workflow when I feel like the industry is increasingly moving towards AI? Kim's a junior developer. And she says, if I try to develop exceptional skills now, I don't have the career capital to move into a job where the flaw of not wanting to integrate AI so deeply into my workflow can be overlooked.

• 1:06:13Cal Newport?

  You got to balance two things with AI. Places where it could help you and places where it's just like your, your management team is like, I read a bunch of, you know, I listened to the interview on Rogan, and we need AI, like, we just do AI, right? You got to walk that line between useful AI and non-useful AI. So keep abreast of new tools as a developer. See which ones make you better. Use them. See which ones don't. Ignore them. The key thing I would say is the, and this is in programming, there's a lot of tools that can make things easier and better. You have to use those tools to raise your level. So if what you're doing is just great, this can automate other parts of what I do. So now I can like get my work done faster, not have to work as hard. That's dangerous. If

• 1:06:57Cal Newport?

  AI as a programmer can help you do certain things faster, use that to become a better programmer. Like, oh, now I am a better programmer than I was before this tool. It's not a faster programmer, a better programmer. If this can handle these things for me, now maybe I can build more sophisticated systems. Or if I can get help from AI in figuring out these complicated library calls, I can now use these libraries that intimidated me before, and now I'm open to like, I can build much more complicated stuff. So the AI that helps you, use it to make you better, not just faster. And the AI that doesn't, don't worry about it. Ultimately, the value, what you produce, what matters.

• 1:07:31Cal Newport?

  All right, let's move on now. We'll take a quick break from AI to do a case study. Now typically, this is where people send in a description of them applying the type of advice we talked about on the show. Today, we have a PowerPoint. Someone sent in a PowerPoint of a system they built for organizing their work life built on ideas from the show. So let's hit some case study theme music and we'll pull this up on the screen.

• 1:08:02Cal Newport?

  All right. So here's the system for those who are watching, so just listening. How do we pronounce that, Jesse? He calls it Zetteltoffle.

• 1:08:08Speaker 04

  Yep.

• 1:08:09Cal Newport?

  Zetteltoffle. It says at the top, it is Al Newport inspired. There we go. All right, let's see what we got here. What is Zetteltoffle? Once upon a time, a Kanban board and a Zettelcast and got married, had a baby, and the Zetteltoffle was born. It handles the configure and control portion of productivity management in a clear, tactile, and analog manner. All right. And he specifies here that he's CFO for hire. So he's like a entrepreneur, knowledge worker. All right, there's a board here we're looking at, labeled Zettel. Oh, my God, it's complicated, Jesse. Oh, my. All right. All right. There's pending tasks, and those are, there's rows for different pending tasks, like budget, general finance, loans. And then I like this,

• 1:08:51Cal Newport?

  there's post-it notes under each of these sections that describe individual tasks. Then there's in progress for each of these categories. So that way you can keep track of like all the finance stuff you need to do. But here are the two things he's actually working on at the moment. And then there's waiting. So I assume what happens there is if he's waiting to hear back from someone about some task, he can move it to waiting so he doesn't forget, and then move something else in progress to keep making progress. So that's kind of like classic Kanban, which I think is great. You're being very clear about your workload. You're not forgetting things, but you're also being careful about how much you're actively working on. So I think that's pretty cool. I don't quite know what's happening over here. He has my day. Oh, I see. I think he's moving these tasks over to his day. He says he actively

• 1:09:36Cal Newport?

  works on at most three things per day. He moves those post-it notes over to a slot labeled 123. They kind of decide like which of these things am I working on today? And then I guess he puts it back if he finishes it. There's a done column over here. And then there's other room in here to keep track of like meetings and some other stuff as well. So this is pretty cool. Like this is like a highly structured. Task management system that he could use to keep track of what's going on and its status, all in one physical analog visual form. You could do this in a Google Doc or Trello, but I, I like the way he's doing it here. It's probably nice to actually move these, these Post-Its around. There's some more details. I won't get into all of this, but I love the fact that he has my time block planner, who is time blocking throughout

• 1:10:19Cal Newport?

  his day. Way to go. And he moves stickies from the planner to Zettelcast on the wall. Okay. So when he has things to capture, instead of just writing them in the task capture part of the time block planner, he puts on a Post-it note that's stuck on his planner and then he can move it from the planner to where it should go on his wall, the Zettelcast in wall. So that's pretty cool stuff as well. The more details about capture configure, I see. So now he's breaking down the details we just looked at. All right. That's pretty cool. There's some other details in here as well. I won't go through all of them, but what I like about this example is it shows the level of intention you can have about how you manage your workload in this current age of like constant digital distractions

• 1:11:04Cal Newport?

  and context switching, having some sort of set way of keeping track of what you need to do, what status it is, getting out of your head, making smart decisions about what to do with your time. It really makes a difference. And for some people, this level of detail I think is really powerful. Other people, you could do something much simpler, but I think that's pretty fun, Jesse, to see the degrees to which people push this, especially with their solopreneurs and they can completely control their day. I like that type of complicated board. I think other people might be a little bit. It adds a lot, but I think it's pretty cool. All right. So thanks for seeing that in Ernst. All right. Because we're a little short on time. I think we're going to skip the call

• 1:11:40Cal Newport?

  this week and move right to our third segment. And we are going to, by popular demand, go back to what we've been doing recently, which is responding to comments from past episodes. So today I'm going to respond to some comments. I have the first one on the screen right now. I'm going to respond to some comments from our recent episode about regaining your ability to think using notepads. All right. So here's what I want to share. A lot of these comments are

• 1:12:15Cal Newport?

  Go to top one here. A lot of these comments are listeners actually sharing their own practice. Actually, I guess we do have room to put my face up here, Jesse. I'm looking at it. I was worried about. Yeah, there we go. Now we're on there. A lot of people are sharing their own practices with notebooks. I thought was interesting. So I want to read a few of these. The first such comment came from Too Much Drive to Thrive, who says, I'm a teacher. I buy all my students a notebook every semester. It costs me about 100 bucks each time. I don't care. There are so many benefits to writing things on paper and reviewing them. I'm not a Luddite either. I have an MA in educational technology. I love that. Spend the money to buy your students notebooks, and it has a big deal. This is someone who studied ed tech. All right, here's another

• 1:12:56Cal Newport?

  comment from Sluggy Super. Something that got me into using a journal every day was stripping the significance of the time that I write. It's not my special meditation or deep thinking activity. I just write when I have a thought. If I don't have any, I write about the fact that I'm writing my notebook. Sometimes I just write my plan for the evening, or sometimes I remember, but making it just another thing I do without thinking made it more enjoyable, and therefore something I do daily. That's interesting. If you don't have too many constraints about your notebook, you just like writing in it. You write in it more, and it helps your long thinking. All right, our next comment comes from Anthony Corrigan 5587, who says, I started to do at least a half hour every day of life sketching and gestures in a notebook with a pen. I'm 21 days in without missing a day, and it's the best thing I've done in

• 1:13:40Cal Newport?

  years. It's like daily meditation at this point. A lot of people who don't like actual meditation really do like reflecting in their journal. So I think that's a great example. All right, here's one more listener practice. This came from Almost Goddess, who said, Gee, I've used this method during my PhD as well. Each time I start a new project, I start a new notebook as well. It's very helpful. Thank you for speaking about it so eloquently. Yeah, I think we have one more best practice here. No, okay, well, let's jump to listener questions. All right. Then we have a couple of comments that are follow-up questions to the episode. The first one comes from Photar93, who says, what are you specifically doing during these thinking

• 1:14:21Cal Newport?

  sessions? Asking questions and answering, making lists, free writing? Sometimes I get stuck because I can't deliberately think, but thinking or ideas come up randomly while doing other tasks. The moment I sit down to write, think, it is blank up there. Usually, when I'm doing specific long thinking with a notebook, there's a particular problem I'm trying to solve, like What is going to go into this chapter I'm writing? How do I want to organize my argument for this part of the book? Um, or hey, this part of my life is, why am I struggling in this part of my life? Like, what's going on here that's not right? And what do I think I need to do to fix it? So I have a particular prompt that I'm trying to answer.

• 1:15:03Cal Newport?

  And then I write ideas that help me answer that prompt into the notebook as they come up. I'll write, if I'm doing something else, I'll go get the notebook and write it there. Or I'll go on a walk and be like, let me specifically try to come up with answers about this ideas that help me figure out this prompt. And then I'll write them in the notebook. If you go on a walk and you're just thinking about that prompt, what's going to happen is your brain is going to load up all the relevant neural networks and you're going to find that you have more insights because now your brain has all the right stuff loaded up and you're more likely to come up with new ideas. So I'm never just randomly writing. There's always like a particular prompt when I have one of these notebooks that I'm, that I'm trying to solve. Like, that makes a big difference. All right, let's do one more comment question here before we wrap up for the day. This one comes from Lori Loesch, 7440.

• 1:15:48Cal Newport?

  That's the following. For this long thinking method notebook, is research material allowed? Or would that then make it deep work? Like, should the notebook writing only be for our thoughts? As I'm writing this, the answer is coming, seems to be a yes. The notebook content is only straight from our minds. Please correct me if I'm wrong. Thank you, and I'm glad I just found your channel, even though I'm trying to get offline more and more. You know, it depends on what you're doing and your preference. Like, I want it, if I'm writing an article or something, the notebook would be for like ideas about that article and I would write it in the tool I want to write the article in, where it's a word processor, it's Scrivener, it's on my computer. There's other things I do, though, where I'm, I'm actually putting in my notebook, pretty completed thoughts.

• 1:16:32Cal Newport?

  So this would be the case for mathematical proofs. I would go for a walk to try to make progress on the proofs. I'm going to write out theorems and do the equations right there in the notebook. Like, there's many a case where when I was writing an academic theory paper, I have my notebook open and then I'm copying over equations into it. Like, oh, I did this math over here, let me copy it in here and clean it up. So it just depends on the type of thinking you're doing and what format it's eventually going to end up in. So that, don't have rules about it. Whatever is helping you do better long thinking, do that. You don't have to have hard rules about it. Well, there we go, Jesse. That was actually a popular episode online on YouTube. I think the idea of taking a notebook and using it, I think it's because people are being so bombarded by AI

• 1:17:17Cal Newport?

  and smartphone apps, they're so distracted.

• 1:17:19Speaker 04

  Yeah.

• 1:17:20Cal Newport?

  That the idea of let me just be my thoughts in paper with no mediation, something that 15 years ago, like a course is like a really, it's like a fresh idea today. So that was cool to see. All right. Well, there we go. So there was another, you know,

• 1:17:36Cal Newport?

  I'm like a recovering AI critic, critic or something like this. I can't help myself. I just, I encounter these things. I'm working through these thoughts. I like to share them with you, but we've got some other good ideas coming up in the near future that aren't about AI. Make sure you send in your questions, right? You could go right to the deep life.com slash listen.

• 1:17:55Speaker 04

  Yep.

• 1:17:56Cal Newport?

  And there's a, there's a form you can fill it out. If you have ideas for topics or whatever, you can send them to [email protected]. He'll take a look, but otherwise I'll be back next week for a new pre Thanksgiving episode. Or is this episode coming out right before Thanksgiving? Where are we in the year?

• 1:18:11Speaker 04

  Yeah, this one's coming up.

• 1:18:12Cal Newport?

  Oh, so happy Thanksgiving for those who celebrate. Our next episode will be on the other end of that. So I'm looking forward to that. And until then, as always, stay deep.

• 1:18:24Cal Newport?

  Hi, it's Cal here. One more thing before you go. If you like the deep questions podcast, you will love my email newsletter, which you can sign up for at calnewport.com. Each week I send out a new essay about the theory or practice of living deeply. I've been writing this newsletter since 2007 and over 70,000 subscribers get it sent to their inboxes each week. So if you are serious about resisting the forces of distraction and shallowness that afflict our world, you gotta sign up for my newsletter at calnewport.com and get some deep wisdom delivered to your inbox each week.