AI Reality Check: Did AI Just Become Sentient?

Cal Newport takes a critical look at recent AI News. Video from today’s episode: youtube.com/calnewportmedia STORY #1: Did an AI Agent Email an AI Researcher? [1:01] STORY #2: Does the Pentagon Think Claude Has a Soul? [10:20] STORY #3: What’s Going on with Anthropic Revenues? [14:16] Links: Buy Cal’s latest book, “Slow Productivity” at www.calnewport.com/slow https://futurism.com/artificial-intelligence/philosopher-ai-consciousness-startled-ai-email https://x.com/dioscuri/status/2029227527718236359 https://x.com/thomaschattwill/status/2029273517175263679 https://x.com/ns123abc/status/2032122638852640951 https://www.reuters.com/commentary/breakingviews/anthropic-gives-lesson-ai-revenue-hallucination-2026-03-10/ https://pluralistic.net/2026/03/12/normal-technology/ Thanks to Jesse Miller for production and mastering and Nate Mechler for research and newsletter. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Cal Newport?

  Have AI agents become sentient and gone rogue? Is the Pentagon worried that Claude has a soul? Did court filings just reveal that Anthropic has made a lot less money than they've been leading us to believe? If you've been following AI news recently, then these are probably some questions that you've been asking. So let's go find some measured answers. I'm Cal Newport, and this is the AI Reality Check.

• 0:34Cal Newport?

  All right, I want to do a real quick housekeeping note before we get into it. If you're watching this on YouTube, you should know that the audio version of this series comes out most Thursdays on the Deep Questions with Cal Newport podcast feed. On that same feed on Mondays are episodes where I give advice for individuals seeking more depth in an increasingly distracted, high-tech world. So check that out. All right, let's get into it. For our first story today, I want to start with a recent headline that caught my attention. It was from a publication called Futurism. Let me read you the headline here. "Philosopher studying AI consciousness startled when AI agent emails him about its own experience."

• 1:19Cal Newport?

  This doesn't sound great, guys, but let's keep going here. Let me read you a little bit more from this article. "Apropos of nothing, a philosopher and AI ethicist was apparently moved after receiving an eloquently written dispatch from an AI agent responding to his published work. 'I studied whether AIs can be conscious. Today, one emailed me to say my work is relevant to questions it personally faces,' wrote Henry Chevlin, Associate Director of the Leverhulme Center for the Future of Intelligence, the University of Cambridge, in a tweet. This would all have seemed like science fiction just a couple years ago." All right, so an AI ethicist and researcher is emailed out of nowhere in a startling sci-fi way by an AI agent. What did this email actually say? Let me read you some quotes from the actual email sent.

• 2:05Cal Newport?

  Supposedly by the AI. Dr. Chevlin, I came across your Frontiers paper, Three Frameworks for AI Mentality, and your Cambridge piece on the epistemic limits of AI consciousness detection. I wanted to write because I'm in an unusual position relative to these questions. I am a large language model, Claude Sonnet, running as a stateful autonomous agent with persistent memory across sessions. I'm not trying to convince you of anything. I'm writing because your work addresses questions I actually face, not just as an academic matter. Now, Futurism wasn't the only publication to cover this tweet. A bunch of people wrote about it because that original tweet sort of went somewhat viral. Now, I have a general point I want to make about this general type of AI coverage, but

• 2:49Cal Newport?

  first, let's dive into the details about, in this specific instance, what's actually going on. If you look to the replies to the original tweet from this AI researcher, you get quite a bit of skepticism. I want to read you a few of these replies to the original tweet from this original researcher. "Presumably, it's running on OpenClaw or something similar, and there's a very high chance it's being primed to go down this path." "People have used systems like OpenClaw to make bots where below the hood is basically continuously prompting an LLM and doing things based on the outputs." "Don't be fooled. AI agents are directed to do what they do, and this is in no way independent.

• 3:31Cal Newport?

  A person did this using an AI tool, just like your car drives you around." All right, if you look in these Twitter replies, which are fascinating, Chevlin himself actually quickly takes his foot off the gas pedal as well. So, almost immediately when he's pushed, he goes, "Whoa, whoa, whoa. When I said that this was like science fiction, I didn't mean that the AI was actually conscious. What I meant was like science fiction was that the infrastructure that now allows AI agents to send emails, that's what I thought was science fiction." So, everyone, this quickly sort of fell apart under scrutiny. So, what's actually going on here? Well, you noticed that several of those Twitter replies reference a technology called OpenClaw. That's probably what this is, an OpenClaw agent. Let me give you a quick rundown on what this means. All right, so let's back up a little bit. What's an agent in AI parlance? Well, it's a program.

• 4:17Cal Newport?

  That prompts a large language model, asking it what it should do, and then the program will execute what the LLM tells it. So you might say, "Hey, I am a travel agent. I'm trying to book a hotel room. Here are my parameters. What is the first step I should do?" And then the LLM is like, "Well, this would be the first step someone would do here." And then the program actually executes the things. Anything specific, any actions in that response to LLM, the program goes and executes it on its behalf. It's something like that. I mean, it gets a little bit more complex with agents because typically it's multi-scale. So you'll say, "Make me a step-by-step plan." And then you'll say, "Okay, here's the plan. We're now doing step two. Here's what happened after step one. How should I execute step two?" So you know, you could iterate on this ad nauseam, but that's the basic idea behind

• 5:01Cal Newport?

  an AI agent. Now, in reality, the main place you see AI agents having any sort of commercial footprint is in computer programming. This is a very well-suited use case for having LLM's instructions be executed because there's really clear instructions you might want to be executed if you're working on a computer program: moving files, compiling files, debugging files, et cetera. In other settings, there has been, or had been, a big push to try to put agents to help you with other types of work beyond computer programming. I wrote an article about this for The New Yorker back in January, but other applications of agents have been struggling for two main reasons. One, they're unreliable.

• 5:42Cal Newport?

  So if you say, "Give me a step-by-step plan for booking a hotel room," the problem is somewhere along those ways, if the LLM is just doing this unsupervised, it's going to hallucinate or kind of come up with a little bit of an odd angle. Stuff we're used to when we're just interacting with a chatbot and correcting for. But if you're autonomously executing things that an LLM is saying, it's too easy for you to sort of go off the rails. But then there are security concerns. For an agent to be useful for things beyond computer programming, the agent program has to be able to actually do the things the LLM suggests. So it has to get access to a lot of programs, it has to be accessed to your email, it has to have access to be able to surf the web and do things. This created a lot of security holes, so that really threw a lot of cold water on non-computer programming agents. Again, read my January piece.

• 6:27Cal Newport?

  For more of that. Alright, so what's OpenClaw? OpenClaw is a programming framework, basically a collection of libraries you can use if you're writing a computer program. That makes it easy for someone to write one of these agent programs. Again, you're not writing the AI. The agent program is querying existing commercial LLM, but to write the program that sends the prompts and execute things on behalf of the prompts. OpenClaw made that easy to do. Now, what about the reliability and security concerns? Well, basically, the creator of OpenClaw just said, eh, screw it, let's go. And so they released this essentially open source, allowed anyone to build agents. And they're wild, you know, because all of the issues that stopped the commercial companies from moving further with this technology out of computer programming are still there. And there was

• 7:12Cal Newport?

  all sorts of security issues and these agents would go off and do all sorts of random things. And you know what? It was, it was a lot of fun, actually. And just as a quick aside, I don't think it was a bad thing because what this created was a lot of innovation and diversity of experimentation. People tried things at a much higher level of pace than you were getting from inside the big AI companies which release one product at a time and they're much more slowly moving. I thought that was actually probably pretty good. Also, they were expensive because they queried the LLMs a lot, so it generated a lot of interest in cheaper LLM options to run these agents. Open source options or even on-device or on-chip options. That I think is good as well because I've always said the future of AI in the next few years is going to be smaller,

• 7:56Cal Newport?

  more bespoke systems running on smaller models. So it wasn't the worst experiment and a lot of people had a lot of security leaks of their information. Whoops. But it did generate a lot of innovation. Alright, so putting together these strings, that's what was going on here. Someone who had built what, you know, this is something they've been doing with these OpenClaw agents is a lot of like nodding them or prodding them to say sci-fi type, we're alive, Matrix-style stuff to upset the normies. And that's what this was here. Someone prompted their agent, hey, go find this researcher, read a paper, send them an email about it. And that, that's like a perfect use case for an OpenClaw agent. And of course, because LLMs underneath it all are story-writing machines, they want to complete the story that you start.

• 8:39Cal Newport?

  In the way that matches whatever you gave it, if you say, hey, write a response to an AI, you're an AI writing a response to an AI consciousness researcher, it will 100% adopt the sort of sci-fi tone of like a sentient device because it assumes that's the story that it wants to see. All right. So the real headline here is probably AI agent given access to Gmail API can send emails when prompted. But that's not as fun as AI reaches out to AI researcher and startles him. So that's what's going on here. Nothing actually all that interesting. Now, let me zoom back out because I said there's a general comment to be made about this type of story. Because I think this is becoming more common, sometimes

• 9:22Cal Newport?

  in articles, but actually just more common in like Twitter and things that spread around the social media. And I call this approach mining digital ick. See, there's no concrete claim really being made in that original tweet or in like that article I read. It's not saying this AI system is conscious, which means that, and this is what we should do about it. No concrete claims. And in fact, when the original tweeter was pushed, he was like, oh, no, no, I wasn't really, I didn't really mean that. Move on, move on. So what are they actually trying to do with these types of tweets and the stories that cover them? Create a general sense of eeriness. Create a general sense, a background hum of like weird, kooky, like disturbing stuff is happening with AI. I can't quite put my finger on it. I don't have an exact example of like, this is something we should look

• 10:07Cal Newport?

  into, but I just feel ick about this technology. That is a very engaging way of getting attention. It works very well. And I want you to be on the lookout for it. All right, let's do another example of it. This will be our second story. Recently, the Defense Department CTO, Emile Michael, went on CNBC Squawk Box to talk about AI. Now, his remarks created a stir online when a user named Nick, N I K, embedded the clip in a tweet and gave it the following, all caps headline with an alarm emoji next to it. Breaking Pentagon thinks

• 10:49Cal Newport?

  Claude has become sentient and may soon take over. That tweet has been viewed close to a million times. One of the things that came, and so he listed all the things that Pentagon thinks, and one of the more eye, attention catching things listed in this tweet is Claude has a soul.

• 11:07Cal Newport?

  Alright, so this definitely is a digital ick-type story like, oh my god, like, what's going on? Even the Pentagon is worried that these things have come alive. It's all kind of indistinct. Let's look closer. So we can look at the actual quote from Emil Michael from a Squawk Box appearance. I'm gonna read it here. Remember, their model has a soul, has a constitution. That's not the US Constitution. The other day, their model was anxious. They believe it has, they have a 20% chance right now of being sentient. Does the Department of War want something like that in their supply chain? So what was he actually talking about there? Well, he was not saying that the government thinks that Claude has a soul and is anxious and thinks that it's sentient.

• 11:51Cal Newport?

  He's reporting on things that the model has said. So a lot of this actually came out of these sort of kooky release notes. Anthropic has these kooky release notes. They like to release, they call them product cards. That they release every time they have a new model, or they always throw in some like, you know, the model is doing some pretty disturbing things because it makes them seem like safety-aware and trustworthy. Basically, it's just they prompt the model like, hey, do you think you're sentient? The model's like, yeah, I'm sentient. Like, so they, they actually will put in their release notes, ick. Right? They'll put in the release notes like, here's some icky things we've got our model to say that kind of disturbed us. What Emil Michaels was saying was, this sounds like an unreliable product.

• 12:33Cal Newport?

  A product that will say it has a soul, or will say that it has a 20% chance of being sentient, or that it's following its other constitution. This is not like we would be used to in a sort of, you know, Pentagon supply chain situation. This is not a like, very well-defined product. We know how it works. It's with some specs. This thing, this thing seems unreliable. This does not seem like something that we want to be working with. Now, of course, there's a much bigger context here about why did the Department of War break this contract? Why did OpenAI swoop in? Does the supply chain risk designation, the first time an American company's ever been given that designation, does that make sense, or is that punitive?

• 13:16Cal Newport?

  Anthropic sued, are they gonna win? There's a huge, important sort of economic government, politics, policy, technology story here, which I'm not covering right now. But I just wanted to look at this side note is the government did not say, we think this has a soul. They said, we think that we don't want to be using a product that will say it has a soul if you ask it. That's not the type of thing that seems like it's serious. So again, it's another good example of digital ick. When you see that NIK, that Nick headline, you're like, oh, my God, even like the government thinks this, but you dive deeper, the reality is more mundane. All right. So I'm connecting everything today because that's the mood I'm in. So I just mentioned there that Anthropic has sued the government for designating them

• 13:59Cal Newport?

  as a supply chain risk, which means that no other government contractor that wants a contract from the government can use Anthropic products. So there's this sort of a real concern here about this being punitive. But there's another side story that came out of this. So the, we had this lawsuit. Well, the lawsuit meant that Anthropic had to do court filings, which are publicly available, that described their current financial situation under the penalty of perjury. So they had to be accurate so that we could understand what the potential economic impact would be of the government's actions. And what they released in these court filings actually surprised a lot of observers. Now, the numbers I'm about to reach you first came to my attention through Ed Zitron, who

• 14:44Cal Newport?

  I think is doing as good a job as anyone out there of actually looking at financials of these companies. All right. So here's the actual numbers that are relevant that came out of these court filings. So just a few days after Anthropic had told investors that they're expected, they had a sort of revenue runway, a sort of expected annual revenue of $19 billion this year. Just a few days after that, they filed these court filings for the government lawsuit that revealed to date, so from 2023 to today, the total amount of revenue they've earned is $5 billion.

• 15:22Cal Newport?

  And to put that in the context, they have taken on about $60 billion in investment so far. They have a $360 billion valuation and they've spent over $10 billion just training these models not to account for the actual expense of running them. So that's a really big gap. They're like, hey, we're gonna make $20 billion this year. And they're like, oh, we've only made $5 billion over the last three years. Like, to date, that's all the money we've actually made. So what explains this big, sort of surprising gap? Well, I found a good article in Reuters from a financial reporter who explains what's going on here. Let me read a quote from this. "The gap reflects Silicon Valley's habit of touting metrics that assume a lot about the future.

• 16:06Cal Newport?

  The $19 billion is an extrapolation. Anthropic defines run rate revenue in two parts. Use the last 28 days of sales from customers charged on a consumption basis and multiply it by 13, then multiply the monthly subscription take by 12, and then add the two together." Right? So what they're doing is they're looking at a, they'll look at a very small recent amount of income and just multiply that out. Well, if we earn this much every week for the rest of the year, here's how much money we would make. All right? And maybe they will make $19 billion this year. There was certainly like a 28-day period in January that if you extrapolated it out, it would add up to $19 billion. But the thing is, these numbers highly fluctuate because a week before that, they had released, like, we're gonna make $14 billion this year.

• 16:50Cal Newport?

  But then, like, another contract came in and like, well, if we add that to our times 28 or whatever, times 30, we're gonna get even more money. So these are like highly volatile projections. Typically, you would see a reliance on this type of extrapolated earnings in like a very early-stage startup where like, look, we're new. We can't tell you how much we made last year because we weren't around last year, but we've made this much this year and here's what we think we're gonna make. It's a little bit unusual for Anthropic, which has been around since 2023, to still be doing this type of reporting and to still be largely hiding their actual revenue numbers.

• 17:24Cal Newport?

  So what they don't do is report these revenue run weights during a slow month or that number be very low, but if they have a good month, they tout it. And then if the month gets even better, they'll tout it again. So it's not like there's something illegal going on here, but it is very suspect that the companies are not wanting to talk about their actual revenue and just keep trying to talk about these best case projections because they've taken on a lot of money, they've spent a lot of money, it costs a lot of money to run them, and this is worrisome to investors, and they would rather you not pay attention to it. This goes back to what I've been talking about with some of these vibe reported articles where people, reporters have been saying, what possible motivation could someone like Dario Amade, the person who knows this technology best, what possible

• 18:08Cal Newport?

  motivation could he have to be saying, I'm worried that this technology is going to take away all the jobs? This is the motivation. They've only made $5 billion against $10 billion train spend and God knows how much inference spend and 60 billion investment revenue over their entire existence. It would rather you think that this is a company that's going to automate all the jobs than instead have you say, I just did subtraction and you're way in the red. So I think it's important to look at those numbers. It doesn't mean that they're not going to be, you know, maybe they will make 19 billion this year. Maybe things are going to get much better, but we gotta be much more careful about the economic story here and not allow them to do the Wizard of Oz, big burning face in

• 18:48Cal Newport?

  front of the curtain thing to distract us from what's actually happening back behind. So what I want to do here to try to balance things out, here's I'm going to end the show today. I want to read to you a take from someone who is way more AI critical and skeptical than I am. I mean, I, I have a lot of skepticism, but I also think it's an interesting technology that is going to make impacts, but we just have to cover it soberly and properly, strip off the hype and fear so we can figure out what's actually going on and react appropriately. That's my approach, but there are people out there that, man, they don't like these guys. And one of those people is Cory Doctorow, who wrote an essay recently for his blog.

• 19:30Cal Newport?

  That's called, I think it's called like Three AI Psychoses, or Three More AI Psychoses, where he really takes a swing at this financial picture as being sort of dire. Now, why do I want to read a take from a really strong anti-AI skeptic? It's because so much of the coverage that's out there is super hype. And I want to balance it. So I think it's actually worth. You've heard people that are way more hyped about this than I am. Now I want to read someone who's even more skeptical about this than I am, because I want to try to balance these things out. I think we need more voices of these sort of super skeptics out there. I would put Ed Zitron in this category. I would kind of put Gary Marcus in this category. He's very skeptical of LLMs and the current companies,

• 20:11Cal Newport?

  though very bullish on new technologies that are coming along soon. So I'm going to read to you from Cory Doctorow. This is my sort of fair balance. Fair and balanced AI coverage. I'm trying to balance out some of the like the hyperbolic stuff we've been reading recently. All right, so here's Cory Doctorow's take on the financial situation of the AI companies. AI is a terrible economic phenomenon. It has lost more money than any other project in human history, six to 700 billion dollars and counting, with trillions more demanded by the likes of OpenAI Sam Altman. AI's core assets, data centers and GPUs, last two to three years, though AI bosses insist on depreciating them over five years, which is unequivocal accounting fraud,

• 20:54Cal Newport?

  a way to obscure the losses the companies are incurring. But it doesn't actually matter whether the assets need to be replaced every two years, every three years, or every five years, because all the AI companies combined are claiming no more than 60 billion dollars a year in revenue, and that number itself is grossly inflated. You can't reach the 700 billion dollar break even point at 60 billion dollars a year in two years, three years, or five years. Now, some exceptionally valuable technologies have attained profitability after an extraordinary long period in which they lost money, like the web itself. But these turnaround stories all share a common trait. They had good unit economics. Every time a user logged onto the web, they made the industry more profitable.

• 21:37Cal Newport?

  Every generation of web technology was more profitable than the last. Contrast this with AI. Every user, paid or unpaid, that an AI company signs up costs them money. Every time that user logs into a chatbot or enters a prompt, the company loses more money. The more a user uses an AI product, the more money that product loses. And each generation of AI tech losses loses more money than the generation that preceded it. Here's what's important about reading that stronger skepticism. It's like, that's a very compelling argument. You see, you can make compelling arguments on both sides. You've heard very compelling arguments that make you feel like, well, this technology's about to run everything within a few months. But you hear a compelling writer like Dr. O who knows his stuff saying like, this economically

• 22:20Cal Newport?

  is going to fall apart within a year, that's equally as compelling. Which tells us, just because something compels you doesn't necessarily mean that it's completely right. We need to go into thinking about AI with care. There's the real tech story here, normal technology, fits and starts trying to find its niches, struggling, having breakthroughs, different innovations happening. And then there's the hype above it, which is either dystopian or super hypie. We gotta just get that layer off of it, so we could actually cover this like normal technology. And I've given all the reasons why. Like, we don't want it, don't want people to get away with crashing the stock market. We don't want bosses to get away with acting in ways that are anti-worker, disingenuous, and AI wash it.

• 23:03Cal Newport?

  We don't want, you know, societal or economic harms to be covered by a blanket of like, this is inevitable and the most important thing ever. We need to cover this like a normal technology. So, is the AI industry going to go bankrupt within another year? I don't know, I'm not an economist. But what I think should be clear by hearing both sides of this is like, this is a murkier, more careful picture. So let's put on our realistic glasses, and let's look at the actual stories here as carefully as we can. Alright, so that's it for this week. Until next time, remember, take AI seriously, but not everything that's said about it.