What The Hell Is DeepSeek?

In this episode, Ed Zitron explains how DeepSeek, a relatively-unknown Chinese model AI developer incubated in a hedge fund, has punctured the generative AI bubble, throwing the US startup scene (and markets) into disarray. --- LINKS: https://www.tinyurl.com/betterofflinelinks Newsletter: https://www.wheresyoured.at/ Reddit: https://www.reddit.com/r/BetterOffline/ Discord: chat.wheresyoured.at https://discord.com/invite/QUUQUP9szv Ed's Socials: https://twitter.com/edzitron https://www.instagram.com/edzitron https://bsky.app/profile/edzitron.com https://www.threads.net/@edzitron See omnystudio.com/listener https://omnystudio.com/listener for privacy information.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  This is an iHeart Podcast. Guaranteed Human. Empower is all about helping you invest well so you can go out and live a little. And their Free Empower Personal Dashboard shows you your full financial life in one place. You'll be able to see your net worth, monitor budgets, analyze your portfolio, set retirement goals, and more. Whether you're saving for something big, want to treat yourself, friends, or family, or meet long-term goals, Empower helps you see where you stand. You've worked hard for your money, and Empower can help you make the most of it. Download the Empower Personal Dashboard or visit empower.com. Not an Empower client paid or sponsored.

• 0:36Speaker 01

  This is Jacob Goldstein from What's Your Problem? Running a business is hard enough. Don't make it harder with a dozen apps that don't talk to each other. One for sales, another for inventory, a separate one for accounting. That's software overload. Odoo is the all-in-one platform that replaces them all. CRM, accounting, inventory, e-commerce, HR. Fully integrated, easy to use, and built to grow with your business. Thousands have already made the switch. Why not you? Try Odoo for free at O-D-O-O dot com. That's Odoo dot com.

• 1:12Speaker 02

  Running a small business takes everything you've got. But with Chase for Business, you're not alone. They bring together local support and a broad range of resources to more than 7 million customers. With a deep understanding of your day-to-day needs, they provide products and guidance built to help you thrive. Right now, earn $500 when you open a new Chase Business Complete Checking account for new business checking customers with qualifying activities. Offer expires January 21, 2027. Chase Business Complete Checking has the flexible tools you need to accept payments, make deposits, and manage your finances with confidence. Learn more at chase.com/podcastbizoffer. Chase, make more of what's yours.

• 1:57Speaker 02

  Fees may apply to Chase Business Complete Checking accounts. The $500 offer is available for new business checking accounts with qualifying activities through January 21, 2027. Eligibility and qualification requirements must be met. Additional restrictions may apply. Please speak with a business banker for more information. JPMorgan Chase Bank, N.A., Member FDIC.

• 2:22Speaker 03

  Call Zone Media

• 2:25Ed Zitron?

  Hello and welcome to Better Offline. I'm your host Ed Zitron.

• 2:39Ed Zitron?

  A lot of you have been getting in touch. Yes, you're getting your DeepSeek episode. In fact, this is the first of a two-parter. This will come out on Friday, which is when you're listening to this, and then it'll follow up on Monday. I apologize. I spent a lot of Monday writing this and also learning about a lot of this stuff in an attempt to distill it as best I could. This situation is extremely weird, and it's developing. And I think even when I put out this episode, there will be new parts of it that I have yet to really get to. I will do my absolute best to explain in these episodes both what is happening with DeepSeek, what it means, what they've built, and what it's going to do in the future. But let's begin.

• 3:21Ed Zitron?

  So as January came to a close, the entire generative AI industry found itself in a kind of chaos. In short, the recent AI bubble, and in particular the hundreds of billions of dollars being spent on it, hinged on this big idea that we need bigger models which are both trained and run on bigger and even larger GPUs, almost entirely sold by Nvidia. And in turn, they're based in bigger and bigger data centers owned by companies like Microsoft, Oracle, Amazon, and Google. Now, there was also this expectation that this would always be the case. Hubris within this industry is kind of part of the whole deal. And generative AI was always meant to be this way, at least for the American developers. It was always meant to be energy and compute hungry.

• 4:05Ed Zitron?

  Throwing entire zoos worth of animals and boiling lakes was necessary to do this. There was never any other way to do it. And I thought, at least I've thought for a while, that this was because they just, they tried to make them more efficient, but they couldn't. There was just something about transformer-based architecture, like the stuff that underpins ChatGPT, so the GPT model under ChatGPT, either. It wasn't the case, though. A Chinese artificial intelligence company that few people had really heard of called DeepSeek came along a few weeks ago with multiple models that aren't merely competitive with OpenAI's but actually undercut them in several meaningful ways. DeepSeek's models are both open source, which means that their source code and research

• 4:46Ed Zitron?

  is public, and they're significantly more efficient as well, as much as thirty times cheaper to run in the case of their reasoning model R1, which is competitive with OpenAI's O1, and fifteen or more times more efficient than GPT-4O. It's actually kind of crazy when you think about it, and as you're gonna hear, this whole thing has jokified me all over again. And what's crazy is that some of them can be distilled, which I'll get to later, and run on local devices like a laptop. It's kind of crazy. And as a result, the markets have kind of panicked because the entire narrative of the AI bubble has been that these models have to be expensive because they are the future, and that's why hyperscalers had to burn $200 billion in capital expenditures for infrastructure

• 5:31Ed Zitron?

  to support this wonderful boom, and specifically the ideas of OpenAI and Anthropic. The idea that there was another way to do this, that in fact we didn't need to spend all this money and that maybe we could find a more efficient way of doing it, well, that would require them to have another idea other than throw as much money at the problem as possible. Yeah, they just didn't consider it, it turns out. And now along has come this outsider that's upended the whole conventional understanding and perhaps even dethroned a member of America's tech royalty, Sam Altman, a man who has crafted, if not a cult of personality, some sort of public image of an unassailable visionary that will lead the vanguard in the biggest technological change since the internet. Yeah, he's wrong. He never was doing that.

• 6:16Ed Zitron?

  I've been saying it for a while. He's never been doing this. But DeepSeek isn't just an outsider. No, they're a company that's emerged as a side project from a tiny, tiny Chinese hedge fund, at least by the standards of hedge funds, like 5.5 billion dollars on the assets under management. And their founding team has nowhere near the level of fame and celebrity or even the accolades of Sam Altman. It's distinctly humiliating for everyone involved that isn't DeepSeek. And on top of all of that, DeepSeek's biggest, ugliest insult is that its model, DeepSeek R1, is competitive, like I said, with OpenAI's incredibly expensive O1 reasoning model, yet significantly, and I mean 96%, cheaper to run. And it can even be run locally, like I said. Speaking to a few developers I

• 7:01Ed Zitron?

  know, one was able to run DeepSeek's R1 model on their 2021 MacBook Pro with an M1 chip. That is a four-year-old computer, not a 30,000 GPU in sight. It's kind of crazy. Worse still, DeepSeek's models are made freely available to use with the source code published under the MIT tech license, along with the research on how they were made, although not the training data, which makes some people say it's not really open source. But for the sake of argument, I'm just going to say open source. And this means, by the way, that DeepSeek's models can be adapted and used for commercial use without the need for royalties or fees. Anyone can take this and build their own. It's kind of crazy. By contrast, OpenAI is anything but

• 7:43Ed Zitron?

  open, and its last LLM to be released under the MIT license was 2019's GPT-2. No, no, wait, wait, shit. Let me correct that. DeepSeek's biggest, ugliest secret is actually that it's obviously taking aim at every element of OpenAI's portfolio. As the company was already dominating headlines this week, it quietly dropped its Janus Pro 7B image generation and analysis model, which the company says outperforms both Stable Diffusion and OpenAI's Darly 3. And those are, by the way, image generation things. So you type in something like Garfield with boobs, and then out comes a Garfield with juicy cans. And that's probably the first time you hear that on the podcast, but probably not the last. And as with its other code...

• 8:26Ed Zitron?

  DeepSeek has made this freely available to both commercial and personal users alike, whereas OpenAI is largely paywall DALL-E 3. This is really, it's a truly crazy situation, and it's also this cynical vulgar version of David and Goliath, where a tech startup backed by a shadowy Chinese hedge fund with $8 billion under management is somehow the plucky upstart against the lumbering Los Ofish $150 billion startup backed by multiple public tech companies with a market capitalization of over $3 trillion. I realize, by the way, I said earlier $5.5 billion under management. This is why you check your notes in advance, but I'm not cutting it. This is fresh. I am inside a closet in New York. The content must flow. Anyway, DeepSeek's V3 model,

• 9:10Ed Zitron?

  which is comparable and competitive with both OpenAI's GPT-4O and Anthropic's Claude-Sonnet 3.5 models, which, by the way, has some reasoning features. Like I said, it's 53 times cheaper to run the R1 when using the company's own cloud services. And as mentioned earlier, said model is effectively free for anyone to use, locally or on their own cloud instances, and can be taken by any commercial enterprise and turned into a product of their own, should they desire to, say, compete with OpenAI, the loudest and most annoying startup of all time. In essence, DeepSeek, and I'll get into its background and the concerns people might have about its Chinese origins, released two models that performed competitively and even beat models from both OpenAI and Anthropic, undercut them in price,

• 9:54Ed Zitron?

  and then made them open, undermining not just the economics of the biggest generative AI companies, but laying bare exactly how they work. The magic's gone. There's no more voodoo inside Sam Altman's soul. It's all out there. And the last point is extremely important when it comes to OpenAI's reasoning model, which specifically hid its chain of thought for fear of these unsafe thoughts that might manipulate the customer. And then they added, slightly under their breath, that the actual reason they did it was a competitive advantage. Now, to explain what that means, when you make a request with OpenAI's O1 model, say, "Give me all the states with the letter R in them", it actually shows you like the thinking. And by the way, these things don't fucking think. They're computer bullshit. Like, they don't think at all.

• 10:39Ed Zitron?

  But I'm gonna use it just for this. So you see it say, "Okay, here are all the American states. Which ones have that letter? I'm checking all of those." It's effectively having a large language model check a large language model. Now, the thing is, the steps they were showing you were all cleaned up. They would look nice, they would be formatted nicely. DeepSeek's chain of thought is completely laid bare, which is very interesting because it really takes the wind out of OpenAI's sails. And on top of that, it allows you to see actually how these things think through things. Again, not really thinking, but still, you can see things about how large language models work that these companies didn't want you to have. On top of this, OpenAI's O1 model has something even shittier to it,

• 11:24Ed Zitron?

  which is, these chain of thought things all cost money. When you see it generate these thoughts, it's actually generating more thoughts than you see, because they're hiding the chain of thought. So OpenAI is just charging you an indeterminate amount of money, an insane amount of money, as I'll get to later. But nevertheless, you don't know what you're being charged for. You don't even know what's really going on under the hood. Or you could use DeepSeek. And let's be completely clear, by the way. OpenAI's literal only competitive advantage against Meta and Anthropic was its reasoning models, O1 and O3. And O3, by the way, is currently in a research preview and is mostly just more of the same. Although I mentioned earlier in the show that Anthropic's Claude Sonnet 3.5 has some reasoning features, they're comparatively more rudimentary than those in O1 and O3,

• 12:09Ed Zitron?

  and I'd argue, R1, which is DeepSeek's model. In an AI context, reasoning works by breaking down a prompt into a series of different steps with considerations of different approaches. Like I said earlier, effectively a large language model checking its own homework with no thinking involved, because, like I said, they do not think or know things. And OpenAI rushed to launch its O1 reasoning model last year because, and I quote Fortune from last October, Sam Orman was eager to prove to potential investors that in the company's latest funding round, that OpenAI remains at the forefront of AI development. And, as I've noted in my newsletter at the time, it was not particularly reliable, failing to accurately count the number of times the letter "R" appeared in the word "strawberry", which was the code name for O1.

• 12:54Ed Zitron?

  Very funny stuff. At this point, it's fairly obvious that OpenAI wasn't anywhere near the forefront of AI development. And now that its competitive advantage is effectively gone, there are genuine doubts about what comes next for the company. As I'll go into, there are many questionable parts of DeepSeek's story. Its funding, what GPUs it has, and how much it actually spent training these models. But what we definitively understand to be true is bad news for OpenAI. And I would argue every other large US tech firm that's jumped onto the generative AI bandwagon in the past few years.

• 13:34Speaker 00

  Empower is all about helping you invest well, so you can go out and live a little. And their free Empower Personal Dashboard shows you your full financial life in one place. You'll be able to see your net worth, monitor budgets, analyze your portfolio, set retirement goals, and more. Whether you're saving for something big, want to treat yourself, friends, or family, or meet long-term goals, Empower helps you see where you stand. You've worked hard for your money, and Empower can help you make the most of it. Download the Empower Personal Dashboard or visit empower.com. Not an Empower client paid or sponsored.

• 14:05Speaker 05

  OnDeck is built to back small businesses, like yours. Whether you're buying equipment, expanding your team, or bridging cash flow gaps, OnDeck's loans, up to $400,000, help make it happen fast. Rated A+ by the Better Business Bureau and earning thousands of five-star TrustPilot reviews, OnDeck delivers funding you can count on. Apply in minutes at ondeck.com. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 14:35Ed Zitron?

  All right, folks, I'm back talking to you about Quince, one of my favorite clothing brands. And I finally pulled the trigger and picked up one of their Cuban-curved chains to go with one of their pima cotton t-shirts. Going to be a good look for me? You'll find out. All of this will sit beneath one of Quince's responsible down-hooded parkas, a ridiculously warm and comfortable coat that I wear constantly in the fall and winter. It can actually really take a beating in the wind. It's wonderful, and I can't recommend it enough. Now, everything at Quince is priced 50 to 80% less than similar brands because they work directly with ethical factories and cut out the middlemen, so you can give luxury-quality pieces without the luxury price tag. I've been buying from them for years before they advertised, and I heartily recommend their stuff, even wearing one of their t-shirts as I speak.

• 15:17Ed Zitron?

  Give the gift of quality this holiday season. Download the Quince app for app-exclusive offers or go to quince.com. Get free shipping on your order and 365-day returns. Now available in Canada and the UK too. And when Quince asks you where you heard about them, let them know it was better offline. It's the best way to support the show.

• 15:41Ed Zitron?

  DeepSeek's models actually exist. They work, at least by the standards of hallucination pronolalimes that don't, at the risk of repeating myself, know anything. They've been independently verified to be competitive in performance, and they're magnitudes cheaper in price than those from both hyperscalers—Google's Gemini, Met's Llama, Amazon Q, and so on and so forth— and from those released by OpenAI and Anthropic. DeepSeek's models don't require massive new data centers. They run on GPUs currently used to run services like ChatGPT, and even work on more austere hardware, nor do they require an endless supply of bigger, faster Nvidia GPUs every single year to progress. The entire AI bubble was inflated based on the premise that these models were simply

• 16:23Ed Zitron?

  impossible to build without burning massive amounts of cash, straining the power grid and blowing past emissions goals, and that these costs were both necessary and really good because they'd lead to creating powerful AI, something that's yet to happen. And it's kind of obvious at this point that that wasn't true. Now the markets are sitting around, they're asking a very reasonable question: Shit, did we just waste $200 billion?

• 16:49Ed Zitron?

  Anyway, let's get into the nitty-gritty. What is DeepSeek? First of all, if you want to super-deep dive into what it is, I can't recommend Venture Beats right up enough. I'll link to it in the show notes, as I usually do. It's really good, and it goes into a lot more detail than I will, but here's the too-long-didn't-read for you. DeepSeek is a spin-off from a Chinese hedge fund called HiFlyer Quant. It's a relatively small and young company, and from its inception, it went big on algorithmic and AI-driven trading. Later, it started building its own standalone chatbots, including a ChatGPT equivalent for the Chinese market. This is what we know right now. I'm sure some of you will say, "Oh, well, who knows if that's really true?" Sure, I think that that's fair. I also think that there are parts of Sam Altman's legend that we should question as well.

• 17:34Ed Zitron?

  I think the circumstances under which Sam Altman got made head of Y Combinator are extremely questionable. I'm saying you can question DeepSeek, and indeed you should. We should be more critical of these powerful companies. But don't do it halfway. If we're going to be worried, let's be worried about everyone. Now, DeepSeek did a few things differently, like open-sourcing its models, although it likely built upon tech from other companies like MetasLama and the ML library PyTorch. To train its models, it secured over 10,000 Nvidia GPUs right before the US-imposed export restrictions, which sounds like a lot, but it's a fraction of what the big AI labs like Google, OpenAI, and Anthropic have to play with. I think I've heard estimates of like 100,000 to 300,000 each, if not more.

• 18:18Ed Zitron?

  Now, you've likely seen or heard that DeepSeek trained its latest model for $5.6 million, as opposed to the insane amounts that I'll get to later. And I want to be clear that any and all mentions of this number are estimates. In fact, the provenance of the $5.58 million number appears to be a citation of a post made by an Nvidia engineer in an article from the South China Morning Post, which links to another article from the South China Morning Post, which simply states that "DeepSeek V3 comes with 671 billion parameters and was trained in around two months at the cost of $5.58 million", with no additional citations of any kind. So you should take it with a pinch of salt, but it's not totally ludicrous. Well, there are some that have estimated the cost.

• 19:02Ed Zitron?

  DeepSeek's V3 model was allegedly trained using 2048 Nvidia H800 GPUs, according to its paper. And Ben Thompson of Strategery has made this clear that the $5.5 million number only covers the literal training cost of the official training run. And this is made fairly clear in the paper, by the way, of V3. And that's the one that's competitive with OpenAI's GPT-4O model. Meaning that any costs related to prior research or experiments on how to build the model were left out. Now, big shout out to Minimaxir, the guy on Blue Sky and Twitter, he's great. He is wonderful and also added that this is fairly standard for the industry. Again, you choose how you feel about this, but I want to give you the information. And while it's safe to say that DeepSeek's models are cheaper to train, the actual costs,

• 19:47Ed Zitron?

  especially as DeepSeek doesn't share its training data, which some might argue means its models are not really open source, as I said, eh, the numbers get a little harder to guess at. Thompson notes that DeepSeek had to craft a bunch of elegant workarounds to make the model perform, including writing code that ultimately changed how GPUs actually communicated with each other. This functionality isn't otherwise possible using Nvidia's developer tools. They really had to get in there, it's kind of cool. DeepSeek's models, V3 and R1, are more efficient and as a result cheaper to run and can be accessed via its API at prices that are astronomically cheaper than OpenAI's. DeepSeek Chat, running DeepSeek's GPT-4O competitive V3 model, costs 0.07 cents per

• 20:29Ed Zitron?

  1 million input tokens, as in commands given to the model, and $1.110 per 1 million output tokens, as in the resulting output from the model. I know that these numbers kind of like just sound like numbers, like maybe you don't have context, so let me give you some. This is a dramatic price drop from the $2.50 per 1 million input tokens and $10 per 1 million output tokens that OpenAI charges for GPT-4O. This isn't just undercutting, this is, this is a bunker buster. If, now, there is a side that I'll kind of get into a little bit later in that you are using models hosted in a country that you don't know, probably China, there are data concerns. But again, you can put this on your own server.

• 21:14Ed Zitron?

  You could put this in Google Cloud. Both Microsoft and Google are apparently thinking about it. Now, the information reported that Google had added it to Google Cloud. No, they did not. They didn't do that. They allowed you to connect Hugging Face. This is a whole bunch of technical stuff that if you understand, you'd be like, "Yeah, Ed, I know." Long story short, the hyperscalers are already bringing DeepSeek out. And I'll get to why that's bad later in detail, but it's also very funny. Now, here's something else that's funny. DeepSeek Reasoner, its reasoning model, costs $0.55 per 1 million input tokens and $2.19 per 1 million output tokens. Now, that sounds expensive, maybe it is, whatever. That's goddamn nothing compared to the $15 per 1 million input tokens and $60 per 1

• 22:00Ed Zitron?

  million output tokens of OpenAI. Woof. If I'm Sam Altman, I'm shitting myself. Ugh. But there's an obvious bar here. We do not know where DeepSeek is hosting its models. Who has access to that data or where that data is coming from or going to? We don't know who funds DeepSeek, other than it's connected to HiFly, the hedge fund that I mentioned earlier that it split from in 2023. There are concerns that DeepSeek could be state-funded and that DeepSeek's low prices are a kind of geopolitical weapon, breaking the back of the generative AI industry in America. I'm not really sure whether that's the case or not. It's certainly true that China has long treated AI as a strategic part of its national industrial policy and is reported to help companies in sectors where it wants to catch up with the Western world.

• 22:43Ed Zitron?

  The "Made in China 2025 Initiative" saw a reported hundreds of billions of dollars provided to Chinese firms working in industries like chip-making, aviation, and, yeah, AI. The extent of that support isn't exactly transparent, surprise, surprise, and so it's not entirely out of the realm of possibility that DeepSeek is also the recipient of state aid. The good news is that we're going to find out fairly quickly. American AI infrastructure company Grok is already bringing DeepSeek's model online, meaning that we'll get at least a very, some sort of confirmation of whether these prices are realistic or whether they're heavily subsidized by whoever it is that backs DeepSeek. It's also true that DeepSeek is owned in part by a hedge fund, which likely isn't short of cash to pump into them.

• 23:24Ed Zitron?

  But as an aside, given that OpenAI is the benefactor of billions of dollars of cloud compute credits and gets reduced pricing for Microsoft's Azure Cloud Services to run its actual models, it's a bit tough for them to complain about a rival being subsidized by a larger entity with the ability to absorb the costs of doing business, should that be the case. Same goes for Anthropic, by the way. And yes, I know Microsoft isn't a state, but with a market cap of $3.2 trillion and quarterly revenues larger than the combined GDPs of some EU and NATO nations, it's kind of the next best thing. But I digress. Whatever concerns there may be about malign Chinese influence are bordering on irrelevant, outside of the low prices, of course, offered by DeepSeek itself. And even that is speculative

• 24:07Ed Zitron?

  at this point. Once these models are hosted elsewhere and once DeepSeek's methods, which I'll get to in a little bit, are recreated—by the way, that's not really going to take very long—I believe we're going to see that these prices are indicative of how cheap these models are to run.

• 24:30Speaker 00

  Empower is all about helping you invest well, so you can go out and live a little. And their Free Empower Personal Dashboard shows you your full financial life in one place. You'll be able to see your net worth, monitor budgets, analyze your portfolio, set retirement goals, and more. Whether you're saving for something big, want to treat yourself, friends, or family, or meet long-term goals, Empower helps you see where you stand. You've worked hard for your money, and Empower can help you make the most of it. Download the Empower Personal Dashboard or visit empower.com. Not an Empower client paid or sponsored.

• 25:01Speaker 06

  Cash flow crunch? OnDeck's small business line of credit gives your business immediate access to funds—up to $200,000—right when you need it. Cover seasonal dips, manage payroll, restock inventory, or tackle unexpected expenses without missing a beat. With flexible draws, transparent pricing, and control over repayment, get funded quickly and confidently. Apply today at ondeck.com. Funds could be available as soon as tomorrow. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 25:35Ed Zitron?

  So you might be wondering: how the hell is this so much cheaper? And that's a bloody good question, and because I'm me, I have a hypothesis. I do not believe that the companies making these foundation models such as OpenAI and Anthropic have actually been incentivized to do more with less. And because their chummy little relationships with hyperscalers like Amazon, Google, and Microsoft were focused almost entirely on making the biggest, most hugest models possible using the biggest, even huger, aris chips. And because the absence of profitability didn't stop them from raising more money, well, they've never had to be fucking efficient, have they? They've never had to try. Maybe they should buy less avocado fucking toast anyway. Let me put it in simpler terms.

• 26:17Ed Zitron?

  Imagine living on $1,500 a month, and then imagine how you'd live on $150,000 a month, and that you have to, like Bruce's millions, spend as much of it as you can to complete a mission, a very simple mission: live. In the former example, you concern survival. You have a limited amount of money and must make it go as far as possible with real sacrifices to be made with every dollar you spend. If you want to have fun, you're gonna have to eat less potentially, or the food you eat will have to be cheaper. You have to live on a budget, you have to make decisions, and indeed, you might learn to cook at home. You might walk more. You might do things that will help you not spend all your money. In the latter example, where you have $150,000 a month that you must spend,

• 26:59Ed Zitron?

  you're incentivized to splurge, to lean into excess, to pursue this vague idea of "living your life." Your actions are dictated not by any existential threats or indeed any kind of future planning, but by whatever you perceive to be an opportunity to live. OpenAI and Anthropic are emblematic of what happens when survival takes a backseat to living. They have been incentivized by frothy venture capital and public markets desperate for the next big thing, the next big growth, to build bigger models and sell even bigger dreams, like Dario Amadei of Anthropic saying that your AI, and I quote, "could surpass almost all human beings and almost everything" shortly after 2027. I just want to take a fucking second. Journalists, if you're listening to this, stop fucking quoting this bullshit.

• 27:43Ed Zitron?

  Shit. Stop it. You're doing nothing. You are failing at your goddamn job. Every single time you quote this bullshit, this nonsense. Shortly after twenty twenty seven, what the fuck does that mean? Twenty twenty eight, twenty twenty nine, twenty thirty? What does surpassing humans and almost everything even mean? This shit doesn't work. This shit is not good. Oh my god. Anyway, back to the podcast, Ed. Calm down. Both OpenAI and Anthropic have effectively lived their existence with the infinite money cheat from The Sims. And I know some of you might say, by the way, it's not an infant money, just add, you go into the console, you get my point. And both companies have been bleeding billions of dollars a year after revenue. And that's, by the way, making billions of dollars and then still losing billions is insane.

• 28:28Ed Zitron?

  And they still operated as if money would never run out, because it, and it wouldn't. If they were actually worried about that happening, they would have certainly tried to do what DeepSeek has done. Except they didn't have to, because both of them had the endless cash and access to GPUs from either Microsoft, Amazon, or Google. And the Stargate thing is just, I will mention it later, just, long story short, they're not gonna put five hundred billion dollars into it. It was up to five hundred. I'm so tired of this shit. OpenAI and Anthropic have never been made to sweat, unlike me in this closet where I'm recording this. And they've received endless amount of free marketing from a tech and business media happy to print whatever vapid bullshit they spout.

• 29:09Ed Zitron?

  And it's just very frustrating. They've raised money at will with, and Anthropic, by the way, is currently raising another two billion dollars, valuing the company at sixty billion dollars. And this was, I think, happening while DeepSeek was going on, which is really funny. And they've done all of this off of a narrative of the, we need more money than any company has ever needed, ever. Because the things we're doing have to cost this much. There is no other way. You must give us more money. My name is Sam Altman. I need more money than has ever been made for my huge, beautiful company that sucks and needs money to train it. Help me, please. My big, beautiful, sick company is dying, but the best and most important company of all time. It's also normal.

• 29:51Ed Zitron?

  Now, do I think that they were aware that there were methods to make their models more efficient? Sure. OpenAI tried and failed in twenty twenty three to deliver a more efficient model to Microsoft called.

• 30:04Ed Zitron?

  Arrakis. I'm sure there are teams at both Anthropic and OpenAI that are specifically dedicated to making things kind of more efficient, but they didn't have to do it and so they didn't. And as I've written before in my newsletter and argued on this very podcast, OpenAI simply burns money and have been allowed to burn money and up until recently likely would have been allowed to burn even more money because everybody, all of the American model developers, appeared to agree that the only way to develop large language models was to make them as big as humanly possible and work out troublesome stuff like making them profitable or turning them into a useful thing later, which is, I presume, when AGI happens, a thing that they're still in the process of defining, let alone doing.

• 30:47Ed Zitron?

  DeepSeek, on the other hand, had to work out a way to make its own large language models within the constraints of the hamstrung Nvidia chips that can be legally sold to China. While there's a whole cottaged industry of selling chips in China using resellers and other parties to get restricted silicon into the country, the entire way in which DeepSeek went about developing its models suggests that it was working around very specific memory bandwidth constraints, meaning that the amount of data that could be fed into it and out of it and into the chips. In essence, doing more with less wasn't something it chose, but it's something they had to do. I've touched already on the technical how of these models in greater depth, and you can really read in that in my newsletter. And you can go to where'syourad.aa, it's at the end of the episode, but I'll also have

• 31:28Ed Zitron?

  show notes to articles like Ben Thompson's from Stratechery, because there are lots of things to read here. I know there are some really technical listeners, and I'm sure you're gonna flame me in my emails. Please go and read it. I'm not wrong. I've checked with a lot of people, too. And by the way, all of this austerity stuff seems to have worked. There's also the training data situation, and another mayor culpa. I previously discussed the concept of model collapse and how feeding synthetic data, which is training data created by a generative model, into another model can end up teaching it bad habits, which in turn would destroy the model. But it seems that DeepSeek has succeeded in training its models using generative data. Specifically, though, and I'm quoting geek-wise John Turo, like mathematics where correctness

• 32:11Ed Zitron?

  is unambiguous, and using, and I quote again, "highly efficient reward functions that could identify with..." Which new training examples would actually improve the model, avoiding wasted compute on redundant data? And it seems to have worked. Though model collapse may still be a possibility, this approach, extremely precise use of synthetic data, is in line with some of the defenses against model collapse I've heard from LLM developers I've talked to. This is also a situation where we don't know the exact training data, and it doesn't negate any of the previous points I've made about model collapse. Now, we'll see what happens there, but synthetic data might work where the output is something that you could figure out using a calculator. But when you get into anything a bit more fuzzy, like written text or anything with an element of analysis, you'll likely encounter some unhappy side effects.

• 32:56Ed Zitron?

  But I don't know if that's really going to change how good these things are. There's also a little scuttlebutt about where DeepSeek got its data. Ben Thompson at Strategie suggests that DeepSeek's models are potentially distilling other models' outputs, by which I mean having another model say Metas Llama or OpenAI's GPT-4O, which is why DeepSeek identified itself as ChatGPT at one point, spit out outputs specifically to train parts of DeepSeek. This obviously violates the Terms of Service of these tools, as OpenAI and its rivals would much rather have you not use its technology to create its next rival. And OpenAI, by the way, has recently reportedly found evidence that DeepSeek used OpenAI's models to train its rivals, and this is from the Financial Times.

• 33:39Ed Zitron?

  Although it failed to make any formal allegations, but it did say that using ChatGPT to train a competing model violates its Terms of Service, and David Sacks, the investor in Trump administration, AI and crypto czar, says it's possible that this occurred. Although he failed to provide evidence, I just want to say how fucking funny it is. The OpenAI is going, "Where? Where are you stealing my stuff? Don't steal my things! Where? Fucking coward, pansy bastard, bitches, fucking hell! What a, what a bunch of whiny babies! Oh no, my plagiarism machine got plagiarized! Where? Kiss my entire asshole, Sam Altman, you little worm! You fucking embarrassment to Silicon Valley! You should be ashamed of yourself for many reasons! But so much this, though!

• 34:25Ed Zitron?

  Where? Oh no, your stuff! My plagiarism machine that requires me to steal from literally every artist and author on the internet. The thing where we went on YouTube and transcribed everything and fed it into the machine. That's that's not stealing. That's good. But you using our model to generate answers. That's just not fair. What a bunch of babies. You guys, Sam Altman's worth billions of dollars. He has a five million dollar car. Cry more, you little worm. Personally, I genuinely want OpenAI to point a finger at DeepSeek and accuse it of IP theft. Mostly for the yucks, but also for the hypocrisy factor. This is a company that, as I've just very cleanly said, exists purely from the wholesale industrial larceny of content produced by literally a fucking everyone.

• 35:09Ed Zitron?

  And now they're crying, "Wheee, I'm Sam Altman. I'm a big baby. I filled my diaper because someone stole from my plagiarism machine. Kiss my ass." Kiss my ass. These companies haven't got shit. OpenAI doesn't have shit. They don't have anything. They don't have a next product. Without reasoning, they haven't got anything. And now they don't have that disgusting justification. That overspending, the fat, ugly, American startup culture, spending as much as you can to build America's next top monopoly. They should be fucking ashamed of themselves. They shouldn't be billionaires. They should be poverty stricken. They should have to pay everyone they stole from.

• 35:51Ed Zitron?

  And it's just, it sickens me seeing the reaction from some people on this, seeing the cynophobia, but seeing this level of defensiveness of a company like OpenAI or Anthropic. And as I'll get into next episode, we are really running out of time here. And I think DeepSeek is really, I think it could be really the end of days for these companies. I don't know how much they've got left, time-wise or even money-wise, and I'm not sure how they even raise money. But in the next episode, I'm gonna deep dive into DeepSeek. And I'll tell you how they sent the US tech market into a panic and what it actually means for the future of OpenAI, Anthropic, and the hyperscalers backing them. This has been a crazy few days. I hope this has helped. And on Monday, you'll find out more.

• 36:36Ed Zitron?

  Thank you so much for listening. The support I've got for the show has been incredible. And the emails I've got about DeepSeek. Uh, I've been trying, okay? I've really been trying. It's the fastest I could do it. But I'm so happy to do this show and I'm so grateful for all of you.

• 36:59Ed Zitron?

  Thank you for listening to Better Offline. The editor and composer of the Better Offline theme song is Matt Osowski. You can check out more of his music and audio projects at mattosowski.com. M A T T O S O W S K I dot com. You can email me at [email protected] or visit betteroffline.com to find more podcast links and of course my newsletter. I also really recommend you go to chat dot where's your ed dot app to visit the discord and go to r slash better offline to check out our reddit. Thank you so much for listening.

• 37:31Speaker 03

  Better Offline is a production of Cool Zone Media. For more from Cool Zone Media, visit our website, coolzonemedia.com, or check us out on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

• 38:06Speaker 00

  Empower is all about helping you invest well so you can go out and live a little. And their Free Empower Personal Dashboard shows you your full financial life in one place. You'll be able to see your net worth, monitor budgets, analyze your portfolio, set retirement goals, and more. Whether you're saving for something big, want to treat yourself, friends, or family, or meet long-term goals, Empower helps you see where you stand. You've worked hard for your money, and Empower can help you make the most of it. Download the Empower Personal Dashboard or visit empower.com. Not an Empower client paid or sponsored.

• 38:37Speaker 07

  Cool fall nights always make me want to entertain more. A few friends around the table, maybe a fire outside when the sun goes down. Before everyone comes over, I stop at Total Wine & More. Right now, there's a cranberry-flavored wine, a crisp apple wine, even a pumpkin spice cream that's perfect in coffee after dinner. And with the lowest prices, I can bring home a few different fall flavors for everyone to try. Find your favorite fall flavors at the lowest prices at Total Wine & More. Spirits are not sold in Virginia and North Carolina. Drink responsibly. Must be 21.

• 39:11Speaker 08

  Small businesses are the pulse of every community. They bring people together, create opportunities, and drive growth. That's why Chase for Business helps small business owners like you by offering personalized guidance and digital tools that can help your business grow. With our guidance and your determination, you can take your business farther and help build a brighter future for your community. Learn more at chase.com/business. Chase for Business, make more of what's yours. The Chase mobile app is available for select mobile devices. Message and data rates may apply. JPMorgan Chase Bank, N.A., member FDIC. Copyright 2026 JPMorgan Chase and Company. This is an iHeart Podcast. Guaranteed Human.