(Preview) 72 Hours of DeepSeek Hysteria, What DeepSeek Means for Big Tech, Lessons on the Efficacy of Chip Controls

Unpacking several days of dizzying reactions to DeepSeek, including a closer look at the costs of model development, why the heightened scrutiny looks like a coping mechanism, DeepSeek’s efficiency breakthroughs, the implications for big tech, and the future of export controls on semiconductors.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Andrew Sharp?

  Hello and welcome to a free preview of Sharp Tech.

• 0:08Andrew Sharp?

  And hopefully a better job by the commanders next year, but a hell of a ride. So I'm not mad about it. Um, on that note.

• 0:16Ben Thompson?

  We're, we're just mad about the Philadelphia Eagles.

• 0:19Andrew Sharp?

  It's really upsetting. Our good friend, Spike Esken, I spent six days talking trash to him and it really got out of hand in the second half there. So I'm ready to move on and let's get into the show. We'll begin with a question from Jeremy. He says, hi folks. Ben discussed DeepSeek's shockingly efficient ability to train models on the last podcast, but I haven't heard anyone discuss whether DeepSeek may simply be lying about the $5.5 million training cost. If they were forced to train their models in brute force fashion, as we originally assumed, I imagine they wouldn't want to report this and therefore indicate that the

• 1:00Andrew Sharp?

  chip ban was working, nor would they want people to know if they found a workaround to the ban, such as training through a US shell company or foreign data center. Theoretically, for example, if a CCP compelled TikTok to train a model with their US infrastructure, what would you expect them to say about the training cost in such a case? Would we have any way of knowing whether they were telling the truth? So I'm going to go back to the first line of Jeremy's email here. "DeepSeek's shockingly efficient ability to train models was discussed on the last pod, but I haven't heard anyone discuss whether DeepSeek may simply be lying."

• 1:38Ben Thompson?

  So Jeremy. I thought you wanted to focus on the fact that number one, we covered this last pod. Number two, I wrote about Entra Techery on Tuesday. It's funny that it's just become like this feverish sort of discussion over the weekend. I just want to say, hey, if you, if you read, if you were at Entra Techery, you were, you were on it early before everyone else.

• 1:56Andrew Sharp?

  Indeed. I mean, people, Jeremy sent that email on Thursday, I believe, and people then spent about 72 hours arguing about whether to trust the representations made by DeepSeek. Do you want to weigh in further on the cost question after a weekend of heightened DeepSeek scrutiny?

• 2:16Ben Thompson?

  Yeah, so the, um, I actually debated including this question when I wrote about it on Tuesday. Because Dylan Patel, who's kind of like the authority on what resources companies have sort of all over the world. Semianalysis.com, they have like whole model, actually, they make a lot of their money by selling these models that are much higher price in the subscription. That tells you down to the detail what all these data centers have in them. And he said back in November that they have like 50,000 H100s, uh, which are the more advanced, uh, or, you know, at that time, the most advanced Nvidia chip, which theoretically they're not supposed to have. But I would say the reason I didn't include it is

• 2:58Ben Thompson?

  at a high level, I think it's a moot point. And at a lower level, I think there's reason to believe them. But let's start with the chip band question. The chip band went into effect, so they claim they're using H800s, which, or HH100s, uh, which is the, or whatever.

• 3:13Andrew Sharp?

  H800s is the one that Nvidia developed as a workaround to continue selling into China. But it's not as performant as H100s.

• 3:23Ben Thompson?

  So the, the main limitation on the H800 is it has similar processing speed, but it has more limited memory bandwidth. So the initial version of the chip band targeted the memory bandwidth. And I thought at the time, this was a very clever sort of approach because you needed, uh, where you really need the, the memory bandwidth, the ability to link tons and tons of chips together is for training these, these very large models. And the, the breakthroughs that they talked about are actually explicitly about this point. Which is they figured out ways to have, to need sort of less overall memory, memory bandwidth. This gets into the, the, their sort of mixture of experts, uh, approach that I talked briefly about sort of on, on the last episode.

• 4:07Ben Thompson?

  Basically, the idea is you have a super large model, but most of the time, you're not using most of the model. You're only using some of the model. And so GPT-4, for example, was a mixture of experts model, but even there, their experts were fairly large. Uh, what DeepSeek has done is they have way more experts that's divided much more. And then they have a very clever approach to load balancing where they decide which experts to call. They have multiple copies of some of the most used experts so that you're not sort of constrained on the ones that are being used all the time. And and sort of this, a very clever sort of load balancing approach that means they need less sort of memory. At any given time, and this applies to to inference as well, to to sort of, as far as these models go.

• 4:51Ben Thompson?

  They also did a lot of very clever, very low level work to optimize the their communications layer. So they're actually, they went below CUDA to like near assembly language level to program. So you have all these shaders on a chip, all these GPU GPU units. And they have some number of them dedicated to managing communication. That is sort of independent and more fine grained than what you're sort of getting from Nvidia. Okay. And so it's a very, it's a very interesting approach. They have a bit in their paper about V3, basically, or it might have been V2, I can't remember, encouraging GPU makers. Look, look, we, you should actually do this for us. We shouldn't have to go to this low level to do it.

• 5:33Ben Thompson?

  And and basically encouraging GPU makers to change slightly how they architect their chips. This is legit stuff. It's super high level. And this is where the sort of capabilities of their team are real. Everything they put forward as to how they did this is very plausible. And is is very plausible that it was done using these chips that were not a violation of the chip band. So is it possible that they also have all these chips? Yes. Is this price, the price that it costs to create one of these models? No, like it doesn't include the R and D costs. It doesn't include all the runs they did to figure out this, this, the way to do this.

• 6:16Ben Thompson?

  All they calculated was the marginal cost of the final run that actually produced the model. Which, by the way, they were clear about in the paper. This isn't like you only need $5 million to make this model. It's the final run only cost, or 50 or $5 million. Which is, again, it is a plausible number. So I decided to not wade into it because what they do is plausible. And also, they're selling inference, which again, a lot of the things they're doing take advantage of these, particularly this mixture of experts approach. And there's another thing they're doing where. The, a big part of the challenge is we have to store all these parameters, particularly in this, in the context in what's called a key value store.

• 6:59Ben Thompson?

  And in your storing all these numbers that you have to have awareness of all the time. And so when I've talked about the challenges of inference, that it's really a memory issue. Like this is why like the, we talk about Apple's, Apple's chips are very compelling because they have this unified memory architecture. That means the amount of memory available to an Apple, Apple graphics chip and their, their neural processing unit is way higher than anything else. It's much higher than like if you get an Nvidia gaming GPU, for example. And so good memory and it's this, this key value store is a big thing. They figured out a way to, instead of storing the whole key value store, you actually store a compress, a compression of that, that basically represents a bunch of these multiplications.

• 7:41Ben Thompson?

  And you get about 90, 85, 90% of the accuracy, but it's good enough that lets you do way more with way less memory. And, and, and again, all these, these people are out there right now trying to reproduce that. And they've reproduced some of the steps, but it's all plausible.

• 8:01Andrew Sharp?

  Okay.

• 8:01Ben Thompson?

  And so the, the, is it possible that they have all these chips? Is Dylan right about that? I think probably yes. Like we know we have all these cells to Singapore of, of these chips that exceed the data capacity or the electric, electricity capacity of Singapore. They're not going to Singapore, right?

• 8:18Andrew Sharp?

  Yeah.

• 8:18Ben Thompson?

  They're probably going to China. So is China getting chips? Yes. Are they maybe using more powerful chips than they said? Yes. Are there real legitimate breakthroughs that make it plausible that they're, they're drastically more efficient than the other providers? Also, yes. And the other thing that, that goes into these comparisons is I just mentioned the inference bit. They are offering inference at very low prices. Now, are they being subsidized to, to make those prices much lower than they would be otherwise? Possibly, but they, they claim they're doing it profitably. It's plausible they are. And also the thing people are missing.

• 8:55Andrew Sharp?

  Whether they are or not is a big question though, right? I mean, that's why. There's now more attention on whether or not this story is actually what it appears to be.

• 9:07Ben Thompson?

  Well, I mean, the problem with doing the comparison points to OpenAI's API pricing and Anthropic's API pricing is OpenAI and Anthropic are, and this is not reporting, but as much as far as I understand it, making significant margin on inference, like software-type margins. So they're charging way more than their costs are. And so I think if DeepSeek for inference is just charging slightly above their marginal cost and OpenAI and Anthropic are charging way more than their inference costs. And by the way, what happens after this announcement? Suddenly, O1 Mini is going to be available to free users and it's going to be way more like.

• 9:49Ben Thompson?

  So they clearly have some latitude of pricing, like of margin, you know, that can be sort of given away. And so I don't think it's, I don't think it's fair to say that it's 30x more efficient or 45x more efficient because we don't know the true cost of Anthropic and OpenAI. I think it's definitely more efficient. It's probably not that much. And by the way, Google, which by the way, still exists, they came out with their Gemini sort of thinking model also this week, whose prices are actually comparable to DeepSeek. And we know Google has all, has very efficient, sort of a very efficient cost structure. They have all these TPUs, things on those lines. So is it plausible that they're lying? Yes, of course it's plausible.

• 10:32Ben Thompson?

  It's China, it's sort of, you know, and we can get into some like, I think the meta reactions in some respects more interesting than the details here, which again, Stratech covered on Tuesday. Like everyone here listening should already know, know most of this. But the, the, so the meta reaction is interesting, but the plausibility is there. And I actually don't think the, you know, I think there's a lot of cope going on, to be totally honest.

• 10:56Andrew Sharp?

  No, I definitely think people are now over correcting. And at least over the weekend, there were people on Twitter basically treating the idea that DeepSeek may be using 50,000 GPUs as evidence that all of this is overblown and not as significant.

• 11:12Ben Thompson?

  Well, here's the thing, if they're doing 50,000 GPUs, they still created this model, which is excellent.

• 11:16Andrew Sharp?

  Right.

• 11:17Ben Thompson?

  Like, like that's, that's, that's the thing. Now, here's the anti-DeepSeek point, which I think all the people that are going crazy about China has passed the US are missing.

• 11:30Andrew Sharp?

  All right, and that is the end of the free preview. If you'd like to hear more from Ben and I, there are links to subscribe in the show notes, or you can also go to sharptech.fm. Either option will get you access to a personalized feed that has all the shows we do every week, plus lots more great content from Strategery and the Strategery Plus bundle. Check it out, and if you've got feedback, please email us at [email protected].