Deep-dive into DeepSeek

There is crazy hype and a lot of confusion related to DeepSeek’s latest model DeepSeek R1. The products provided by DeepSeek (their version of a ChatGPT-like app) has exploded in popularity. However, ties to China have raised privacy and geopolitical concerns. In this episode, Chris and Daniel cut through the hype to talk about the model, privacy implications, running DeepSeek models securely, and what this signals for open models in 2025. Sponsors: Notion http://notion.com/changelog – Notion is a place where any team can write, plan, organize, and rediscover the joy of play. It’s a workspace designed not just for making progress, but getting inspired. Notion is for everyone — whether you’re a Fortune 500 company or freelance designer, starting a new startup or a student juggling classes and clubs. Domo https://ai.domo.com – The AI and data products platform. Strengthen your entire data journey with Domo’s AI and data products. Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Daniel’s blog post on DeepSeek https://predictionguard.com/deepseek-blogDeepSeek R1 on Hugging Face https://huggingface.co/collections/deepseek-ai/deepseek-r1-678e1e131c0169c0bc89728dDeepSeek https://www.deepseek.com/Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:03Speaker 00

  Welcome to Practical AI, the podcast that makes artificial intelligence practical, productive, and accessible to all. If you like this show, you will love the Change Log. It's news on Mondays, deep, technical interviews on Wednesdays, and on Fridays, an awesome talk show for your weekend enjoyment. Find us by searching for The Change Log wherever you get your podcasts. Thanks to our partners at fly.io. Launch your AI apps in five minutes or less. Learn how at fly.io.

• 0:44Daniel Whitenack?

  Well, welcome to the very first Fully Connected episode of Practical AI in 2025. In these Fully Connected episodes of the Practical AI Podcast, Chris and I keep you fully connected with everything that's happening in the AI world and hopefully share some learning resources to help you level up your machine learning and AI game. I'm Daniel Whitenack. I'm CEO at Prediction Guard, and I'm joined, as always, by my co-host, Chris Benson, who is a Principal AI Research Engineer at Lockheed Martin. How are you doing, Chris?

• 1:20Speaker 00

  Doing good today, Daniel. It's a lot of interesting things happening out there in the AI world, and I love these conversations where we do these Fully Connected kind of deep dives into things that are of personal interest to you and me, which is how we choose them.

• 1:35Daniel Whitenack?

  And there are a lot of exciting things coming up. In these episodes, it's a little bit easier for us to kind of free-form talk about a few things. But for those listeners who have been seeing our logo for some time on the podcast feeds, just FYI, there'll be a change to that coming up, but no need to swap out our feed or anything. That should be good. We're still doing great things with the Change Log, and they've made a few changes to their shows and their lineup. Publishing them in different ways. They have a show about that if you want to learn about it, but we'll still be going strong and excited kind of for probably a much needed refresh.

• 2:20Daniel Whitenack?

  You know, I don't know. I think it's been like six and a half years or something, hasn't it, Chris?

• 2:25Chris Benson?

  It has been.

• 2:26Daniel Whitenack?

  We can change in six and a half years. I don't know.

• 2:29Chris Benson?

  Yeah. I mean, six and a half years in GPU time is a long time.

• 2:32Daniel Whitenack?

  That's expensive. Yeah. So yeah, just FYI, long time listeners to be aware, you might, might scroll through your podcast app, look for a, look for a new logo sometime in the near future. But yeah, I think obviously that's bigger news than DeepSeek, but I guess we can devote most of the episode to what is the story of our week, couple of weeks and who knows how long, which has been DeepSeek R1.

• 3:04Chris Benson?

  I know. This is, you know, I was thinking as we were saying that, speaking of GPU time, in this case, maybe a lot less GPU time.

• 3:12Daniel Whitenack?

  Yeah, a lot less, maybe an unqueer amount.

• 3:16Chris Benson?

  That's true. Good point. Cause they only talked about the final run that was successful in terms of the spend on it. So.

• 3:23Daniel Whitenack?

  Yeah.

• 3:24Chris Benson?

  But I guess we're getting ahead of ourselves for those who are not as familiar with it.

• 3:27Daniel Whitenack?

  Yeah. So for those, probably many of those that are listening to this particular episode have come across DeepSeek. But for those that have not seen anything, maybe you've been under, under a rock somewhere. Chris, what are, what are we talking about with DeepSeek?

• 3:45Chris Benson?

  Ah, so we have, there's a Chinese startup that we're talking about here that has released a large generative model, LLM, that is, I guess, and I'm going to gloss over some stuff right here just because we'll, we'll dive into the specifics, but is very highly performant. It's comparable to the best models that OpenAI has had out there. But the thing that's really rocked everybody's world is the fact that it was trained at much, mu-

• 4:20Chris Benson?

  Much less cost, at least the parts we know, which we'll dive into that that detail as we said. There's some things we know and there's some things we don't know. But it appears to have been achieved at a much, much lower cost than all of the competing models from anywhere in the world up to this point. And so, in short, the AI world, and I guess everybody outside the AI world that cares about this stuff is in this giant debate and conversation about the implications. Is it a big deal? Not such a big deal? Why is it a big deal? You know, is it overblown? And of course, Daniel and I are about to dive into all of that right now.

• 5:01Daniel Whitenack?

  Yeah.

• 5:02Chris Benson?

  There's, it's a target-rich environment, as we like to say in defense.

• 5:05Daniel Whitenack?

  Are they surveilling us while while we use the model?

• 5:08Chris Benson?

  Exactly.

• 5:09Daniel Whitenack?

  Could you, should you, might you run the model in all sorts of different ways? Yeah, there's tons of confusion around this, Chris, which is one interesting thing that hopefully, hopefully, after you've listened to this podcast, you're not more confused. We don't make that guarantee, but hopefully, hopefully, that's, that's the case. Yeah, it's interesting. So I think one of the stories around this, and there's multiple kind of narratives that we can go into here. So much to talk about. One of the narratives is around how some Chinese startup with a much lower budget

• 5:50Daniel Whitenack?

  or spend on the model building, built such a good model and essentially gets parody to models from OpenAI and others. In particular, the comparison has been made to the O1 model, which if you remember, we talked about this on the show. This is OpenAI's sort of thinking, quote unquote, model. So the model, when it generates output, so you put in a prompt, the LLM generates text output. A beginning portion of that text is sort of thinking content, meaning they're training the model to sort of spit out logic of how to solve maybe a deeper problem

• 6:32Daniel Whitenack?

  or reason about the the input prompt before it actually gives its final answer. If you're in the ChatGPT interface, you can see this kind of in a different color or grayed out. If you're using the API, I don't think that they send that back in the API. You still pay for it, but I don't think they send it back. This is a similar type of reasoning model. So DeepSeek R1. And in this reasoning, this kind of very kind of flagship model of OpenAI, for example, the same kind of task. DeepSeek is kind of getting this, what we could call parity. Now, you know, different benchmarks are out there, et cetera.

• 7:15Daniel Whitenack?

  And, you know, each model has its own biases and different behavior. But yeah, the first kind of narrative around this in the news is, whoa, this came out of nowhere. There's this new company. It's just a startup. They did this on the cheap. So they kind of published numbers around 5 million, 5.5, 6 million, somewhere in that range for the, for the final training of this model. And compared to what it took to, to train OpenAI's O1, that's like a drop in the bucket. So yeah, this, this first narrative, what, what are your thoughts on this, Chris?

• 7:54Chris Benson?

  Uh, well, I think that there's a lot more information we want. You know, you know, they published that final number, as you mentioned, but there, I've seen a lot of posting about, you know, what, what did it take? What were all the unsuccessful runs that they had? The experimental runs, things like that. There's, there's just so much that's not known about this. So they, they really cherry picked what they chose to, to publish about it.

• 8:18Daniel Whitenack?

  But I mean, have you used it, Chris?

• 8:21Chris Benson?

  Oh, I actually, so surprisingly, uh, you know, I, because of, you know, my job, I tend to avoid some of the, the Chinese technologies in a direct use way, but I did load it onto my personal system. I have it on now. They were having some struggles on login up for the last hour at least that I've been trying and got it on. But yeah, I have it up. I know that, uh, I coyly sent you a text, uh, yesterday with a screenshot of me using it where I was just, just playing with it. And I'd seen somebody else do this. And so I asked it, what happened in Tiananmen Square in 1989? And it replied, I am sorry, I cannot answer that question. I'm an AI assistant designed to provide helpful and harmless responses. Which I thought was so, you know, that was about what I expected, actually.

• 9:06Daniel Whitenack?

  Yeah.

• 9:07Chris Benson?

  To be perfectly honest. But just a reminder of the geopolitics of AI, you know, which definitely plays big is that this is a Chinese government approved data set.

• 9:16Daniel Whitenack?

  And so. And, and I think there's to that point. So first off, DeepSeek is an amazing team of people that actually just didn't show up on the scene like a week ago.

• 9:28Chris Benson?

  Yep.

• 9:29Daniel Whitenack?

  They've been doing actually really great open source and science work, you know, for, for a while now. So the DeepSeek team has been around, there's been previous DeepSeek models. They have released those models on Hugging Face, which to be fair to our US counterparts, OpenAI has not released their models openly on Hugging Face to where you can run them and do research with them and analyze them and generate outputs. So that's maybe a, a point to highlight. They, they have been open in that sense, but there's been some, I think, confusion around this concept of this very small team.

• 10:11Daniel Whitenack?

  Like, there's a couple, couple of guys in their bedroom with a gaming card in their gaming PC and they train this model that beat OpenAI. That's kind of the narrative that's being pumped around. In, in reality, they have access to tens of thousands of GPUs. I saw a post by Philip Schmidt, uh, who's at Hugging Face. Now, I don't know all of the details of, of where he gets certain information, so take this for what it's worth. But one of the things he said, similar to you, Chris, is the sort of five to six million dollar mark. That's kind of the final base model, no reinforcement learning, doesn't include smaller runs, doesn't include data generation,

• 10:56Daniel Whitenack?

  which is a key piece of this. So, um, we'll talk about that when we come to the model here in a second. But actually generating. The prompts for this model is a significant part of it. And the kind of RL training, as I, as I mentioned. So the costs were definitely greater. There were more resources that were brought into this. It's not a company that just sort of popped up in someone's bedroom. So some of that narrative is, you know, it, it's true in the sense that the company is small. They clearly were working under constraints and they did a very impressive thing.

• 11:36Daniel Whitenack?

  Working under computational constraints in the environment that they're working on. And they released the model openly in, in Hugging Face. So kudos to them on that. Some of the other narrative points around that, I think are a little bit fuzzy.

• 11:55Chris Benson?

  So why, I'm, I'm curious just on your, on kind of how your take on that is. Yeah. So kudos for, for being open source on Hugging Face. And, and it's, you know, and you and I, over the last year, year and a half have been predicting, you know, open source, uh, kind of over inevitably at some point, especially as things were plateauing a bit, that open source would inevitably, you know, have its impact in this way. And so I, I don't think, it was a question of, of when and not if. And so having said that, and seeing this, this particular group putting it on Hugging Face, why do you think that they declined to include all of the training information on how they got there? In terms of the, the cost? What do you think might be motive?

• 12:40Chris Benson?

  And I realize I'm asking a speculative question.

• 12:43Daniel Whitenack?

  It's interesting. I mean, I, I think one point to make there is it's not out of character for anyone that's posting these models. In the sense that like Meta's Llama 3.1, I mean, there's a. You see these model producers produce "technical papers", but these technical papers don't share details to where, in theory, you could reproduce this, right? And there's details about the data that they're not revealing. There's details about their process that they're not revealing, even when they release one of these models. And so it's not out of character for that to be how things happen. And so really, part of me is like, well, why would they be motivated to do that?

• 13:27Daniel Whitenack?

  It would be out of their philosophical commitment to open science, I think, would be the thing that would motivate them to do that, which certain parties do. So like Allen Institute for AI with their OMO models, et cetera, have made it a very conscious effort to be truly open in terms of data, process, model, assets, et cetera. That is definitely an exception that proves the rule. Is that the right phrase?

• 13:56Chris Benson?

  Yeah.

• 13:56Daniel Whitenack?

  Yeah.

• 13:57Chris Benson?

  I know what you're saying.

• 13:58Daniel Whitenack?

  Yeah, something like that. You get what I'm saying.

• 14:00Chris Benson?

  But there's an acknowledgement that in the larger field, that the technical papers that come out are as much marketing papers as they are in accomplishment papers as they are.

• 14:10Daniel Whitenack?

  I mean, there's certain elements that are interesting to know. Obviously, you can know the model architecture. You're running it, it's open. So there's things you can learn from that. I do think that this represents a kind of shock to the system where you and I have been saying, we're basically getting to parity with open models and closed frontier models. For all intents and purposes, for most enterprise use cases, we're already at parity. But in the general public's eyes, we definitely weren't. And to some degree, we weren't in terms of certain types of models and that sort of thing. I think this is definitely a shock to the general public's perception

• 14:54Daniel Whitenack?

  that there is model optionality out there. There's going to be a proliferation of these models from various different places, which then leads into natural discussions about where are these models coming from? And can I trust them? And How does it behave differently than what I'm used to using? And can I run it securely? All of those sort of things pop up and they popped up sort of immediately and captured a lot of, a lot of attention around that. I agree.

• 15:41Speaker 03

  What's up AI practitioners? Adam here from ChangeLog. Want to tell you about how much I love Notion. I know Daniel and Chris love Notion as well because we use Notion to organize everything. And behind the scenes here at change.org and CP.fm, we work with a lot of cool teams externally. And we create dashboards and workflows and operating systems essentially to work well with others outside of our domain. And the cool thing is, is Notion is so flexible that we could do anything with Notion. And the coolest thing I'm loving about Notion is their Notion AI. I can search across all my notes, all my docs, get context, get summaries. It's all AI powered, all inside my Notion, powered by all the content in my Notion.

• 16:26Speaker 03

  So I can work with external teams, internal teams, I can build workflows. And all this AI is really helping my team, my tools, my knowledge base be empowered to do our best work. And unlike other tools out there, you gotta jump from one thing to the next, to the next, and it's just not seamlessly integrated. Notion is seamlessly integrated, infinitely flexible, and it's beautiful. It's easy to use, mobile, desktop, web, shareable, web shareable. I mean, you name it, Notion can do it. And the fully integrated Notion AI helps us work faster, write better, think bigger, do tasks more efficiently. Things that would normally take us hours now takes us minutes, maybe even seconds in some cases. And yes, we are a small organization comparative to Fortune 500 companies, but they are used

• 17:10Speaker 03

  by over half of Fortune 500 companies. And teams that use Notion send less emails, they cancel more meetings, they save time searching for their work, and they reduce their spending on tools, which helps everyone. Everyone stay on the same page. Try Notion today for free when you go to notion.com/practicalai. That's all lowercase letters notion.com/practicalai to try the powerful, easy to use Notion AI today. And when you use our link, of course, you are supporting this show and we love that. Notion.com/practicalai.

• 17:56Daniel Whitenack?

  Well, Chris, I do want to get to some of the technical details that we know about, you know, what the model is and versions of the model, but maybe before that, it would be useful to address the elephant in the room, I guess, which is the security element of this or the cybersecurity, privacy issues related to this. So there's been

• 18:23Daniel Whitenack?

  the geopolitical elements around, oh, you know, is the US ahead? What does this say about dominance in the space? That's one thing. There's another thing, which is my company, which previously potentially had problems with pasting things into chat GPT that they weren't supposed to, like, you know, whatever it is, customer details or IP or whatever. Those kind of that shadow AI usage was already happening in companies, right? And people were concerned that their employees were pasting things into chat GPT. Well, now there's this sort of new player in the space. People are pulling that up because it's the the new amazing AI app.

• 19:07Daniel Whitenack?

  And it turns out that is run by a different company and that data is going to a different place and is being housed on, you know, Chinese servers. So there's, there's that element. So we need to kind of parse through that, but also this has produced a separate confusion from my perspective around.

• 19:29Daniel Whitenack?

  The quote "security of the model," is DeepSeek secure? I think that's like a very, um, we have to clarify what we mean when we say that question. Cause I don't know what you, how, I don't know the things you've seen, Chris. There's a lot of stuff out there that is not very helpful in this sense.

• 19:48Chris Benson?

  Yeah, a lot of fear, uncertainty and doubt, and some of it may be justified. Some of it may not be, probably a lot of it may not be. For me, the scarier thing is, is not the model itself. It's the infrastructure around the model where it's being housed. What external entities to the core data scientists at that, at that company have access to it. And I think that's where, that's where a lot of the concern is going to be. It's, um, if you have a separate, if you've downloaded it from Hugging Face and you're running it on your server. That's not to say that, that, uh, every facet of security is being accommodated, uh, certainly. But at least, uh, you've taken some of the issues potentially out of the security equation.

• 20:30Chris Benson?

  So, you know, I, I definitely, I have on my personal phone, I have the, the app, which is unusual for me, but knowing that we were going to do this, uh, and, and wanting to play with it a little bit. But I am, I am very wary of my, uh, of, of what I don't know about that at this point.

• 20:49Daniel Whitenack?

  Yeah. Yeah. So you drew out something really important, Chris. And, um, I actually, I wrote a blog post about this that I'll link in the show notes of this, this episode. If you're interested, you can take a look and it might be a good resource. If, if you hear your engineering management or people in your company, you know, hyping the fears around DeepSeek, you know, and, and maybe you want to use that in, in a secure environment. Maybe that would be a good tool that you could point them to. But all that to say, I think the main thing that I wanted to highlight in that was this element that you just described. So there's really two ways to access this model. So there's two ways to utilize DeepSeek R1.

• 21:34Daniel Whitenack?

  One is via a product offered by the DeepSeek company, which is a software product that you access and they host. This would be parallel to a lot of other software products like OpenAI Host ChatGPT. That is their product, which has a model interface embedded in it.

• 21:57Daniel Whitenack?

  But it is a product similar to like you using Airbnb, right? You go to Airbnb, you put in your personal information into Airbnb. They have certain terms and service that they hopefully follow. But you have no view into what's going on under the hood of Airbnb or ChatGPT or this DeepSeek AI product. And so it's really not the model in that case that is not secure, quote unquote, in terms of you putting data into it. It is the product built around that model. And it is very clear from the terms and service

• 22:40Daniel Whitenack?

  that DeepSeek has posted that they will gather all of your, well, I shouldn't do a blanket statement like that. They say exactly what they will, what they will get from you. But they're saving a lot of your personal data and information. They will use that for future model trainings. And that is housed on, you know, quote, servers in China. So that is the terms, that is the explicit terms and service. If you're okay with that, that's the usage of the product, right? Not the model.

• 23:11Chris Benson?

  It's funny that you bring that up because like most people in most software products that I use, I don't necessarily go through all the terms of service as carefully as I really should. And, you know, because we're always

• 23:25Daniel Whitenack?

  No one does.

• 23:25Chris Benson?

  No one does. But I will confess that when I was, when I was downloading the DeepSeek app and it brings that up in registration, I did. And I was horrified to read it. And I had already determined I was going to do that. I was using all personal stuff, nothing related to work, that kind of thing. But even so, not only did I kind of do a big swallow on, on bringing the app down, but it made me really think about the kinds of things that I would put into the interface very, very carefully given. And it's like you said, going around the product as opposed to the model.

• 23:58Daniel Whitenack?

  Yeah, yeah. So this is one access pattern, right? Access through the DeepSeek, either their mobile app or I think it's chat.deepseek.com, their chat interface similar, again, to similar to ChatGPT. And really, you should have some of the same related concerns with a ChatGPT Absolutely. or an Anthropic, as you would with DeepSeek, in the sense that you really want to know how your data will be used and what are the privacy considerations around that. This adds a new element in that it's a sort of foreign entity dealing with that data, right? So there's a different element of that. But that's not the model, that's the product.

• 24:40Daniel Whitenack?

  The model, which they have released on, again, on Hugging Face, is, and when we say model here, for those that haven't maybe been around the podcast for a while, when we talk about a model, there's sort of two elements of that. There's the code needed to actually run that model, sort of process your inputs and generate outputs. And then there's a set of parameters, a set of data that's loaded into that code, that parameterizes that code such that it can run. Both of those have been released, and in fact, actually, the code needed to run DeepSeek, now I'm going to make a clarification here. So the code needed to run some versions of DeepSeek is not even DeepSeek's code,

• 25:25Daniel Whitenack?

  at least if you're using the Hugging Face ecosystem. It's a, it's a software package called Transformers, which is open source. You can go look at every line on GitHub. It's maintained by thousands around the world. It's completely open and transparent. And so the code element isn't, you know, there's, that's being looked at and being developed. The, and the model is implemented in there. The data element is available on Hugging Face and has potentially its own concerns as you download that into your environment, which we can talk about here in a second. But both of those are open. You can download them and run them, even in, like, if I spin up a VM.

• 26:06Daniel Whitenack?

  Or some computer, I could download those assets, cut off that computer from the internet, both outbound and inbound, right? And run that model in complete isolation where no data goes to DeepSeek, no data goes to China. They're not sending and connecting to that computer, right? So just to make it uber clear here, that is the model. When we say model, that's what we mean. We don't mean the product. And that can be run, again, with considerations in a secure environment. Now, I should say the caveat here. I do believe as the time of this recording, the Transformers library hasn't been updated

• 26:51Daniel Whitenack?

  to support the full DeepSeek R1 architecture, which is actually very typical when a new model is released. Sometimes it's not always supported in the upstream Transformers, which means that there is remote third-party code that you have to load to run the full model, which is not true for all of the versions of the model. I expect that will change in a matter of, I don't know, maybe it's changing as I speak now. It'll happen fast, like days, weeks, whatever. That will be kind of merged into upstream and then that concern will kind of go away.

• 27:25Chris Benson?

  So just to kind of follow up on that thought, would it be fair, given what you just said, to say that if you were running it on the Transformers infrastructure and you did have disconnected inbound and outbound networking from it? Just to take all of those extraneous concerns out, would you, would you have any reservations about running it in a scenario where security was important?

• 27:53Daniel Whitenack?

  Yeah, so that removes the kind of phone home, my data is going to China.

• 27:59Chris Benson?

  Yep.

• 28:00Daniel Whitenack?

  Vulnerabilities around remote code execution or something on my computer. Those sorts of concerns are, I would say, taken care of. Now, there are, I mentioned the data files that you would be loading, those model parameters. There are insecure ways that those could be. Loaded in J Frog and others have shown vulnerabilities in that. Those are mostly taken care of by using the right model formats, which DeepSeek is doing. It's called safe tensors. If you want to look into it, you can. So I wouldn't have any reservations about that. Now that brings up a secondary question, though, which is another point of confusion. So I'm glad you brought it up. The secondary question is, okay, if I'm running this, it's not phoning home. My data is not going to DeepSeek or any foreign entity.

• 28:44Daniel Whitenack?

  If I'm running it in this secure way, are there other concerns that are unrelated to this sort of phone home? Privacy issue type of thing. And I think one of the things that you brought up before was potential biases.

• 28:59Chris Benson?

  Biases. And the original training set, yeah.

• 29:02Daniel Whitenack?

  In the model, right? So you brought up this example of asking about Tiananmen Square. Actually, I think because we also asked a similar question very. I don't know if this is actually fixed in the app as of the time of this recording. But at first, when you had asked that question in the DeepSeek product, the actual application, it would print out the full answer of like, it would actually answer what happened. And then it would all collapse and give you like a canned, like, sorry, I can't.

• 29:33Chris Benson?

  Right.

• 29:33Daniel Whitenack?

  Answer that. So based on that, I would know, or I would assume that the actual model that you would download and run in that secure environment or, you know, on your laptop and one of these kind of local hosting things, that model is not biased. In terms of that response, that model specifically is not, it's, you could get it to answer about Tiananmen Square.

• 29:58Chris Benson?

  Sure.

• 29:58Daniel Whitenack?

  It's a product decision similar to like when Gemini, you know, was trying to create diversity in their image output and generated some really interesting looking things. Or what ChatGPT or anyone does, when you send in a prompt, they inject stuff into that prompt. They do post processing. It's a product, right? You don't have visibility into any of that. And so they're doing obvious product things there to introduce artificial biases. Now, I do think that it is possible that in the sort of alignment, fine tuning process, DeepSeek. Had their own vision of how they wanted to align that model,

• 30:40Daniel Whitenack?

  which may not be malicious in any sort of way, or biased in weird political ways. It might just be their choice of how they wanted to bias that model. In other ways, maybe it is motivated by certain things. I don't know. But that model will have its own sort of biased behavior. The other thing that I think has been shown in a number of places, with Secure did a study of this, and it is a model that is also way more sensitive to prompt injection attacks than kind of the many other state of the art models, which produces another type of vulnerability at the application layer. So you've taken care of like the model hosting security issue,

• 31:25Daniel Whitenack?

  but all that to say, that doesn't mean at the actual use of the model or in the integration layer, you shouldn't still be asking relevant questions, which again, I highlight some of those things in the blog post if people are interested.

• 31:43Speaker 04

  Well, friends, AI is transforming how we do business, but we need AI solutions that are not only ambitious, but practical and adaptable too. That's where Domo's AI and data products platform comes into play. It's built for the challenges of today's AI landscape. With Domo, you and your team can channel AI and data into innovative uses that deliver measurable impact. While many companies focus on narrow applications or single model solutions, Domo's all-in-one platform is more robust with trustworthy AI results without having to overhaul your entire data infrastructure, secure AI agents that connect, prepare, and automate your workflows, helping you and your team to gain insights, receive alerts, and act with ease through guided apps tailored to your role,

• 32:28Speaker 04

  and the flexibility to choose which AI models you want to use. So Domo goes beyond productivity. It's designed to transform your processes, helping you make smarter and faster decisions that drive real growth. And it's all powered by Domo's trust, flexibility, and years of expertise in data and AI innovation. And of course, the best companies rely on Domo to make smarter decisions. See how Domo can unlock your data's full potential. Learn more at ai.domo.com. That's ai.domo.com.

• 33:04Daniel Whitenack?

  So Chris, there's also the element around this that we always like to do when we get into our deeper discussions around any particular model, which is what are the unique technical or architectural elements of this? What types of versions did they release? This actually might be very confusing for people when they see like DeepSeek R1, Distilled, Quin, 32B, right? There's a lot of words there that might not make sense. Jay Almar, who we love on the podcast and has been on the podcast, he runs this,

• 33:43Daniel Whitenack?

  has posted for years, a lot of great blog posts about kind of illustrated transformers and other things. As a learning resource that you might want to take away from this particular episode, he posted an illustrated DeepSeek R1 article, which goes through some of the details. What's interesting here, Chris, I don't know if you got through any of that, but the overall picture of how they did this fine tuning is fairly similar to, I think, how many people have been doing fine tuning for some time.

• 34:19Chris Benson?

  And I guess that one of the things I wanted to bring up on this is, I believe, correct me if I'm wrong, it was based on one of the Llama models, right?

• 34:30Daniel Whitenack?

  Well, the DeepSeek architecture has been around for some time and is a specific architecture. It's similar to the Llama architecture, like it is also involving layers upon layers of transformers.

• 34:45Chris Benson?

  Right.

• 34:46Daniel Whitenack?

  In terms of the exact architecture of this DeepSeek R1, it does involve mixture of experts' layers in the model. So there's layers and layers of transformer blocks in the architecture, and then these mixture of experts' blocks, which you might see people refer to like activated layers or parameters. These mixture of experts' layers don't always, you don't process the input through all elements of that layer of the model each time you run the model, which creates some efficiencies, both for inference and often for training purposes as well.

• 35:27Daniel Whitenack?

  But yes, it's a similar setup, some slight differences, which also kind of those slight differences are the reason why we mentioned earlier, likely you kind of have, at least currently, as we're recording this, you might have to import some third-party code to support the model in the upstream transformers, which is likely to change quickly.

• 35:48Chris Benson?

  How does that affect the fine tuning, though, in terms of how DeepSeek approached it versus maybe how Llama has been approached and stuff? Are there differences?

• 35:56Daniel Whitenack?

  Yeah.

• 35:56Chris Benson?

  Are you seeing a lot of similarity there? I know Sam Altman made a comment, and I'm not quoting him directly, but it was something to the effect of, once somebody else has already done something that you're basing on, it's a lot easier to do that. And that was his kind of minimization of what DeepSeek had done. I'm kind of curious, how does that affect this in terms of the fine tuning?

• 36:18Daniel Whitenack?

  I mean, the overall, like I say, the overall process, and when I say overall process, this is often kind of a pre-training step of very raw data that is completely unsupervised. A fine-tuning step, which is supervised, and maybe an additional fine-tuning step, which is like a preference tuning. That sort of overall picture of how the training is done seems to also be true here. This is like the overall picture of how they did that.

• 36:47Daniel Whitenack?

  Now, there are some unique elements of this in that they created this DeepSeek R1.0 model, which is kind of, and they use this interim reasoning model to actually help generate some of the data for that supervised fine-tuning step. So this is where, going back to the original discussion in our conversation, that 5 million number corresponds to maybe that final, or one of the final training steps, but not necessarily the data generation. So they used interim models.

• 37:24Daniel Whitenack?

  That they, you know, their intention wasn't to release, it doesn't perform great in terms of a general purpose model, but it might perform well to generate long chain of thought examples, like this reasoning, these reasoning examples to add into the training or the training data that's supervised fine tuning, which allows you to augment your fine tuning data, use less human resources to create that fine tuning. So I forget the exact figures, but Meta did spend a ton of money in terms of the data curation with human data labelers to create those data sets for, for the Llama models and probably still are. In this case,

• 38:05Daniel Whitenack?

  at least some of that data was this synthetic data that was, that was generated by this interim model. So that's kind of one interesting step of, of the process that maybe, you know, that maybe is relevant to some of the budget and efficiency considerations.

• 38:23Chris Benson?

  Sure. And maybe that's, you know, part, part of the motivation of leaving that out altogether is, you know, that wasn't a direct cost necessarily to them, or at least not in the, the way that it had been when it was originally manufactured.

• 38:34Daniel Whitenack?

  Yeah, yeah. So there's the, this DeepSeek R1 model, which again is, the architecture is not fundamentally different from architectures of what we've seen in the past. There were some creative things done in the training process, both that significant portion, assuming it was a significant portion of data that, which was synthetic or generated data. They also used some kind of automated processes and model-based processes to filter and curate that generated data to actually filter out good examples from kind of all of the candidate examples. So there were some very creative things in that data generation piece, but

• 39:19Daniel Whitenack?

  the other stages of this were not. Not fundamentally stages of training that we haven't been familiar with with other model releases. There are a number of model versions that have been released from from DeepSeek. So there's the DeepSeek R1, that sort of flagship, which is like 700 billion parameters or something. It's very large. You're going to need, at least at full precision, you're going to need many, many GPUs to run this. I think Philip from Hugging Face said like his, what he said was like 16x 80GB GPUs, like 16x H100s. Yeah.

• 40:01Daniel Whitenack?

  To give you context, I think an H100, well, if you have it up all the time at on-demand pricing and a cloud is going to run you something like 60 to 80 grand a month, something like that. So you need 16 of those to run the model, the full model in full precision with at least Nvidia GPUs. Then they've released other variants of this model. And that that full model is that mixture of experts model, which has the element of the kind of external or third-party code added into it. They've also released distilled versions of that model. So we can get into that here in a second, but just wanted to make clear kind of what the main model looked like.

• 40:46Daniel Whitenack?

  And these distilled versions of the model, if you go to Hugging Face, you can actually look at the collection from DeepSeek for DeepSeek R1. And what you'll see is a whole bunch of different DeepSeek models, which again is often a point of confusion with people, like what do all these things mean? So we've got DeepSeek R1. We have Distill Llama 70B, Distill Quin 32B, Distill Llama 8B, et cetera. These ones are really significant for people to maybe understand. So these models are what's called dense models.

• 41:29Daniel Whitenack?

  So they don't have the mixture of experts element. They run all of their parameters all the time and they're distilled models. So what DeepSeek did is they took Their flagship DeepSeek R1, they created their flagship model. And then they use the process of knowledge distillation to create smaller versions of that model that leverage the kind of power of the larger model in the following way. So we've talked about this on the show before, especially we had a, we had a episode with Noose Research, they talked about this a lot.

• 42:04Chris Benson?

  Yep.

• 42:05Daniel Whitenack?

  If you're curious, go and check that out. But you essentially use the larger model to generate a bunch of example outputs. And then you use those inputs and outputs that were generated by the larger model to then train a smaller model with very, very high quality data that boosts the performance of the smaller model beyond what you could get from a smaller model just from training it by scratch. So when it says DeepSeek R1 Distill Llama 70B, it's a distilled version of Llama 70B that use DeepSeek R1 to generate that synthetic data for the fine tuning process, which is

• 42:47Daniel Whitenack?

  great because they have versions of this model from between 1.5 billion up to 70 billion. It's great because actually, you know, especially the smaller ones, you could run it on your laptop. Certainly the, the kind of sweet spot ones around that 8 billion to, you know, 32 billion, those, those would take a card or a couple of cards, you could run them on. And so this gives more accessibility to these models. And of course, people have already proliferated from there with various other optimizations like GGUF and other ones that can run on, you know, your MacBook processors and that sort of thing. So just to clarify the kind of ecosystem around the, the models there, those are a

• 43:32Daniel Whitenack?

  couple of things to keep in mind.

• 43:34Chris Benson?

  Yeah, that's, that's useful to know. It's so a couple of big questions, you know, as we are starting to wind up here, they kind of to address, to pull out of the technical a little bit for a moment and kind of address the ecosystem at large, the, the AI community. What do you, what would you predict that DeepSeek now being here and how things wash out? Obviously, there's, there's, you know, the market reacted. I think they took half a, half a trillion dollars away from Nvidia, plus another half a trillion.

• 44:05Daniel Whitenack?

  Don't they have a lot of trillions of dollars?

• 44:06Chris Benson?

  They have a few, they have a few. And so, but, but that will probably, you know, stabilize out going here. The, what do you think that we're looking at over the months ahead? You know, beyond the, the day of, you know, market reacting today kind of thing and we're talking about. But as you look at six months out, nine months out, that kind of thing. What do you think the real impact of DeepSeek is going to be on the larger AI global community?

• 44:35Daniel Whitenack?

  Well, we've, we've been saying it for a while, but I think the wider business community has not realized this. And one of the kind of fallouts from this or the, the things that will shift, I think, are really people taking seriously that in the future, part of what your business needs to consider is model optionality, right? Sort of these GPT models kind of were king for a long time. But now like, you know, apparently if you got $5 million sitting around, you could create a best in class model. And, you know, what does that mean? It means all of these are going to proliferate very quickly. This is not the last of these types of models we will see. They will proliferate very quickly.

• 45:20Daniel Whitenack?

  And you having kind of model lock in quote unquote, like you built all of your AI functionality around this particular model, whether that be open or closed. That's not going to work out great for you in the long run, just because of the models changing. And so, building in this ability to swap models, to have control and configurability. I think that's one of the kind of trends there. The other one I would say is now that. You're considering bringing these models into your own infrastructure. Like there's parity with OpenAI in many respects. It brings up all of these questions that we were immediately prompted with, right? Like, well, if you bring that into your environment, what are the security concerns related to that?

• 46:05Daniel Whitenack?

  How can you run it robustly and reliably? What should you be monitoring in in production? Right? So it brings up all of these additional questions, which I think overall will be will be really good for people to consider because they're probably things they should have been considering for the past year as, as you know, so many things were built on kind of one model family. So yeah, those are a couple of thoughts. I don't know if you have additional ones.

• 46:31Chris Benson?

  I I've been speculating on whether, you know, as as valuations have been going up and up and up for all these different AI startups all over, you know, across the entire globe, and the budgets have just grown astronomically. Is this the moment where where at this point investors looking at it going, why do you need 100 million? Why don't we give you 5 million and see what you can do with it?

• 46:55Daniel Whitenack?

  Look what they did with it.

• 46:56Chris Benson?

  Look what they did with it and stuff. And so, you know, whether the cost of of of operations in AI startups is now going to going to affect. And if that's the case, and with potentially not everybody being able to be quite so productive with their 5 million, you know, what does that mean? Are we going to go through a little bit of a cleanup round in the AI startup world or whatever? Any thoughts on that as we finish up?

• 47:23Daniel Whitenack?

  Yeah, it's interesting. I mean, it definitely has an impact if you're kind of a model building type of startup, for sure. And we've seen other examples of this even last week, you know, hearing from Gen Mo and what they're doing with video generation models with a very small team and what they're able to accomplish. I think it definitely has implications on that side. I think it also has implications for those it sort of. Hopefully, I think people will start thinking about less the kind of model building ventures, which will be interesting, but also this is going to make model building

• 48:04Daniel Whitenack?

  and fine tuning even more accessible to kind of the enterprise, which will kind of fuel both tooling and infrastructure type of investments as well. Now, those I don't think will have the kind of inflated like, oh, I need $100 million to train a model sort of scenario. But yeah, we'll see how that shakes out.

• 48:27Chris Benson?

  I think in, I think in enterprise world out there, maybe as a final thought on this, I think that you're now going to see existing budgets, which are, which often for large companies, you know, are, are in the hundreds of millions of dollars, you know, where they're not AI specialty companies, but they, but it matters enough to have a big budget. The expectation that I've always used other models and we've built lots of infrastructure around that, maybe there's pressure now to actually go and work on specifically models for your business that you're creating because you have, you know, 5 million times X available to do that in the new way of thinking. So it may certainly change what the expectations are in enterprise world.

• 49:10Daniel Whitenack?

  What I think will be stressed in that case is the data curation and human involvement in that process.

• 49:18Chris Benson?

  Yes.

• 49:19Daniel Whitenack?

  That's not, I mean, there's clearly a big investment on that side here. So even if you spend 5 million on that, you know, final training, there's definitely a process that goes into that.

• 49:32Chris Benson?

  I agree.

• 49:32Daniel Whitenack?

  Yeah.

• 49:33Chris Benson?

  Good conversation today.

• 49:34Daniel Whitenack?

  Yeah, definitely, Chris. Definitely check out some links that we'll put in the show notes to various articles about this, both on the technical and the kind of more hyped geopolitical stuff. So check it out. Thanks for joining again. Great to talk, Chris.

• 49:50Chris Benson?

  Yep, absolutely. See you next week.

• 49:59Speaker 00

  All right, that is our show for this week. If you haven't checked out our Change Log newsletter, head to changelog.com/news. There you'll find 29 reasons, yes, 29 reasons, why you should subscribe. I'll tell you reason number 17, you might actually start looking forward to Mondays.

• 50:20Speaker 05

  Sounds like somebody's got a case of the Mondays.

• 50:23Speaker 00

  28 more reasons are waiting for you at changelog.com/news. Thanks again to our partners at fly.io, to Breakmaster Cylinder for the Beats, and to you for listening. That is all for now, but we'll talk to you again next time.