DeepSeek R1 - Everything you need to know

Greg's DeepSeek Cheetsheet: From Installation to Expert Prompting: https://www.gregisenberg.com/deepseek Ray Fernando, a former Apple engineer, gives an in-depth tutorial on DeepSeek AI and local model implementation. The conversation includes detailed guide for setting up local AI environments using Docker and OpenWebUI, and how to implement DeepSeek on mobile using the Apollo app. Key Points: • Comprehensive overview of DeepSeek AI and its reasoning capabilities • Discussion of data privacy concerns when using Chinese-hosted models • Detailed tutorial on running AI models locally using Docker and OpenWebUI • Comparison of different AI model providers (Fireworks, Grok, OpenRouter) • Mobile implementation of AI models using Apollo app Timestamps: 00:00 - Introduction 02:34 - Overview of DeepSeek 05:29 - Data privacy concerns with Chinese-hosted models 08:02 - Running Models Locally with OpenWebUI 10:06 - Comparing Hosting Providers: Fireworks and Groq 16:54 - Cost Comparison of using/running AI Models 18:35 - Improving Prompts for Better Outputs 22:41 - Running AI Models Locally: A Step-by-Step Guide 37:09 - Mobile AI implementation discussion 45:27 - Future implications and closing thoughts 1) DeepSeek's R1 Model - What's the big deal? • On par with ChatGPT's reasoning capabilities • Open source & free to use • BUT hosted in China (data privacy concerns) • Incredible for analysis and content generation 2) Alternative Ways to Use DeepSeek: • Fireworks AI ($8/million tokens) • OpenRouter • Grok API • Local hosting (safest for sensitive data) 3) Running AI Models Locally Step-by-step setup: • Install Docker • Use OpenWebUI • Download models via Ollama • Configure API connections Pro tip: Takes 5 mins to set up, saves hours of worry about data privacy 4) Mobile AI is HERE! • Apollo app lets you run models locally on phone • Download smaller, optimized models • Works offline • Perfect for quick analysis on the go 5) Temperature Settings Explained: High temp (0.8-1.0) = Creative mode Low temp (0-0.3) = Logical mode Choose based on your needs! 6) FUTURE PREDICTIONS • Watch-based AI coming soon • Emergency response applications • Real-time negotiation assistance • Advanced audio analysis 7) Getting Started Tips: • Start with public data on DeepSeek • Experiment with different models • Use temperature settings wisely • Focus on practical use cases Notable Quotes: "We're in this new deep-seek world where if you figure out the model that works for you and the tasks that you want to accomplish, you might be able to out-compete whoever you're competing against." - Greg "Please don't be fearful or don't feel like you're left behind. If you're just finding out about this, you're not that far behind. We're all actually still trying to understand what this intelligence can give us." - Ray Fernando LCA helps Fortune 500s and fast-growing startups build their future - from Warner Music to Fortnite to Dropbox. We turn 'what if' into reality with AI, apps, and next-gen products https://latecheckout.agency/ BoringAds — ads agency that will build you profitable ad campaigns http://boringads.com/ BoringMarketing — SEO agency and tools to get your organic customers http://boringmarketing.com/ Startup Empire - a membership for builders who want to build cash-flowing businesses https://www.startupempire.co FIND ME ON SOCIAL X/Twitter: https://twitter.com/gregisenberg Instagram: https://instagram.com/gregisenberg/ LinkedIn: https://www.linkedin.com/in/gisenberg/ FIND RAY ON SOCIAL X/Twitter: https://x.com/RayFernando1337 YouTube: https://www.youtube.com/@RayFernando1337 Ray's Website: https://www.rayfernando.ai

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Greg Isenberg?

  Ray Fernando on the pod. He's a 12 year ex Apple engineer. He streams AI coding. He's building an AI startup in real time. I needed to have you on because what are we going to talk about today, Ray?

• 0:18Speaker 01

  Today, we're going to talk about prompting and we're going to be specifically prompting with the new reasoning models with DeepSeek R1. And there are a lot of caveats as far as these models because they're now able to think and reason. And what that can do is can even lead to superhuman capabilities. And so what does that mean? Is that these models have now become so advanced and this specific one from DeepSeek is out of China. And what that allows you to do is basically they've made it open source so that it's available for us to study, but it's apparently also on par with ChatGPT's like 01 model, 01's reasoning models.

• 1:01Speaker 01

  And why it's taken the world by storm is because the fact that it's also free to use on their website, so deepseek.com.

• 1:17Speaker 01

  And when we say free, there's also a little bit of a caveat if you don't really know. So I just want to also cut and cover a little bit of the architecture today and explain kind of what you're going to get into if you use something like DeepSeek. And then maybe how you can also run this in something that can run like in a container like in North America or in some other area, because your data is really important, especially if you're doing anything for business. And then also the third kind of secret bonus there would be how to actually run this locally on your machine so you can get the capability of these models and you know, run that locally for your own private businesses, whether you're a lawyer, you're a doctor or whatever. There's a lot of different implications that you probably will want to look into.

• 2:00Speaker 01

  So I think that this episode is going to be super helpful if you're even just beginning and you don't really know some of the advanced stuff or no code. That's okay. It just takes learning English or using English to describe these things, to get the output and the intelligence of these models to do some really cool stuff. So I'm pretty excited.

• 2:18Greg Isenberg?

  All right, let's get into it.

• 2:20Speaker 01

  Cool. Excellent. To start out, to use these models, you have a couple of options. And one is going directly to deepseek.com. And this is actually currently hosted in China. So a little bit of background here is that your computer is here. Like, for example, I'm in North America. And if I go to deepseek.com or download the app from the App Store, the app will actually be talking to a region over in China. And for what it may be, whenever you send your data over to another country, they have their own rules and laws and regulations. So I would be very careful as far as anything you put into the system, as far as if you have any sensitive data, because it would not belong to a region that you may live in or have control in.

• 3:06Speaker 01

  There are other alternatives which we're going to cover, which would be using a web UI and going to these different API providers like Fireworks or Grok. And then we're also going to do something like covering, running something locally on your machine, so it doesn't go out to any of these providers. And you can even run this if you're flying on a plane, which is really exciting. So as an example for DeepSeek, we're just going to do this because this is something that's currently public information, and I don't really mind having this stuff sent out. So as far as prompting, one thing that I frequently do is I have a live stream, and I basically transcribe my videos and stuff. So I basically, you know, I made a little app that will transcribe videos. And I just take my live stream here and just run it through the transcriber there.

• 3:50Speaker 01

  And what it will do is just generate transcripts from the video. And it usually does it pretty fast. It processes on my device and then it sends it up to Grok for the endpoint. So when it's done, it looks basically something like this. And you're able to copy this transcript and put it into something like DeepSeek if you wanted to. And so in order for you to use the model, what you can do on deepseek.com is to just go ahead and click where it says "DeepSeek." So when it turns blue, that means the Deep Think is enabled. And that means it's also there. And if you wanted to enable web search, you can do that. So we can probably do that for the next prompt here. So I paste in my transcript. Here I hit shift and I hit enter a couple of times.

• 4:30Speaker 01

  And so what we can do is give it additional instructions for it to do what we want to do. And one of the things that I like to do is I actually have built a little prompt that I'll actually share with y'all so that you can actually do some analysis and generate a blog post up a transcript. And so that is actually located here in my little notion thing. And so one of the things I have is we're going to cover how to do some of these prompts and stuff. And I'll actually show you how you can generate some of these advanced chaining prompts, because this is will really take advantage of these models to think through all of that text and do some work on our behalf. So this is really, really cool. It's basically like hiring an admin to go through all of your stuff and make things for you.

• 5:13Speaker 01

  So we're going to go ahead and hit submit. And when we do that.

• 5:16Greg Isenberg?

  I will say, you know, I wouldn't put a tax return on deepseek.com. It's not the type of thing I would put on. So be, you know, you do want to be a bit wary of what you're putting on when you're on deepseek.com. Now, I was playing with perplexity earlier and perplexity actually has some of these models built in, but it's hosted in the United States of America. So that's, that's a bit different.

• 5:45Speaker 01

  That's correct. And there you may want to ask your app providers what they do. One of my favorite apps for coding is actually cursor. And I asked them, hey, where do you have have your deep seek model hosted? And they told me they use the fireworks API. And that's, you know, actually not in China. So that's great. So it's like, okay, cool. That's awesome. And they're using the full model.

• 6:06Greg Isenberg?

  Quick break in the pod to tell you a little bit about Startup Empire. So Startup Empire is my private membership where it's a bunch of people like me, like you, who want to build out their startup ideas. Now they're looking for content to help accelerate that. They're looking for potential co-founders. They're looking for tutorials from people like me to come in and tell them, how do you do email marketing? How do you build an audience? How do you go viral on Twitter? All these different things. That's exactly what Startup Empire is. And it's for people who want to start a startup but are looking for ideas. Or it's for people who have a startup, but just they're not seeing the traction that

• 6:50Greg Isenberg?

  they need. So you can check out the link to startupempire.co in the description.

• 6:57Speaker 01

  So these models have these parameters that you may hear of. And like, you know, like the really large parameter model, like 600 billion plus parameters just means that it has more intelligence to leverage. And it tends to take longer in its thinking, but the results are really, really, really great. And some of the models that will probably run locally on the machine a little bit later, they're going to be like distilled. So you just basically take the essence of it. And then those models basically are going to run, they run a lot faster and they're just as efficient, but they may not think as long or they may not give the results. And it's really up to you to try out, which I highly encourage. So one of the problems is that sometimes the server is really busy. And that can happen because right now it's so popular.

• 7:41Speaker 01

  And I probably after the publishing of this video, it'd probably be even more popular. So you can hit this little pencil and you can hit send again to try to resend it. And so that's kind of where I thought, well, if I'm sending this over and there's a bunch of reliability issues, why don't I try to do something like, you know, that I can host my own or just hit the API themselves? If Cursor is doing that, why can't I do it as well? So I can actually show you a technique for how to do this. So you can hit the API and so you don't send your data to China. And that actually involves using this thing called Open Web UI. So I can show you that. So while this is thinking, if it even returns a result or does anything here, we're going to go ahead and pop on over to the other side.

• 8:24Speaker 01

  So on the other side, we're going to go to here. And so I have an instance of what's called Open Web UI. And it looks very similar to like a ChatGPT. And to get this set up, I'll probably go through a little bit more details, but I'll just go ahead and show you what this looks like. So in here, I have the model selected. I can go to DeepSeek. And so what's great is that you can connect to an API provider. And I'm using Fireworks AI. So Fireworks AI here is currently hosting DeepSeek model. And they allow you to use the model just by, you know, using, getting an API key and then putting in the exact model string and so forth here. And so from here. If I go to the open web UI, I'm able to select them and say, okay,

• 9:09Speaker 01

  this is my DeepSeek accounts. I'm going to go ahead and just paste the exact prompt that we had, paste it in with my transcript and everything here, and I should be able to get everything out. So let me just double check here that I got everything. So it's still timing out here. Yeah, server's busy. Try again later. So yeah, that's not fun. So what I'm going to go ahead and do is scroll to the top and hit this little copy button and then go over here. Just make sure I put everything in there. I have the whole transcript. Yeah, so the whole transcript's here. And so what's going to happen here is that when I hit send, it's going to send this off to Fireworks AI. And what's great about this thing that's actually running in this open web UI is that it's using the API and it's not actually sending the data to China.

• 9:53Speaker 01

  So just for, as it's doing its thinking here and showing us what's going on, I'm going to go back to overlay this model of our data in our container and kind of show you what this is looking like in the background. So here in TL Draw, we actually have our Mac and PC. And so I used web UI here and I'm actually using the Fireworks API. So I'm going to the cloud and this cloud is located in North America. So the data basically resides here in North America and it's going to be delivered back to my device. So that's what we're doing. When we're on the DeepSeek website, we were going out to the China region. So that's just a heads up of kind of how that's working in the background. And so next I'll show you the difference of what's going to happen in like the speed difference

• 10:34Speaker 01

  with what Grok hosting provider provides. And then a little bit later, we'll get into a little bit of details for how to get this set up. So as you see, it's kind of outputting this stuff here. Because these models are still so new, these web apps are still adjusting to take a look at the reasoning stuff. And so what I'm going to go ahead and do is hit this little pencil up here to the very top. I'm going to make this a little bit bigger so you can see. And then from here, when you hit this little pencil, it creates a new chat. At the very top, you can select the model dropdown. I'm going to type in DeepSeek. And the one that I have set up from Grok is called the Distilled Lama 70B. And so this model is actually like a smaller distilled version that they're hosting, but it's incredibly fast using the Grok API.

• 11:19Speaker 01

  And so if we hit this here, it seems like nearly instant by the time all the stuff kind of starts finishing. So we'll see this model actually going out. It's doing its thinking and now it's actually providing the response just like that, super fast. So if we take a look, this this has thought for a few seconds and actually shows us the reasoning that was going on. So this is actually going through my transcript, trying to really understand what was going on with my transcript. This was an interview with LDJ, who spoke a lot about DeepSeek and technical details of things that, you know, I really couldn't remember. And so then it basically makes this like, you know, a very simple blog post. So we'll see if my other one that was running the larger models finished. And you can see the difference between the two models.

• 12:03Speaker 01

  The distillation model is just going to give us like a small little blog post versus the the full model that's also running on Fireworks API is actually giving us quite a bit of detail. It's going to take more time, but take a look at what it's doing right now. So it's above here was all this thinking stuff, but this is actually now doing an analysis on my transcript and generating a really nice blog post. And so it's telling us about the calculations that LDJ talked about in the stream, geopolitical implications of what's going on with the new AI arms race, also future predictions. And we talked about these details in the live stream and it literally picked them up and is now creating a graph from this. This is how crazy these models are, if you can really think about it. And here's some key takeaways as well.

• 12:48Speaker 01

  So that's an amazing thing and I'll be able to share these prompts with you and, you know, so that you can actually run these analysis on your guys' own transcripts as well. Yeah. So here's the SEO enhancements and final thoughts. Yeah.

• 13:02Greg Isenberg?

  So when you're in, when you're in business or you're building a startup, Having an unfair advantage is so important, right? Like being super efficient and keeping your costs low, creating your product to be the best possible thing. Now, we're in this new DeepSeek world where the model, I call it a DeepSeek world, but it's really, it's a llama world. It's a DeepSeek world. It's a world where if you figure out the model that works for you and the task that you want to accomplish, you might be able to outcompete whoever you're competing against. Now, I've done a similar prompt on ChatGPT and with some of my YouTube transcripts.

• 13:47Greg Isenberg?

  And it's not unusable, but it's more of a thought starter. It's like, oh, okay, like I can take most of this and I can rejig this and add this and add that and probably get to a blog post that is good. But it does require a lot of human energy to go and do this. When I see what's coming out of this, what's really, really mind boggling is the fact that it almost looks like, just quickly scanning this, this looks pretty, pretty human level incredible. Like a senior writer would do something like this.

• 14:27Speaker 01

  Yeah, or a research engineer that you hire to really thoughtfully take a lot of notes, spend a lot of time analyzing and like put together how you would want to report. And it's even more incredible because these instructions can be configured. So if you want like a graph or you want to type a thing, we can take this prompt and put it into DeepSeek itself to say, can you give me this type of output instead? And it'll do that for us. You know, it's like, how do we improve the prompt? Or what do you want to see from your outputs all the time, from your live streams? And like, I think the thing that I've seen, this is kind of the biggest breakthrough that's happening is that with, I'm seeing this also with O1 Pro, by the way.

• 15:11Speaker 01

  O1 Pro and DeepSeek reasoning models, these reasoning models spend extra time and actually pay attention to your instructions. And so every little detail that they're seeing, they're like, oh yeah, I haven't done that yet. Okay, let me go ahead and make sure I still do that. And that's something that I super deeply appreciate. And for me, it's worth the extra 200 bucks I pay a month to OpenAI. But this is really quickly turning my head and like. Oh my goodness. Did you, could you understand like what just happened here? It's like, I'm a little still taken, I'm still taken away at this output. Like you're saying, it's very detailed. And it's to me, I feel like this is totally a game changer. And I think one thing that people aren't really talking about right now is actually this additional rush

• 15:56Speaker 01

  to understand who can host this. In order to host these huge, like 600 plus billion parameter models, you need all those GPUs, you need services like fireworks. Grok is trying to spin that up, you know, Grok was able to get a distilled model. There's just so much demand, there's going to be even more demand for these chips. And yeah, this is just the beginning and I'm trying to figure out, you know, who, which provider can host this for me reliably so that I can do this for myself, but also share back and put this into apps for other people as well. Because this is going to be, this is great. And I don't want the data to go to China. I just want the data to stay in North America. Or if I get a European container, I can do the European container and meet all their legal requirements that need to happen for that as well. So that's super exciting. Yeah.

• 16:41Greg Isenberg?

  What's the cost, the pricing for fireworks?

• 16:46Speaker 01

  Yeah, the pricing for fireworks, we can look this up real quick. I think it's about $8 per million tokens. Where normally, I think ChatGPT was like 15 input and $60 for output for O1 Pro. I can just double check that real quick. So pricing, let's see.

• 17:09Greg Isenberg?

  Yeah, from what I remember, it's cheap, it's like significantly cheaper than going to O1 Pro.

• 17:19Speaker 01

  Yeah, so O1 Pro or O1 API cost, yeah. And pricing.

• 17:30Greg Isenberg?

  And this is gonna add up, right? Like you might, you might be like, oh yeah, who cares? A million tokens. But once, once you add this to your workflow and you're pumping out content or you're doing research, you know, on an ongoing basis, or you've built a business around how to do this, these tokens will add up.

• 17:52Speaker 01

  That's exactly it. Yeah, they'll add up. Also, at the same time, OpenAI has currently promised that the O3 model will come out and the mini model will come out, which would be on par with this model. So that prices will also probably significantly drop as well, because they just get more efficient with time. And so that'll be really interesting to see. And, you know, I'm rooting for it because for me, as a consumer, I want the power of all the intelligence. And to do these types of things, I think it's going to be pretty important. And I think, to that note, I think it'd be interesting to show how, if anyone hasn't found out about this, it's this thing on OpenAI's thing called the platform dot openai dot com. So you just sign up for accounts or developer account. It's a little playground.

• 18:37Speaker 01

  And what you can do is actually hit this little generate star button. And so we can describe a prompt that we want for any type of model. And what it will do is reconfigure the prompt for the language model so that it can actually be more efficient at doing something. So if there's a task that you do quite a bit, you're like, I just want, you know, please like make, you know, keywords for my Amazon listings and then hit like generate. What it'll do is basically, whoops, sorry. If we hit generate here and hit this at the very top right and hit update, what it will do is actually reconfigure this prompt just from a one line type of thing to include more details.

• 19:19Speaker 01

  So we can take existing prompts and try to improve them through this mechanism. So as you can see, this is like how people get really nice long chains of thought or reasoning or outputs. So one way to think about these things is to first kind of put down like what instructions you want, what type of output do you really expect? Maybe what you don't want as a, as a good starting point. And then that'll help you generate prompts that can be a little bit useful. A lot of the prompts that you're seeing that I spit out are basically things that have come out over time because of, you know, of my use cases. It's like, okay, I want this instead of that. And so as an example,

• 19:57Speaker 01

  One of the things that I was thinking about is like, how do you verify like claims for a specific type of thing? So if you have an article and how do you understand if something is actually true or not? So I have one here that says information verification. And so one of the things that these models that I was currently showing you is that the fireworks and the Grok are specific API endpoints. And right now, there isn't like a specific web search thing that's currently tuned into them. So if you wanted to do web search, you have to go through the deepseek.com route or the app. And keep in mind, you're also sending data into this container there. So like, sometimes you could just use it for public articles for things that you really don't care about.

• 20:41Speaker 01

  So if you're on DeepSeek, let's just see if they actually have stuff available here for us. So you just go to the search thing, turn that on. I paste my prompt in. And then what I'm going to go ahead and do is like maybe grab an article like the Techno Optimist Manifesto from Mark Andreessen. It's a very popular article and sometimes, you know, it's really long to read. There's like a lot of information here. And you're like, oh, my God, there's like so much in there, right? It's like, how can I even get started with this thing? And how do I even verify the claims of this stuff? And I think this is probably sometimes like a good thing to start here. So what I do is I hit shift and enter and I put the like article at the very top. And once I do that, I just go ahead and paste that in there and then just go ahead and hit send.

• 21:26Speaker 01

  So what that's going to do for us is going to use the web search and try to like look through just like how we saw earlier that was happening with the API. Every type of claim that's in there, try to see if they can search the internet for it and try to see if it can do anything. So this is going to try to do its thinking thing. This is obviously very popular and you know, DeepSeek, the website is getting flooded with people because of basically it being free. And so, yeah, just keep in mind, like Greg says, like, yeah, I would not be putting taxes in there. I probably wouldn't be putting medical records in there. Things that you know, you don't want to see generated if somebody asks a question that's related to you. Because that can be a little crazy.

• 22:09Speaker 01

  You'll be like, wow, all of a sudden my data is showing up somewhere that I was not expecting. So. Yeah, this is this is currently airing out. It's no surprise, and this is kind of why we're kind of thinking about doing some other alternatives here. So yeah, I think that's that was set there. So I think another thing maybe that can be useful here is probably getting this stuff set up. So if you wanted to run this locally, maybe we can kind of go over that workflow. What do you think?

• 22:38Greg Isenberg?

  I would love, yeah, I mean selfishly, I would love to know that.

• 22:43Speaker 01

  Okay, okay, cool. Awesome, let's do that. Yeah, I think that that'll be great. So in order like for this section, in order for us to run this model locally, the best interface that I've found bar none is something called Open Web UI. So Open Web UI, it's really quick to get started. All you need is to download Docker. So just go to Docker.com and you can just download the desktop app. So download the one that you need for your machine. If you're running an Apple M machine, which I am doing, I just download the Apple Silicon version and accordingly. So once you get that installed, it's going to present to you a user interface like a dashboard here. And so that's kind of something that you'll have, you know, kind of going.

• 23:25Speaker 01

  Mine's is already showing the app is running here, but that's how you already know it's installed. It will require the terminal, but it really won't hurt you too bad if you have it running. So the first command that you will basically run is this one that's listed on their quick start. So the quick start will be listed here and we'll have this available as a guide for those to download. And all you have to do is follow these two steps really. So the first step is pull the container. So this is you just copy this and then you put it into the terminal. And then it's going to do this little pulling thing and it'll probably download like, you know, several gigabytes of files onto your machine. And then the next step is literally what they call running the container. So with Docker, the whole app and everything is all contained in one. So that way you don't have to spend a bunch of time doing extra terminal things.

• 24:09Speaker 01

  This is probably the only two terminal things that you will run. If you're running a PC, you probably want to run, especially for Nvidia, you'll want to run this command, GPUs all. So all you have to do is just copy this one if you're running Nvidia, and that'll take advantage of your GPU. And it'll run more efficiently when you're running it locally. So the one I like to do is just the single user mode, which doesn't require sign in. That way, if you're the only one that's using it at your house or on your network, that's probably the best way to do it. So you just copy this command here, and then you put it into the terminal, and it'll say, hey, great, it's up and running. And then all you have to do now is just go to the website, localhost 3000. And then, once you're running on localhost 3000,

• 24:53Speaker 01

  you're going to be presented with some user interface thing like this. And I have a model that's currently loaded here. That's kind of why it's showing us here. But you're not going to have any models loaded. So in order for you, the next step here that we have to do is actually have a couple of options. One of the things that I do is like you can just download a model locally. And I use a thing called olama. And so olama.com is something you'll want to download there. And so that way you can run any local model. And it's simple as just finding the model and so forth. So once you hit download, it's going to download it for your machine and you install it. You'll see like the way that it's downloaded, you actually see this little llama guy that's at the very top and a little figure there. And that's how you know it's currently running.

• 25:39Speaker 01

  And so, once that's currently running there, you'll see the models that are currently listed here in the model section at the very top. And so DeepSeek R1 is going to be the one that we want to use here. So we'll go back to our web UI instance. And then we're going to go ahead and hit where it says user at the bottom. From there, we're going to go to the admin panel. And from the admin panel, there's like a section, a settings area. And so this settings area has an area of our connections with a little cloud icon. And this is kind of where we're going to connect our other providers here. So let me make this a little bit bigger so that everyone can see. So as you can see, the Ollama API is already configured for us, which is nice.

• 26:20Speaker 01

  And this is already going to have the Docker container there, which is great. And so when you hit the little pencil here and you hit plus, change the model, you can type in the model like DeepSeek. And if you don't see it available, which it may not be there, you'll see this option at the bottom that says pull DeepSeek from Ollama.com. And so that'll actually search Ollama here to get it for you. So like, for example, if we wanted to download the fee for model, I'll just type that one in, just as an example, so you can see. P4. So I have no, I don't have that model currently downloaded. I just can hit here and it's just going to go ahead and find it and then it downloads it.

• 27:01Speaker 01

  So in no time, basically this model will just be downloaded on my machine. And then I could just type in fee for, and I'll be able to use that in the future going forward. So what we can do is go ahead and type in, you know, we're starting a new chat and we're going to basically select the model DeepSeek-R1. And you'll see it'll be colon latest is kind of what it's listed at. And that's actually how you know that's the one that's currently running locally. And so once you select that one, you can just say something like explain options trading. And then go ahead and hit enter. And so what this does is, you know, it's basically, you know, thinking, and you can see the thinking tokens of what's going on when it's thinking.

• 27:46Speaker 01

  And so all of this is actually running on my computer, which is amazing. One of the ways that I can tell is there's this command line called ASI top. And it actually shows us all of the resources that it's eating up. Thankfully, I have 128 gigabytes on my machine because I do live streams. I do this, all this stuff at the same time. And you can kind of see how much RAM it takes up right now with me hosting the stream plus, you know, running this model locally. So yeah, this is the, actually what it does here. One of the things that we could even do is try to test that prompts that we were using earlier so that we can run this command locally. So earlier what we did was we were running like a whole analysis on something and it would just fail out.

• 28:30Speaker 01

  So this thoughtful analysis that I was showing you. We can try to see if we can run this on a local model and just see the difference as well. So this is basically the transcript that I had earlier, plus the analysis stuff. And if I go to open web UI and then just go ahead and go back here and create a new chat. And hit paste. And then run. So this is going to see it's thinking here and it's using up all the resources on my local machine to run this model. And it's quite a lot of tokens. And it's, it's still fairly impressive what a smaller model can do that's running on my machine. And you'll have different versions that you can use, I think. And so this one's using the seven billion parameter model.

• 29:15Speaker 01

  If you get something that's a little bit higher, it's probably going to get you a little bit more detailed response. And I would definitely play around with these things. Another important setting, I think that you can tweak, and we can probably run this as a next chat. Is while this is going here, there's control section. So this control section at the very top will show us, let's see what dismiss this. So the controls, one of the controls that you'll probably want to change around to get different results is the temperature. So it's setting the temperature from like, you know, point eight, the default to like a lower temperature will actually make it like hallucinate less is kind of what people say. And so it'll tend to follow instructions better and then not kind of veer off into different tangents.

• 30:00Speaker 01

  And then another one, if you go all the way to one, it'll just be extremely creative. So you can think about those as far as maybe if you're doing some creative writing, some non logical reasoning. That can be really helpful if you want to kind of think out of the box and have it kind of go into La La Land. It's really up to you and your content, but I would definitely do two different responses with different temperatures. And test those things and see if you see any difference in your output. For me, sometimes I find the temperature zero to be very helpful for very logical reasoning outcomes, especially around code. But it's, it's really up to, it kind of varies and I just kind of want to give you a heads up on that.

• 30:35Greg Isenberg?

  Yeah, I appreciate that. To me, I would rename that temperature as.

• 30:43Greg Isenberg?

  You know, wine versus coffee mode.

• 30:46Speaker 01

  Love it.

• 30:47Greg Isenberg?

  Wine, wine might get you a little more creative.

• 30:51Speaker 01

  Right.

• 30:51Greg Isenberg?

  You know, if you're, if you're one more rational execution style, maybe you want coffee mode. I think that is something that I, you know, we have a LCA. It's our design firm for the AI age. We have a, I feel like that's what's missing from a lot of these AI products is just like a little humanity and, and, and lightness.

• 31:14Speaker 01

  Right.

• 31:15Greg Isenberg?

  So I expect over the next couple of years, we'll, we'll start seeing, uh, you know.

• 31:20Speaker 01

  You know, what'd be funny to basically have like a spinner where you, you can actually flick it yourself and you kind of see it land on something and then just like, hey, go.

• 31:28Greg Isenberg?

  Totally. That would be cool.

• 31:29Speaker 01

  Cause sometimes you really don't care, right? You just like, I just want to spin the bottle and see what happens. Like.

• 31:35Greg Isenberg?

  Totally.

• 31:36Speaker 01

  It's this kind of YOLO mode, kind of, yeah, yeah.

• 31:38Greg Isenberg?

  Yeah.

• 31:38Speaker 01

  Yeah. Cause I think the like, like you say, there's huge opportunities in the AI space to be playful. And, and I think that's what's interesting is you have these intelligence of the, the models. Uh, and then now you have to have people who will build interfaces to interface with them. And there are a lot of companies who are trying to do that. And, um, you know, you can get very far with just some prompting as we're seeing here. Uh, and then we're trying this exercise here is to try different models. So if you think about it, Ollama is sort of the gateway, um, you know, to all these different types of models that you can try out and see if it even works for your use case. Um, and, and this web UI is actually really nice user interface to keep track of that. Uh, it's saved locally on your machine. You can go back to them at any time.

• 32:23Speaker 01

  Um, you can, there's additional options at the bottom here, which is really nice. So, um, you can actually have this read aloud to you. So, uh, if you're a person suffering maybe from dyslexia or you actually prefer audio, you can have that for you. Uh, this will give you some information there. You can continue the response. Sometimes if you have too much information, it still needs to continue going. So you hit the continue and it'll just continue on, uh, or regenerate the responses. Um, so that's, that's kind of some of the, the basics there. So, yeah. So this is the output of this model and. I'm fairly impressed for being a 7 billion parameter model and running locally on my machine, that it took that entire transcript and did this analysis type of thing. That's,

• 33:07Speaker 01

  I'd say is pretty close to the bigger model. And in terms of details, it's not as detailed as the other one. If we kind of take a look. So like the previous, this is one that it came out with before, you know, with this nice big blog post type of thing. So it's, it's pretty good. And it's, it's running, you know, locally. I can run this on the plane. As far as that. So, yeah. So to get, get started, basically, again, just open web UI. There's a getting started. It's literally a couple of steps to run. Make sure you have Docker installed there. And then Ollama is going to show you all the different models. So if you go to the models, you'll see kind of stuff that's popular and trending right now. And that'll kind of get you some of that as

• 33:48Speaker 01

  well, as far as getting started. There is a, you know, we're also talking about fireworks AI. So that's fireworks. It's a good resource for you to, you know, go take a look and put that model in. So like, if you want to put that model into your Ollama, you would kind of do the same thing here. So go to user, and then you go to admin panel. And then you would go to settings up here. And then from the settings, you're going to go ahead and hit connections. And so what you'll do is go ahead and hit the little plus connection. And so you have to put in the base URL, and you'll also have to put in the API key. So the base URL here for fireworks

• 34:26Speaker 01

  is this specifically here. It says api.fireworks.ai/inference/v1. In the example documents, you'll see slash chat slash completions and things. You don't need those because that's part of the OpenAI framework is that you just put everything up to V1. And then you'll generate an API key from that model over in fireworks. So AI. So if you go to the model here in fireworks, and you go to the, your name, and then if you go to API keys, once you go to API keys here, you just hit create API key, and that'll pop up. And that's the key that you want to put in there. Similar to Grok Cloud, you just go ahead and hit create API key.

• 35:11Speaker 01

  So once you go to console dot grok dot com, there's an API key section here. And then you'll want to hit create API key. And that'll pop up the dialogue with those API keys. And so that endpoint will look something like this over here. So that'll be if we hit configure. API dot grok dot com slash openai slash view one. And then you put your key in there and you don't have to do anything with these IDs. These will be pulled directly from that endpoint. So whatever models you have available will be there. And so now, when you hit the plus sign, you'll see like this nice list of models from fireworks. So there'll be the fireworks ones. So account slash fireworks. You can play with any one of those.

• 35:52Speaker 01

  And then the other ones that are just with the normal name are from Grok. So they have those as available for there. So you can, you can play with a lot of these models, which is nice and compare them. And then the ones at the bottom are the ones from Ollama. And so they'll show like, you know, the colon latest is kind of how you can tell. And if you hover over them, you'll see like some additional information over the parameter count, what quantization level it is. So Q4 means it's quantized to four bits. And that also has a play in its intelligence. Obviously, the higher level of quantization, you know, means more memory. So it's like 32 bit, 16 all the way down. So like the, the lower the number, the like, not less intelligence, but you may not get the output that you want is expected.

• 36:37Speaker 01

  So that's kind of part of that process. It's a lot of different things here, but I think the most important thing is just, yeah, how do you host this locally? How to start playing around with it? And that's kind of like a really good primer to get started for doing these models and stuff. Yeah.

• 36:54Greg Isenberg?

  I love it. I don't know if you've played around with it, but is there any way to do this on mobile? Like, could you play with local models on the mobile device?

• 37:05Speaker 01

  Yeah, there is an app called Apollo. Have you heard of that? Apollo.

• 37:10Greg Isenberg?

  Yeah, I just haven't used it.

• 37:13Speaker 01

  See, I don't think App Store, App Store. I'm going to see if I can go here. Apollo.

• 37:19Speaker 01

  Let's see. Okay, private local AI. Yes. So I have this app on my phone and they allow you to download the models directly, just like you would with Olama as well, but it's, it has its own interface, which is really nice. And so I wonder if I could share my screen. I think I can. So

• 37:41Greg Isenberg?

  On your phone?

• 37:42Speaker 01

  Yeah.

• 37:43Greg Isenberg?

  Yeah.

• 37:44Speaker 01

  They have a phone mirroring.

• 37:46Greg Isenberg?

  Exactly.

• 37:47Speaker 01

  Apollo.

• 37:50Speaker 01

  Okay. Oh, I have to lock my phone. Okay, cool. So I lock it and then it should be able to connect. Okay, cool. Nice. Awesome. So yeah, let me kind of minimize this here. And

• 38:07Speaker 01

  yeah, okay. Let me just go to a different screen here. Probably one that's less cluttered. And do phone? Whoops. Put that over here. Cool. Maybe this will work. I think. Yeah.

• 38:26Greg Isenberg?

  Yeah.

• 38:27Speaker 01

  Sweet. Yeah, I actually have the, yes, another place to get your models apparently is also through OpenRouter. And so, yeah, so this is kind of the Apollo app. You're like, okay, cool. I can, can I start chatting with this? You know, as soon as I play with the thing, a couple configurations you have to do is you hit this little hamburger menu at the very top left corner, and then you hit settings. So on the, on the phone app, you hit settings and it's going to say AI providers. And when you click there, you have three different options. OpenRouter, which is another API provider, and you can also get access to pretty much every model there, which is also very handy. I think they give you some free credits, but then you would put, you know, your credits there.

• 39:09Speaker 01

  And then you have the local model and then you have custom backends. So with the local model, they actually can tell how much memory you have on your device and they'll actually have a little download button for those models. The ones that are not available with a download button basically means you can't run that on your device because you don't have enough memory to run them. So these downloads Are pretty big, like four gigabytes and some of them, you know, are several gigabytes. So just depending on the space on your phone. So you can actually run the distilled llama eight B eight, uh, eight bit MLX version. And I have the distilled Quinn version at seven B. So it just depends on your, oh, that one's actually not compatible. Uh, which one do I have downloaded? So I think on mine, let's see.

• 39:54Speaker 01

  The one I have available is the deep seek R1 from Apollo. I think I have it from open router. It's running. So let's take a look here. AI providers, open router. Yeah. So the, the one that I have set up right now is from open router. So open router will show you all the models. I have, you can select deep seek R1 from there, which is awesome. So you can have a conversation. So this just requires me, you know, being connected to the internet. We start a new chat. You're like, uh, tell me more about options trading. And so here you're still talking to the model, uh, but you're actually just using open router. And so that's a little bit different than, you know, sending your stuff directly to deep seek.

• 40:38Speaker 01

  Uh, and they should be able to do that. It's possible that this model is busy or it's currently down. That can happen. So, yeah, that happens. Yeah, while that's going, I think we could even start another new chat. Let's see, it's this model. You can select a different model. So. Let's see.

• 40:58Greg Isenberg?

  It's crazy how many models there are now.

• 41:01Speaker 01

  There's so many. Yeah. It's like, how do you know which one does it? I feel like you just go off vibes. Like what's, what's my friend telling you? Yeah. Like, what's the real vibes right now? So the vibes right now, obviously, R1 is like the real hotness. People are like totally into that right now. And it makes sense because, you know, reasoning at a much lower costs. So, um, let's see.

• 41:26Speaker 01

  There's probably something go wrong with my API key or something. So AI providers, I can select the local model to run. You know, I want to see if there's something small here that we can download. So we could do, yeah, this Distill Quinn, just for, just for speed purposes, we'll just download the gigabyte one. So this is going to download. Wow, that's really fast.

• 41:47Speaker 01

  The Quinn model 1.5B. And so that'll run DeepSeek locally. And so basically, it's just downloading it directly from, I think, Hugging Face. And then the model's being loaded on my phone. And this, this is actually optimized to run on Apple hardware or Apple silicon. So that's, you know, one way that you can kind of take a look at it to run this thing. And so what's nice, yeah, if this phone runs out of internet, or I need to ask some questions or do some stuff, I now have, I will have this R1 reasoning model that's a much smaller version to run on device. And I think that's a, another good point about AI that's running. And like, you don't always need the most powerful thing running for every single type of thing. I think it's really important to

• 42:30Speaker 01

  understand different use cases, you know, because maybe you don't need that depth of reasoning, you just need something that's really quick. Or you just need something that's really good at like gathering lots of information and just telling you some topics or something like that. And that could just be done really quickly. So it's kind of like picking the right tool for the job and experimenting. So the, we're at a good age today where you can actually get these models and experiment with them. So now I should be able to select this guy and run it. So let's go ahead and hit done and start a new chat. And then over here, we're going to go ahead and select the model. So here, we're going to type in, oh yeah, it's already has it at the top. So you see this little icon that signifies that it's running locally. And then we're going to hit cancel.

• 43:13Speaker 01

  Hit done. Okay, great. So now we're running with that local model. And I think we're just using a default system prompt about it being Apollo. And you're like, yo, tell me about options trading. And it should basically start to cook. So it's using my phone's power there. And it's now thinking. And so if we click this little dropdown, we'll actually see the reasoning tokens. And so that's, yeah, I have reasoning on my phone, no internet, completely running locally.

• 43:46Greg Isenberg?

  2025 is insane.

• 43:50Speaker 01

  Yeah. And imagine being able to run this on your watch. Like, that'll just be

• 43:57Speaker 01

  because this is already showing its capability. Like, we're doing this input. If you're, if you can make an app that can run on a watch all locally, you know, just think about like the transcription stuff, right? You have a very, very lightweight model. You send the audio, you know, from the watch, you know, especially if a loved one, maybe they've fallen or something. It can just turn on the speaker and try to understand the situation. And then if it listens to paramedics or something about asking questions and they don't really know, maybe the watch can show, hey, there's this app here. I'm going to show you the emergency card this person has for their medications. Or this is something that's happened in the last, you know, five minutes before this event or something. This is kind of the way that people think about designing apps with these models

• 44:41Speaker 01

  is trying to think about these use cases. Because now you have really powerful devices, just all like on the size of your wrist, that can run these models and the power of Apple's MLX infrastructure and also their AI technology is the fact that, you know, these, these are really optimized to run these models, you know, very small, as we're seeing right now. So that thought for 49 seconds and it gets us this output. Yeah, please excuse my small screen, but we'll probably have a zoom in on this.

• 45:08Greg Isenberg?

  Yeah.

• 45:09Speaker 01

  For the edit. So yeah, yeah, yeah. That's, that's pretty sweet.

• 45:15Greg Isenberg?

  So many startup ideas, by the way, like from, from that alone. You know, and so many, you know, I love that. I love that you shared that example of, you know, someone maybe falling and hurting themselves. I think even coordinating with your, your AirPods.

• 45:33Speaker 01

  Yes.

• 45:34Greg Isenberg?

  There's just a ton of opportunity there as well. Translation, you know, not just pure translation, but it's like someone is saying, saying XYZ. But what are they really saying? You know what I mean?

• 45:48Speaker 01

  Oh, yes.

• 45:49Greg Isenberg?

  Maybe like, imagine like negotiating in the future, except you have like, you know, the, you know, pretend you're a lead negotiator as like almost, you know. A local AI LM that's helping you figure this out. So there's just, you know, we can have a million ideas, but it's just really exciting to see what that, you know, where this could go.

• 46:10Speaker 01

  Yeah, I think that also goes to the point a little bit about what some of these models do. So one thing I just learned very recently about GPT-4 and ChatGPT's Omni models is the fact that this model's breakthrough, a little bit different than R1, is the fact that it can actually understand audio and tone and all these extra implications that we don't know about, especially for negotiation. Like, imagine if you can understand someone's breathing rate just from listening to the audio. That's the capability of something like, you know, the 4O models with audio. You just give it the audio and it's going to know tone. It's going to know cadence. It's going to pick up things that we just normally don't think about. But people who are maybe skilled negotiators can understand what those implications mean and then can say, hey, give me some outlier things.

• 46:55Speaker 01

  Every time I give this person a response, you know, it can answer in like milliseconds what the differences are. And I've heard these terms of like micro expressions. And if you have an Omni model, you can actually mimic these, you know, micro expressions and say, okay, this person is off when we ask them these types of things or changes their position. And those are the things that you really can't get today with some of the, you know, of current reasoning models, but except for like the Omni models, which is the, you know, 4O models. And so it's going to be really exciting when they actually dropped O3. I think a lot of people are going to be taken by storm of like, what's actually really going to come out from them. It's going to be a really, really big leap.

• 47:36Greg Isenberg?

  Anything else you want to cover today?

• 47:38Speaker 01

  I think this is a really good primer for folks to get started on the power of prompting, and especially with these reasoning models, just to get started. So we covered being able to get started with prompting, understanding where your data is going. You know, if you're using deepseek.com or using the apps, that will go directly to China and their restrictions and things that they have for your data privacy. So just beware. I wouldn't personally be putting any personal information in there that you don't want exposed. And then there are other providers that you can use right now and other people are That are still spinning up at this very moment. So for now, Fireworks, Open Router, Grok as far as inference. And then we also covered here, running the models locally so that you can actually, you know, run them on your phone.

• 48:22Speaker 01

  Using the Apollo app I was using, it's a paid app, but I find a lot of value from it and I'm not sponsored or anything like that. I just love the work that these people are doing and it's really good stuff. You can connect to these endpoints with them. And then the other part is running this through Ollama locally on your Mac and using web UI with Docker and stuff. So I think there's a lot to be taken care of here as far as trying to use these models and come up with different app ideas. And if you have an idea, just start to use the playground to try to generate some prompts for it and see if you can get the output that you want. And that could be the beginnings of your next multimillion dollar idea that you don't even know is there, right? So it's, it could be hidden in plain sight and I think that's the power.

• 49:05Speaker 01

  If you want to reach out to me, you can just go to rayfernando.ai and book some time. We can have a conversation, we can get you set up because some of this stuff is very cumbersome. It's just easier for me to walk you through this and so I'm available there as well. You can find my YouTube channel, Ray Fernando 1337 on YouTube. Feel free to check that out. I do a lot of live streams where I check out new technology and try to play around with these AI models and try to discover what's going on. And also try to bring on experts to explain things a little bit more for us. So Greg, such a pleasure to have me on the show. This is really amazing.

• 49:39Greg Isenberg?

  You're a legend, man. Thank you for coming on, sharing, sharing your insights here. This has been super helpful. I thought it was helpful. So if people agree, go, go comment on YouTube. I read and respond to almost every comment. Like and subscribe for more of this in your feed and let us know if we should bring, and when I say we, it's me, you know? Let me know if I, you know, should invite Ray back on again to show us more stuff. I would certainly love to do that more in 2025. Crazy times, Ray. This whole deep seek, you know?

• 50:19Greg Isenberg?

  Title wave is just

• 50:20Speaker 01

  It's Yeah, I'm glad that it's something is here to like make more people aware that there's a lot of intelligence and how fast it's moving. And I want to also add that like, please don't be fearful or don't feel like you're left behind. If you're just finding out about this, you're not that far behind. We're all actually still trying to understand what this intelligence can give us. And so the prompts and the things that you develop is a good place to start. And you know, it doesn't have to feel complicated. And you know, whatever you can get your hands on, make sure you do that. And you know, be aware of where your data is going. But at the same time, play, discover, share, share back with the community, and definitely share any cool stuff that you've done in the comments, for sure.

• 51:05Greg Isenberg?

  All right, my man. I'll see you later. Thank you so much.

• 51:09Speaker 01

  Take it easy. Thanks.