Local AI Clearly Explained

I run this episode solo. I explain local AI in plain terms: the model runs on hardware I control, and a cloud model runs somewhere else. I map the four pieces of the local AI landscape — the model, the warehouse, the software, and the workflow — and I define the words that beginners meet first: parameters, tokens, context window, quantization, and GGUF. I walk through the Google open model stack (Gemma 4, Google AI Edge, LiteRT-LM, AI Edge Gallery), compare the other open model families, and show three ways to run a model today. I close with a first workflow you can copy and three startup ideas that use local AI as the wedge. And a special thank you to Google for supporting the podcast. Timestamps 00:00 – Intro 01:35 – The Open Model the Landscape 03:09 – Vocab Decoder 06:48 – Google Gemma Clearly Explained 10:29 – Other Open Model Families 14:20 – Path 1: Run Gemma in LM Studio 18:17 – Path 2: Ollama 20:15 – Path 3: Google AI Edge 21:07 – Hardware Cheat Sheet 21:52 – First Workflow to Build 22:47 – Workflows Before Fine-Tuning 25:06 – Local vs Cloud vs Hybrid Eval 26:33 – Framework for Local AI Startup Ideas 27:22 – Startup Idea 1: Home Health QA Reviewer 29:24 – Startup Idea 2: Offline Field Report Copilot 32:10 – Startup Idea 3: Pre-Send Reviewer for Professional Services 34:47 – Build Your Local AI Lab 37:55 – Closing Thoughts Key Points Ask whether the model is good enough for the job, and the business opportunities become clear. Local AI has four pieces: the model, the warehouse (Hugging Face), the software (LM Studio or Ollama), and the workflow you build around them. Gemma 4 E4B is my practical starting point; E2B fits phones and older machines. Hybrid architecture wins: local does the private first pass, cloud does the heavy reasoning, and a human approves anything important. Start with one repeated workflow — one folder, one model, one output — and run it 10 times. I see a 24-month window to build local-AI-native software for verticals that still run early-2000s tools. The #1 tool to find startup ideas/trends - https://www.ideabrowser.com LCA helps Fortune 500s and fast-growing startups build their future - from Warner Music to Fortnite to Dropbox. We turn 'what if' into reality with AI, apps, and next-gen products https://latecheckout.agency/ The Vibe Marketer - Resources for people into vibe marketing/marketing with AI: https://www.thevibemarketer.com/ FIND ME ON SOCIAL X/Twitter: https://twitter.com/gregisenberg Instagram: https://instagram.com/gregisenberg/ LinkedIn: https://www.linkedin.com/in/gisenberg/

• 0:00Speaker 00

  I think local AI and open models are going to create a ridiculous number of business opportunities over the next 24 months, and I don't think most people actually have the map yet. They've used ChatGPT, they've used Claude, but when they hear local AI, Hugging Face, Oh Llama, LM Studio, AI Edge, it sounds like it's for this developer world and that normal founders are just not supposed to touch it. And I think that's a mistake because the opportunity here is actually pretty endless. By the end of today's episode, you're going to understand what local AI is, when it matters, how to run open models at work, where Hugging Face fits in here, which GEMMA model I'd start with, how I'd run a model locally with

• 0:41Speaker 00

  LM Studio or Oh Llama, and how this turns into real business ideas. And I'll give you three startup ideas I'd actually consider building using local AI, including who the customer is, what the first version does, why local matters, and how I'd sell it. Basically, this is going to be a masterclass around local AI, how to run models, how to build apps, how to make money from it, and I'm going to explain it for the average person who isn't technical. Quick shout out to Google for sponsoring today's episode and for caring about local AI and open models for entrepreneurs. Today's episode, I'm going to use GEMMA and Google AI Edge as the main examples, but the goal is to give you a full map so you can actually understand the space and build with it and use whatever model suits you.

• 1:25Speaker 00

  Okay, let's dive in.

• 1:35Speaker 00

  So put simply, local AI means the model runs on hardware you control. The hardware could be your MacBook, it could be your Windows laptop, an Android phone, an iPhone, it could be a browser, Raspberry Pi, it could be in a workstation in your office. I just got a DGX Spark, which is like a high-end one, but the important part to note is, a phone now could run local AI. Cloud AI means the model runs somewhere else and you access it through a website or an API, and that's the basic difference. The business question is, where should the intelligence live? If I'm doing deep research and strategy and hard reasoning, or something where I want the strongest possible model, I'm probably

• 2:20Speaker 00

  going to be using a frontier cloud model. If the work involves private files, like sensitive customer data, offline usage, fieldwork, low latency, audio input, or an internal workflow that runs again and again and again, local AI starts to make a lot of sense. A smaller model in the right place can actually be very valuable. That is the idea I want you to keep in your head. The first question most people ask is, is this model smarter than the biggest model in the cloud? The actual more useful question to ask actually is, is this model good enough for the job? And does running it locally

• 3:03Speaker 00

  make the product better? Once you ask it that way, you start seeing these business opportunities, which we'll go into. So there's four pieces to the local AI landscape. The model, which is the brain file, that could be something like GEMMA, Llama, or Mistral. The warehouse, which is where you find the model, you might have heard of Hugging Face, I think they're trying to get acquired right now at $13 billion. That's what they do. The software, which is what runs the model, that's something like LM Studio or Oh Llama. And then the workflow, which is the product you're building around all of it. And those are the real four pieces. The model is the brain file, GEMMA is a model family, Llama

• 3:47Speaker 00

  is a model family, Quen or Mistral, you might have heard of Phi, these are model families too. Some of these are actually better at reasoning and some of them are better at coding, some of them are smaller, some of them are faster, some of them are better for images, some are easier to run on your own machine. And then you need somewhere to find these models. That's what Hugging Face is. That's the first place I would go. They're the biggest at it. The easiest way to explain Hugging Face is that it's a model warehouse. You go there and you can find model cards, licenses, file formats, examples, benchmarks, community versions, and sometimes versions that have already been compressed, so they're

• 4:30Speaker 00

  way easier to run locally. If you're new to local AI, one of the best exercises is actually just to open Hugging Face and read a model card really slowly. You're going to learn a lot. At first, though, you can ignore half the scary-looking details and just look for a few basic things, in my opinion. What is the model for? How big is it? What license does it use? What hardware are people running it on? Does it support text, images, audio, tool use, or embeddings? Are there quantized files available? Once you can answer those questions, the space gets a lot more intimidating, because I know when I first

• 5:11Speaker 00

  looked at these cards, initially I was like overwhelmed. So just those are the key questions to ask. Then you need software that runs the model. For most people, I would just start with LM Studio or Oh Llama. LM Studio feels like a normal desktop app. You download it, you search for the model, you click download, and then you can just chat with it. My opinion is it's probably one of the most friendly first-time user experiences if you're non-technical. Oh Llama is a little more builder-oriented or developer-oriented. You install it, you run a command like ohllama run GEMMA for colon E4B, and now you have a model running

• 5:53Speaker 00

  locally with an API your apps could talk to. Then underneath those tools, you're going to start hearing about things like llama.cpp and MLX, and I'll explain what those two things are. Llama.cpp powers a lot of the local model inference. MLX matters if you're on Apple Silicon, and if you're thinking about shipping real on-device apps in the Google ecosystem, that's where Google AI Edge and Light RTLM come in. Basically, Google AI Edge is the broader on-device AI development world, and Light RTLM is the runtime layer

• 6:34Speaker 00

  for the language models. This is what you study when you want to move from, I ran a model on my laptop, to I want this model inside an iOS app, or an Android app, or a web app, desktop app, whatever it is. We got to talk about some key vocabulary, just about the most important things you need to know about these words that come up time and time again in local AI. I'm just going to give you simple, clear definitions of what they are. By the end of this part, you'll know, you know, just the core basics of local AI vocab. So I'm sure you've heard this one before, parameters, like 2 billion, 4 billion. These are what's called the internal weights of the model, and more parameters

• 7:18Speaker 00

  just usually means more capacity for harder tasks, but it does require more memory. So parameters are the internal weights of the model. The beginner shortcut is that more parameters usually means more capacity, and more capacity can help with the harder tasks. So the trade-off is usually memory, speed, and hardware. So a 2 billion or 4 billion model is the kind of thing you might use for edge devices, phones, fast workflows, and smaller tasks. A 12 billion parameter model is more of a middle ground, and a 26 or 31 billion model is getting into the stronger workstation territory, depending on your

• 8:01Speaker 00

  hardware and how the model is built. I recommend like not going out there and spending $5,000, $10,000, $20,000 on a workstation just yet. By the end of this episode, you're going to understand how to just, you know, set up some of these things on your phone or on a laptop, a spare laptop that you have from 2021. Then there are tokens. So tokens are the chunks of text that the model reads and writes. Locally, you care about speed and memory rather than the per-token bill. Then there is the context window. The context window is basically how much information the model can work with at once. Then there is quantization. The word sounds more technical than it needs to. Honestly, I can barely

• 8:44Speaker 00

  pronounce it. Quantization is the compression for models. It allows giant models to fit on normal laptops. For example, you might have heard of Q4, Q8 formats, that's quantization.

• 8:58Speaker 00

  If the full model is the giant version, the quantized model is the version that can actually fit on a normal laptop. So you might lose a little quality, but suddenly this thing magically runs. You will see things like Q4 or Q8. And as a beginner rule, Q4 is just usually easier to run. And Q8 keeps more quality, but it needs more memory. If you're just getting started, Q4 is just a reasonable place to begin. So I would start there. Then there is GGUF. It's a common file format for local models that make inference easier on normal machines like you and I have. And in the Google AI Edge world, you'll see something called the Light RTLM.

• 9:42Speaker 00

  This is the model format and runtime path you care about when building on-device apps with Light RTM. So the simple map is this. Hugging Face helps you find and understand models. Gemma is Google's open model family and Google's a trusted brand. I run my business on top of Google, so it just makes sense. LM Studio helps you try models locally without much friction. Ollama helps you run models locally in a way that a bit more technical people can plug into apps. GGUF is a common local model format.

• 10:21Speaker 00

  And Google AI Edge and Light RTLM are the path towards shipping on-device AI products. That's what you need to know. So let's talk about Google's open model family because I feel like there's a lot here. It's a bit overwhelming and I'm just going to break it down so you understand what you need to know about the whole Google AI open model family. So Gemma is Google's family of open models. And Gemma 4 is built for the efficient, local and on-device use. So they have Google 4 E2B, which is the smaller edge model for phone workflows. You have a bigger E4B, Gemma 4 E4B, which is the, is pretty much the most practical

• 11:06Speaker 00

  starting point for most local tests. Then you have Gemma 4 12B, which is a middle ground with more capability for laptops. And then you have Gemma 4 26B/31B, which is, you know, the stronger local workstation territory. That is the main model picker. Then, and a lot of people don't know this, there's specialized Gemma models that are just really useful to know. So you have things like Embedding Gemma, which is just for search. So specifically, it helps you turn text into embeddings, which lets you search by meaning. If you want to search your own docs or customer notes or support tickets, sales calls or knowledge-based localies, embeddings matter a lot.

• 11:51Speaker 00

  Then they have something called Function Gemma, and that's a tool use and structured function calling. That means the model can help software take actions in a way more structured way. It is part of the path from the model gave me an answer to the model helped the product do the next step. Then you have a few more like Pali Gemma, which is more vision focused. You have Shield Gemma, which is more safety focused. Then you have Gemma Scope, which is more understanding how models work under the hood. You can leave most of the family alone on day one. The practical path, like on day one, if you're a beginner, start with Gemma 4 E4B,

• 12:31Speaker 00

  understand the workflow, then you can move up or down or sideways, actually, depending on what you are building. So the way I understand the whole Google AI ecosystem is you have Gemma as the open model family. You have Google AI Edge, which is the on-device AI development ecosystem. You have Light RTLM, which is the runtime for running languages, models across all the devices. You have AI Edge Gallery, which lets you try on-device models and see the experience just more directly. And if you need huge scale, you know, things like strong managed infrastructure or frontier level cloud reasoning, you still have Gemini and Google Cloud that you can use

• 13:15Speaker 00

  or another frontier LLM that you can use. The, the reality is a lot of big products and serious products are going to use a hybrid setup. They're going to use cloud for certain things and you're going to use local for other things. As an example, imagine a local AI tool for a professional service firm. So the local model is going to read the sensitive drafts, you know, checking for the issues. It's going to strip or summarize all the private details and prepare a clean version of the problem. Then, when the customer wants deeper reasoning, a cloud model can help with the sanitized version. That to me feels like a more natural architecture than just putting everything into the cloud,

• 14:00Speaker 00

  which a lot of people don't want. You basically have local handling the private files as a first pass. And then cloud handles the heavy thinking when you need it. A human can approve the work before anything important goes out. This is how I'm starting to think about building a lot of these products. Beyond Google Gemma, I'll give you a quick primer on the other families or other open model families you'll hear about and some of the pros and cons. Llama is a meta's model family and it's probably the default, and it's probably the default open model reference point for a lot of developers because it's a pretty big ecosystem.

• 14:42Speaker 00

  The upside is the community, the tooling, the examples, support. The downside is you still need to read the license and the model card, especially if you're building a serious commercial product. Quen is Alibaba's model family and has become very strong, especially around coding, multilingual work, long context and agentic tasks. The China thing is real. A lot of people use Quen because it performs really well. But if you're in the enterprise, a government, healthcare, finance or sensitive data environment, you need to separate running open weights locally from sending data to a hosted service. And you need to check what your company is comfortable with or what you're comfortable with.

• 15:26Speaker 00

  DeepSeek is similar in the sense that it's was made, it's made a lot of people realize how strong Chinese-based open models really could be, especially for reasoning and coding. The upside is performance and cost, it's pretty cheap. The trade-off is that some buyers will have procurement, security or geopolitical concerns. So I'd be thoughtful about where I'd use it, how I deploy it, even if you want to, even if you want to use it and go down that path. There's also GLM or people know it as Z.ai. It's another one you'll see pop up a lot.

• 16:08Speaker 00

  I actually did an episode on it, especially if you spend time on Hugging Face and Ollama or just local model Twitter, you're going to see it a lot. The thing to know is that some of these models can be weirdly good for specific jobs. So I wouldn't ignore them just because they're not the obvious brand name. You can test them, you can read the model card, you can check the license and just play with them. But what you might deploy in the sense of for your business or for what you're doing might be very different. There's also Mistral, which is the European model family. I think they're based in France. If you care about efficient models and, you know, they do a lot of releasing, a lot of practical developer use cases, they're pretty good.

• 16:53Speaker 00

  It's a strong model with a pretty builder-friendly posture. But the downside is the lineup is a little confusing. Some models are open, some are commercial. So, you know, some question marks there. Microsoft also has their open model family. It's called FI. I think it's interesting if you care about smaller, faster, lower latency models. But for a lot of use cases, I haven't seen it work very well. And honestly, there are new models showing up all the time. It feels like every other day. And that's why Hugging Face matters. You're not going there just to find the big models.

• 17:35Speaker 00

  You're going to find these like weird specialist models, these like community fine tunes, quantized versions of stuff, these forks and model cards that tell you whether something's actually usable for the workflow. So you don't know, you don't need to memorize all of this. But the takeaway basically is that there's these ecosystems. And your job as a founder or just, you know, someone who is playing with these models is to pick a model family that fits your workflow, that you connect with that company. You like how they do things and then go from there. You can play with a lot, learn a lot, and then, you know, pick a family. So how do we make this whole thing real?

• 18:20Speaker 00

  Like if you actually want to run Gemma, here's how I would do it. I would start with LM Studio. I would download LM Studio, it's free to download. You open the app, you search for Gemma 4. If your machine is solid, try E4B. But if your machine is a bit slower, older, I would look for E2B. And then I would look for the quantized version. If you're using the GGUF path, because you want the model to run just a lot more comfortably. Once it downloads, open a chat and just ask it something really simple. You know, I'd use like a business prompt because I want you to feel the value immediately. It's sort of an aha moment.

• 19:03Speaker 00

  Maybe it's something like, read these customer notes and turn them into a one-page memo about what customers are struggling with, what has changed and what the business should fix this week. And then just paste like some customer notes or just fake customer notes, just if you want to see the value. The point of this exercise is just basic. The model is now running on your machine and you're using AI without sending that prompt to a cloud model. I believe everyone should try that and feel what that is because I do think that it's, it's just going to be a lot more common and there's just, it's going to unlock your brain in a completely new way.

• 19:42Speaker 00

  After that, go to LM Studio's developer section and start the local server. Because that, that just gets a lot more interesting because other apps can talk to the model on your laptop. Your computer becomes this little AI server. So you can have a script or a prototype or, or just an internal tool that can call the model through local host and you get an answer back. I think that's when you start to see how products are going to get built in the modern age. The second path is Ollama. So install Ollama and run Ollama poll Gemma

• 20:23Speaker 00

  for, then run Ollama run Gemma for colon E4B. Now you have Gemma running locally from a command line. Ollama also gives you a local API port. I think it's on 11434. It is useful because you can connect your own app or script to it. If you want to test a larger model later, you can try the 12 billion, 26 billion, or 31 billion versions, assuming your hardware can handle it. And you can ask an LLM if your hardware can handle it, or you can do it yourself and just suffer through the slowness and the pain of it.

• 21:08Speaker 00

  The third path is Google AI Edge and Light RTLM. I would only use this path if I wanted to build an actual app and a model inside of it. For example, maybe I'm building a mobile app and the model is running on the phone. Or it could be like a browser app where the model runs locally. Or maybe it's a desktop app with a private workflow. Or something on an edge device. Light RTLM is designed for that world. Android, iOS, web, desktop, and edge environments. That is the path from local AI as a demo to local

• 21:50Speaker 00

  AI as a product. So here's the hardware cheat sheet that I would use. If you have eight gigabytes of RAM, start small and keep the first test simple. But if you have something like 16 gigabytes of RAM, you can do some useful experiments with models like E4B and smaller quantized models. If you have 30 gigabytes, 32 gigabytes of RAM, you have way more room to do larger local workflows. If you have a strong GPU or workstation, like a DGX Spark, the bigger models become just much more realistic. And for phones, I would think a lot less about model size and more about the job. So can the model understand

• 22:32Speaker 00

  a photo? Can it summarize audio? Can it classify something quickly? Can it help a worker in the field? Can it run without a strong connection? Can it do something useful inside the app before the user even thinks to ask? Now, let's build the first workflow in our heads. So I would make a folder under desktop called Customer Notes. And inside that folder, I would put 10 support tickets for a specific business. Let's say it's a home health agency or a med spa, or a water damage restoration company. The notes might say something like, I tried to reschedule but couldn't find the link, or the technician didn't explain what

• 23:15Speaker 00

  happens next, or, hey, no one actually confirmed my appointment, or I was charged twice here. Then I would run a local model like Gemma and ask it to create a file called what customers are telling us dot MD, the markdown file. The output should include the repeated complaints, the exact customer language, the likely root cause, the part of the business that seems broken, and the one thing the operator should test this week, the high priority stuff. This is a good first local AI workflow because it's useful and it's simple. What do you have here, right? You have this private messy data. The model runs next to it and the output

• 23:57Speaker 00

  is a memo someone could actually use. And then, once you actually go and, you know, you're going to go and do this and get the output, you're going to, like the unlock I was talking before, like it's going to unlock something in your brain. You're going to see this pattern everywhere. A folder of customer calls become a market research memo. A folder of support tickets become a product roadmap signal. A folder of PDFs become like a risk checklist. A folder of drafts become a pre-send reviewer. This is why I always start with workflows before I'm fine tuning anything. People hear, you know, open model and immediately want to train their own model. And I get it. I get

• 24:39Speaker 00

  why. I was actually the same way. It sounds really cool, but I feel like that's like an advanced move. The practical move, the beginner move, where you should start is just to find a repeated workflow. First, you pick one folder, one model, one output, and you run it like 10 times. You see where it gets confused. You see where you can improve the prompt and add examples. You add a checklist and then you create like a small eval. You know, what's an eval? An eval is just a small, it's just a test that tells you whether the model did the job well enough. For this workflow, for example, the eval could be like really simple. It could be like, you know, take the same 10 customer notes and

• 25:23Speaker 00

  run them through Gemma locally, and then run them through a strong cloud model, a frontier model, and then just compare the outputs. And then, you know, you ask, did Gemma, you know, catch the same complaints? Did Gemma pull the right quotes? And did it follow the format? Did it miss something? The comparison actually teaches you where local is already useful and where you still want that stronger cloud model and, you know, how you should think about the hybrid model I was talking about. That's really how I think about local versus cloud decisions. Use local for private, repetitive, fast, offline, device native, and high volume workflow, stuff that you want to run all

• 26:06Speaker 00

  the time. Use cloud for deep reasoning, giant context, broad research, in cases where the strongest model changes the quality of the answer. So you use both when the product has sensitive data and hard reasoning. A lot of valuable products will work that way. You know, local first pass, you do the cloud escalation, human approval for anything important. I think that's the way work's going to get done. So I want to give you three startup ideas where local AI actually matters. And these are the kind of businesses I would look for, niche, useful, cash flowing businesses that you don't need to raise venture for,

• 26:46Speaker 00

  and tied to a painful workflow. The filter is pretty straightforward. So I look for a customer with sensitive data, repeated review work, bad software usually, expensive mistakes, like mistakes that will cost them a lot, and a workflow that happens close to the, to the device. That combination is like the interesting zone for me. So let's go through the three ideas. I want you to steal these ideas. And at the very least, it'll get your creative juices flowing with how you can use local AI to run model, build apps, and make money. Idea number one is a local QA reviewer for home health agencies.

• 27:28Speaker 00

  So home health agencies have nurses and caregivers, and they go into people's homes, and they write, you know, visit notes and updating care plans and dealing with billing and compliance. The paperwork is a pain. It takes a lot of time. If you've ever witnessed it in person, but it matters so, so much. Like a missing detail can create a billing delay, and a vague note can create extra admin work. And a mismatch between the visit and the care plan can create a ton of risk, and we don't want that. So, the first version is a local desktop app for the agency. The agency drops in visit notes and care plans and dictated transcripts. And then the model is going to review them before the submission

• 28:13Speaker 00

  and should look for flags. So it's going to flag things like this note mentions dizziness, but vitals are missing, or the caregiver described a medication change, but the follow-up instructions is pretty unclear, or the note may not support the billed service level. The buyer mostly cares about fewer documentation problems before the billing or the audit or a supervisor review. So if you solve that, you have their attention. Now, I don't want to just give you the idea. I mean, how would you actually grow this? If I was starting this business, how would I grow this business? I would actually start it as a service. So I would find five small home health agencies, and I would offer to review a batch of notes.

• 28:55Speaker 00

  I would do the review with AI, helping behind the scenes with the local AI. And I would inspect everything manually with like human beings, myself first. I would write down the 20 issues that keep showing up. And those issues become the checklist. And then the checklist eventually becomes the product. So you have this wedge. It's pretty simple where you're catching documentation problems before they cost the agency time or money, and then you build from there. I love this business and totally would start it. The second startup idea is an offline field report co-pilot for restoration contractors. So think water damage, or fire damage, or mold remediation, things like that. Those

• 29:37Speaker 00

  teams are out there field taking photos, recording notes, documenting damage, and creating reports for homeowners and insurance adjusters. I unfortunately had this, so I know a little bit about it. The job is actually pretty visual. It's also physical, right? There, it happens like away from a desk. And the report matters because the report becomes the handoff between the technician, the customer, the office, and the insurance process. So how would we build a product here? The first version is a mobile app. So a technician walks through the property, takes photos, record voice notes, and the app drafts the report before they leave the site. So it can flag missing pieces while

• 30:19Speaker 00

  the technician is still there walking around. You mentioned the basement, but there are no basement photos. You took a photo of ceiling damage, but there are no moisture meeting reading, things like that. Or, or maybe like the affected room is like missing. Could be the homeowner explanation is way too technical. Here's a clearer version they can understand. And the last part of that is underrated. In a stressful home damage situation, clear communication is part of the product, right? So if you had that, that would be key. How would I grow this business? Well, I would pick one niche first. I wouldn't go after everything. So say I'm going after, you know, water damage restoration. I would talk to owner operators. I'd look at their current report templates, study the software

• 31:04Speaker 00

  they use, which is some old stack, and I'd build around the checklist that's already in their head. The demo is actually the easy part. You know, send me three old jobs and I'll show you how fast your techs could create reports. If that works, then the product could expand from there. That's just a wedge, right? It can go into QA and estimates and insurance packets, customer updates, and training new technicians. But I would start with the field report because it's specific and obviously super annoying. And, you know, I just think that There's, when you look at some of these old softwares that these people are using,

• 31:45Speaker 00

  I recently had some water damage at my apartment and I was seeing some of the software and it's antiquated. Like it's stuff from like the early 2000s. So I think that there's this opportunity to create local AI native software and wedge now. And that's why I said in the beginning, like I think there's a 24 month window and opportunity to do some of these products. Let's go into startup idea number three. So startup idea number three is a local pre-send reviewer for professional services. So every professional service firm or 99.9% of them has a version of this workflow.

• 32:27Speaker 00

  Someone writes a client email, a proposal, a memo, a contract summary, an investment note, an HR note, and then someone, and then ask someone else to check it out before it goes out, like a review. And it happens constantly. Law firms, accounting firms, wealth advisors, recruiting firms, even consultants have a version of this. So the first version is you build a local desktop app that reviews outbound drafts before they leave the company. So for a wealth advisor, it could be flagging language that sounds like a guaranteed return, which is a definite no-no. For a law firm, it'll flag a sentence that sounds too definitive.

• 33:10Speaker 00

  For HR, it's going to flag sensitive employee information that should stay out of the threat. For an agency, it flags a promise that the scope does not support. And for an accountant, it flags a number that doesn't match the attached file. You'd be surprised how often that happens. The product is basically a second set of eyes for sensitive work. It's basically schmuck insurance is the way I think about it. And maybe that, that would be the name, schmuckinsurance.com. Someone tell me if that's taken. How would I grow the business? I would start with one vertical and one document type. For example, I would do email review for independent wealth advisors.

• 33:52Speaker 00

  Not everyone, probably not the big banks to start, independent wealth advisors. I would interview 10 advisors and ask them which emails make them nervous. I would collect anonymized examples. I would turn their real concerns into a review checklist and I would build a local tool that checks drafts against that checklist. Obviously, this is so sellable because the buyer understands this behavior and they already asked someone to check the draft. So you're just basically giving them a faster first pass that lives closer to their client data and internal rules. I love this idea and hope, I hope a few of you take it. By the way, if you're not building one of these ideas tomorrow, I still think you should learn local AI

• 34:37Speaker 00

  because it does change how you work with your own files. So I think just like from a personal productivity perspective, it's still super, super helpful. So if you're working at a company, say, and you just want to be more productive, so you have more time to scroll TikTok or watch movies or hang with your family, make a folder called Local AI Lab and then put 10 files that matter to your work in that folder. It could be anything from sales calls or meeting transcripts, old tweets, ideas that you have. Then run Gemma, whatever model you choose to make it produce one useful artifact.

• 35:19Speaker 00

  And then ask it to create a weekly business pulse or ask it to find what's changed in customer conversations or meeting notes. Ask it to group feature requests by the actual pain behind it. Ask it to review drafts and tell you what your audience keeps responding to. The key basically is to produce a file, a memo, a checklist, a brief, a report, or a review that you can reuse. A chat answer is nice, but a useful artifact changes that workflow. This is the first rep I would recommend. A model reads the folder, the model writes the file, you inspect it, you improve the workflow and then you run it again.

• 36:01Speaker 00

  If you do that a few times, your brain really starts to connect the dots. You start noticing where private data is trapped in folders. You notice which reviews happen over and over again. And you notice which workflows depend on someone checking a form, reading a note, comparing two files, cleaning up a report, or writing the same kind of memo week after week. Hopefully this episode got your creative juices flowing because once you see the pattern, you start spotting local AI businesses everywhere. You can learn enough of the map to spot where these models belong without turning yourself into a local engineer overnight.

• 36:40Speaker 00

  I believe some AI, local AI belongs in the cloud and some, some AI belongs in the device. And a lot of the best products over the next couple of years are going to combine them both. So if I was starting today, what I would do is I would run Gemma locally. I'd read model cards on Hugging Face. I'd learn the difference between LM Studio and Oh Llama. I'd play with Google AI Edge and then look for one boring workflow where local AI actually makes the product better. Those categories are things like private data, offline work, camera audio context, or low latency, or if there's a high repeated API cost. If there's a buyer who feels better when the model is just close to them,

• 37:24Speaker 00

  a workflow where a small agent team could re-check, summarize, and prepare work every day. That's like the hunting ground. Local AI is just way easier to understand once you stop treating it like a model benchmark conversation and start treating it like a product conversation. You have to ask yourself, where is the work happening? Where is the data? Where is the device? Where is the trust issue? Where is an annoying review loop? And then you answer those questions and you just start seeing the idea. So overall, I hope you understand a little about the core things you need to understand about local AI, some of the models, some of the apps you need to download, some of the workflows that you can build, and some of the business opportunities that exist.

• 38:09Speaker 00

  I just don't see that many non-technical people playing with local AI. And over the last two months or so, I've gotten deeper and deeper into it. And it's just, like I said, it's been connecting the dots and I'm grateful for it. I hope you have a creative day. I read every single comment in, on YouTube and respond to most. So I'll see you in there. Share, share this with a friend who you think could benefit from understanding local AI in a clear way. And I'll see you next time. Happy building.