Pod Insights Inspire AI: Transforming RVA Through Technology and Automation Ep 32 - Software 3.0: How LLMs Are Becoming the New Operating System About this episode Send us Fan Mail The digital world stands at a pivotal moment as we witness the emergence of Software 3.0 – a revolutionary paradigm where your words become code and language itself is the new programming interface. Throughout computing history, we've progressed through distinct software eras, each fundamentally changing who creates technology and how. Software 1.0 represented traditional hand-coding – powerful but brittle, requiring specialized knowledge and precision. Software 2.0 in...
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Jason McGuinthe? Welcome back to Inspire AI, the podcast where we explore how artificial intelligence is reshaping business, creativity, and the way we live. I'm your host, Jason McGinty, and today, we're diving into something big, really big. Software, the backbone of our world, is going through a seismic shift. A new kind of computer has emerged, and its programming language is English. This isn't science fiction. This is software 3.0. Let's start by zooming out. If you look at the history of computing, you can actually see three distinct eras of software development.
0:44 Jason McGuinthe? Each has changed who gets to build software, how we build it, and what the computer actually is. The first era is the hand-coded era. We'll call it software 1.0. For most of the last 70 years, software was written by hand, by humans, in programming languages like C, Java, and Python. This is what Andre Karpathy calls software 1.0. It's explicit, it's logical, and it's painstakingly precise. In software 1.0, you write instructions. The machine follows them.
1:28 Jason McGuinthe? It's powerful, but it's also brittle. If you forget a semicolon or mix up an index, the whole thing breaks. This is the era of IDEs, source control, compiler errors, and stack overflow binges at midnight. Then came machine learning. Instead of writing rules by hand, we started training models to learn patterns from data. This is software 2.0, code written by data, not by developers. You feed in examples, run an optimizer, and get back a model, a neural network with millions of parameters, trained to make decisions on its own.
2:15 Jason McGuinthe? This shift was huge. It unlocked image recognition, recommendation systems, broad detection, tasks that were nearly impossible to hand code. But the catch? You need massive data, deep expertise, GPUs, tuning, and lots and lots of math. It's incredibly powerful, but it's not very accessible. And now? We've entered a third era: software 3.0. Instead of writing code or training models, you talk to the machine. Yep, you prompt it. You say, "Write a function that parses JSON,"
2:59 Jason McGuinthe? or "Generate a product description in a casual tone," or "Plan my day based on this calendar." And the machine responds, not with a fixed reply, but with generated behavior in natural language and in your own words. This is why Carpathy says, "We're now programming in English." Prompts have become the new source code. LLMs are the new runtime. It's not just a new tool, it's a new paradigm.
3:35 Jason McGuinthe? And the big idea is this: if prompts are programs and LLMs are interpreting and executing those programs, then the LLM itself is starting to look a lot like an operating system, one that doesn't just run apps. It orchestrates reasoning, connects tools, and adapts in real time to your goals. And we're not just rewriting software, we're redefining what software is. Which brings us to that big idea.
4:11 Jason McGuinthe? How exactly are LLMs becoming the new OS? Let's dig into that. What does it really mean to say that LLMs are becoming a new kind of operating system? Sounds bold, but it makes more sense the more you think about it. In a traditional computer, the operating system is what sits between you and everything else. It manages memory, runs your apps, and gives you an interface, whether that's a terminal, a desktop, or a touchscreen. But today, more and more of our interactions with software are going through a large language model, which of course is a massive shift.
4:54 Jason McGuinthe? Instead of clicking through menus or writing code, you just talk to the computer. You say things like, "summarize this document", "refactor this code file", "find trends in this spreadsheet and turn it into a slide deck", and suddenly, it's not about commands or buttons. It's about intent. And the LLM interprets that intent, makes decisions, and executes the task for us. So when Carpathy says LLMs are becoming the new OS, he's not just talking metaphorically. He's talking about a new kind of interface layer,
5:38 Jason McGuinthe? one that understands language, makes decisions, and coordinates tools on your behalf. Think about it like this. When you open a traditional operating system like Windows or Mac OS, you're the one driving. You launch the apps, move files, switch windows. But with an LLM, it's like the OS meets you halfway.
6:06 Jason McGuinthe? You say what you want, and it figures out how to get there. Sometimes it writes the code, sometimes it runs the steps, sometimes it even spins up tools you didn't know existed. And just like in OS, it's become the hub, the platform. Other apps are now being built on top of the LLM. Apps like Cursor for coding, or Perplexity for research. They treat the LLM the way older apps treated the OS. That's why this isn't just an upgrade. It's a redefinition of what software even is. We're not just using AI-powered tools. We're now living in an AI-powered environment.
6:51 Jason McGuinthe? And if that sounds wild, just wait. We're still early. Like 1960s mainframe early. LLMs are still mostly in the cloud. They're still expensive. They're still limited by context windows, latency, and hardware. But we're moving fast. And what comes next could look a lot like a personal AI runtime. A future where your AI knows you, remembers your workflows, and becomes your default interface for everything you do. So yes, we're entering the era of software 3.0.
7:35 Jason McGuinthe? And the OS, it talks, it listens, and it thinks in tokens. So as Karpathy draws a brilliant analogy, he says LLMs today have traits of utilities. They're metered, cloud-based services. You pay per million tokens. For those of you who haven't been entrenched in AI for very long, what a token is, is basically it's a unit of data processed by AI models during its training and inference, which then enables prediction or generation and reasoning.
8:16 Jason McGuinthe? You can think of a token like a part of a word or a short word. And since you're paying per million tokens, they're treated like electricity. When OpenAI or Anthropic goes down, the world feels the blackout.
8:34 Jason McGuinthe? They also resemble fabs, AKA high-capital factories, like those that produce semiconductors used in microchips.
8:45 Jason McGuinthe? Training a model like GPT-4 or Claude Opus requires enormous investment, which only a few players can do. But perhaps most powerfully, LLMs are like operating systems. They're ecosystems.
9:02 Jason McGuinthe? They manage memory, which are, you know, context windows, orchestrate processes, like function calling or agents, and interface with both humans and machines. They're not just a feature, they're a platform. And we're still in the early days. Like I said, 1960s of computing, currently where cloud access dominates and personal LLMs are rare, but possible. And the desktop revolution for LLMs hasn't happened. Yet. So what happens when we embed LLMs in our tools? You get partially autonomous products, like Cursor,
9:48 Jason McGuinthe? which is an AI code editor, or Perplexity, a research assistant that cites sources and summarizes findings.
9:59 Jason McGuinthe? These tools don't replace humans. They collaborate with them. So how do they work? First, context management, where the AI remembers and structures your work. Then there's multi-step orchestration. The AI must coordinate multiple models or tasks behind the scenes. Then there's custom GUI, so you can visually audit and approve AI suggestions.
10:28 Jason McGuinthe? And finally, there's the autonomy slider. You choose how much control you give the AI. From autocomplete to run wild and refactor my entire repo. This design philosophy isn't about building agents that act alone. It's about building tools that keep humans in the loop. Fast, auditable, effective. Think about it. The Iron Man suit doesn't replace Tony Stark. It amplifies him. It extends his physical capabilities, enhances his situational awareness, and automates low-level tasks, but he's still in control.
11:12 Jason McGuinthe? In the same way, LLMs are tools that enhance human capability, not autonomous beings that replace us. We're not building robots to do our jobs. We're building suits to make our work better, faster, and smarter. One of the most beautiful consequences of software 3.0. Now, anyone can write software. You don't need five years of coding experience to get started. If you can write a clear sentence in English, you can build. This is the essence of vibe coding. Prototyping with feel and flow, not formal specs.
11:54 Jason McGuinthe? Carpathy himself shared how he built an iOS app without knowing Swift, a programming language used to build iOS apps. And then, another app, MenuGen, that turned menu photos into images of dishes, simply because he wanted it and could prompt it into an existence. We're moving from software development as engineering to software development as creativity. And this shift has massive implications for education, entrepreneurship.
12:31 Jason McGuinthe? And inclusion. But there's a twist that most people miss. LLMs are not just tools. They are users of your software, too. We're entering a world where digital agents will read our docs, visit our websites and integrate with our APIs. So we have to build with them in mind. This means creating llm.txt files, which are a lot like robots.txt that help systems understand your website. It also means that offering markdown-based documentation is essential. Replacing things like "click here" in your documentation
13:16 Jason McGuinthe? with curl commands or structured JSON examples. Building endpoints that agents can use, not just people.
13:26 Jason McGuinthe? If humans and agents will co-pilot the Internet together, we have to meet them halfway. Let's zoom out as we wrap up. Software is being rewritten, not just in code, but in how we think about computers, creativity and intelligence itself. Software 1.0 was written. Software 2.0 was trained. Software 3.0 is prompted. And for the first time, the ability to build doesn't belong to just engineers. It belongs to anyone with an idea and a sentence. We're entering the decade of agents, not in the sense of hype,
14:11 Jason McGuinthe? but in the sense of slow, thoughtful, powerful augmentation.
14:16 Jason McGuinthe? The Iron Man suit is real. The GUI for reasoning is emerging. And the autonomy slider is in your hand. So you choose how much control to delegate to the AI. There is so much work to do here, so many products to rethink, and so many dreams to build. And if you're listening to this and thinking,
14:41 Jason McGuinthe? "This is the moment I've been waiting for." You're not alone. Whether you're a developer, founder, educator, or a lifelong learner, this is your time to get fluent in Software's Read ITO.
14:56 Jason McGuinthe? As always, thank you for tuning into Inspire AI. Subscribe, share, and join us next time as we continue future-proofing with AI together. Until then, keep prompting, keep building, and keep your hands on that autonomy slider.
Previous Ep 31 - Beyond Smart: What Happens When AI Enters Moral Territory? Next Ep 33 - Digital Deception: How Scammers are Using Technology to Target You Page updated September 30, 2026 · transcript and summary are machine-generated