AI Isn't Making Engineers 10x As Productive with Colton Voege

In this episode, Ed Zitron is joined by software engineer Colt Voege to talk about how AI isn’t 10xing the productivity of software engineering, the truth about coding LLMs, and what we can actually expect AI to do for software in the future. YOU CAN NOW BUY BETTER OFFLINE MERCH! Go to https://cottonbureau.com/people/better-offline and use code FREE99 for free shipping on orders of $99 or more. BUY A LIMITED EDITION BETTER OFFLINE CHALLENGE COIN! https://cottonbureau.com/p/XSH74N/challenge-coin/better-offline-challenge-coin#/29269226/gold-metal-1.75in --- LINKS: https://www.tinyurl.com/betterofflinelinks Newsletter: https://www.wheresyoured.at/ Reddit: https://www.reddit.com/r/BetterOffline/ Discord: chat.wheresyoured.at https://discord.com/invite/QUUQUP9szv Ed's Socials: https://twitter.com/edzitron https://www.instagram.com/edzitron See omnystudio.com/listener https://omnystudio.com/listener for privacy information.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  This is an iHeart Podcast. Guaranteed Human.

• 0:04Speaker 01

  Empower is all about helping you invest well so you can go out and live a little. And their Free Empower Personal Dashboard shows you your full financial life in one place. You'll be able to see your net worth, monitor budgets, analyze your portfolio, set retirement goals, and more. Whether you're saving for something big, want to treat yourself, friends, or family, or meet long-term goals, Empower helps you see where you stand. You've worked hard for your money, and Empower can help you make the most of it. Download the Empower Personal Dashboard or visit empower.com. Not an Empower client paid or sponsored.

• 0:36Speaker 02

  Small businesses are the pulse of every community. They bring people together, create opportunities, and drive growth. Chase for Business helps business owners like you with personalized guidance and convenient digital tools all in one place. With that guidance and your determination, you can take your business further and help build a brighter future for your community. Learn more at chase.com/business. Chase for Business. Make more of what's yours. The Chase mobile app is available for select mobile devices. Message and data rates may apply. JPMorgan Chase Bank, N.A., member FDIC. Copyright 2026 JPMorgan Chase and Company.

• 1:17Speaker 03

  This is Jacob Goldstein from What's Your Problem? Running a business is hard enough. Don't make it harder with a dozen apps that don't talk to each other. One for sales, another for inventory, a separate one for accounting. That's software overload. Odoo is the all-in-one platform that replaces them all. CRM, accounting, inventory, e-commerce, HR. Fully integrated, easy to use, and built to grow with your business. Thousands have already made the switch. Why not you? Try Odoo for free at O D O O dot com. That's Odoo dot com.

• 1:59Ed Zitron?

  Hello and welcome to Better Offline. I'm your host, Ed Zitron. As ever, buy our merchandise, subscribe to the newsletter, it's all in the episode notes.

• 2:18Ed Zitron?

  And today I'm joined by programmer Colton Voege, who wrote an excellent piece about a month ago called "No, AI is not making engineers 10x as powerful." Col, thanks for joining me.

• 2:28Speaker 05

  Thank you for having me.

• 2:30Ed Zitron?

  So, tell me a little bit about your day-to-day work. What do you do for a living?

• 2:35Speaker 05

  Yeah, so I'm a software engineer and I work specifically in what's called web application development. So, that's kind of like building rich applications that work in the web browser. So, think of something like Amazon or Google Drive or something where you can do a lot of interactive features within the application within a browser.

• 2:54Ed Zitron?

  Right. So, kind of like the fundament of most cloud software.

• 2:58Speaker 05

  Yeah, I would say that if not the majority of engineers work in web application development, then probably the plurality do right now.

• 3:06Ed Zitron?

  And it's also a large amount of how people interact with software is now in this way, though.

• 3:12Speaker 05

  Yeah, exactly. Yeah. And that's why most developers perhaps work in this field now.

• 3:18Ed Zitron?

  And you'd think something like that would be perfect for AI coding environments, surely.

• 3:24Speaker 05

  It is definitely the most applicable place for AI coding of all of them. There's sort of a thing where people are like, you know, "Oh, you know, LLMs don't really work really well for my language." And they'll be talking about something like Rust, which is more of like a a high-performance language rather than a web language.

• 3:45Ed Zitron?

  And what is a high-performance language in this case?

• 3:48Speaker 05

  Uh, just something that's designed to work on like weak hardware or at like extremely high speeds. So, like, for example, like, video games aren't built in Rust. They're usually built in a language called C++. But they're built in a high-performance language because, you know, you're trying to max out as much as you can do.

• 4:08Ed Zitron?

  You're taking advantage of the high-performance compute you have.

• 4:12Speaker 05

  Yeah. Whereas with a website like, you know, like Amazon or something like that, you're really not pushing the limit in terms of interactivity. And so, more modest languages are fine.

• 4:24Ed Zitron?

  Got it. So, you laid it out really nicely in the piece, so you're gonna have to repeat a few things, I imagine. But run me through why AI coding tools can't actually make you a 10x engineer or 10x better engineer.

• 4:40Speaker 05

  Yeah, so basically AI is really good at, AI is almost shockingly good at generating code and it's almost shockingly good at generating code that runs. And it can answer a lot of questions that are like difficult or at least, you know, annoying is probably the better word. It's really good at dealing with things that are annoying. It's really good at, you know, so in coding, we have this concept called boilerplate and it's, it's just code that you have to kind of rewrite a lot. And as an engineer, ideally, you don't have to write a lot of boilerplate. Ideally, you automate things and abstract things so you, you don't need to do that very often.

• 5:22Speaker 05

  But it's sort of like a requirement of the job and it's really good at writing that because, you know, it's just, it's kind of like a low intent, high volume code.

• 5:32Ed Zitron?

  Right.

• 5:33Speaker 05

  A quantity over quality type thing. So it's really good at stuff like that. The problem is that generating code really isn't the hard part of being a software engineer. It's, it's one of the things that really matters and it's like, obviously, if you go to college to learn how to code, what you're going to spend most of the time doing is typing code. But it's, it's really just like one thing that you're doing. Like in reality, you're doing a lot more thinking about like, okay, how does this work with the systems I already have? How do I avoid creating what's called tech debt? So tech debt is basically like an easy way of thinking about it is when you write code that makes writing future code harder. So an example would be, you know, you want to write like a piece of logic that let's

• 6:17Speaker 05

  say computes sales tax. If you're, you know, making an online store, you'd want to compute sales tax based on where they are. You only want to write that once. You don't want to have two different places that, you know, handle sales tax because then if, you know, Illinois changes their sales tax rate, you have to change two places instead of one.

• 6:34Ed Zitron?

  Doesn't, doesn't the very nature of code also mean that it naturally creates tech debt?

• 6:40Speaker 05

  You know, you can try to mitigate as much as you can, but there's even an idea like that all code is tech debt and, and. What LLMs, so kind of what I mentioned before, you know, LLMs are really good at producing boilerplate. But i- if you get to the point where boilerplate is really easy to produce and you're not constantly thinking like, "Okay, how do I avoid doing this in the future?" You start, yeah, e- exactly. You start generating more and more code and just like generating code isn't, it's kind of like writing in terms of like, you know, writing a book, like a good writer writes fewer words rather than more.

• 7:16Ed Zitron?

  Yeah, brevity is the soul of wit. No, it's, because what I've been realizing is that there is this delta between people that actually write software and people that are excited about AI coding. Because I had Carl Brown from the Internet of Bugs on and it was this thing of, yeah, being a software engineer is not just writing 100 lines of code and then be giving the thumbs up to your boss. You have to interact with various different parts of the organization. Early on in the piece as well, you had this bit called "the math" and I'll link to this obviously in the episode notes where it's, "Any software engineer who's worked on actual code in a natural company knows that you have these steps." And where it isn't just like, "I code and then I give the thumbs up and I'm done for the day."

• 7:57Ed Zitron?

  You have to go through a reviewer, you have to wait for them to get back to you, you have to, you have to context switching as well. We have to change different windows and do different things. There's a shit ton of just intermediary work that is nothing to do with actually writing code, right?

• 8:12Speaker 05

  Yeah, absolutely. I mean, it's, it really is more of, you know, there, there's parts of coding that is more science and there's parts of coding that are more art and more, you know, in, in college, it's really common if you're getting a computer science degree to take communications classes because you just have to interact with a lot of people. So the, the standard way that a thing gets done in a software project is you have somebody called a product manager and that person's, that person's job is to think about what the product is as it exists now and what should be in the future and, and like what features we should build, like basically more than anything, it's, you know, what, what should we build next? And then you have designers who are going to decide how that thing should look.

• 8:53Speaker 05

  And engineers have to be this meeting influence because they're the only ones in the process who actually know. What it's like, how hard it is going to be to build something. So a frequent interaction is like, you know, product managers are like, oh, we should add this. And an engineer has to kind of step in and be like, no, that's borderline impossible. Or, you know, that would take six months.

• 9:14Ed Zitron?

  Right. And coding L L Ms don't fix that problem. They don't. Do they actually make it easier to develop products?

• 9:22Speaker 05

  I, I, there are uses for L L Ms in coding and I, I quite like.

• 9:28Ed Zitron?

  I'm not even denying that. I'm just like, how, like, what do, what do those uses end up actually creating?

• 9:36Speaker 05

  Basically, you know, what I found is that I don't really like using L L Ms for most product work, even though they're good at, you know, say like, oh, add a button here that does this. They can be good at that, especially if you have a code base that, that works really well with L L Ms. So some programming languages, some tools, some, we call them libraries and coding. You know, they're basically like shared software. Some of those things work really well with L L Ms because the L L Ms are just trained on the internet. So sites like Stack Overflow and sites like Leak Code and stuff like that. And so i- if they have a large body of this code to work within their training data, like these tools and these languages, they, they tend to be better at writing them.

• 10:20Speaker 05

  And so, you know, I work primarily in JavaScript, as do most web application engineers, and, and L L Ms are quite good at JavaScript. But I, I still don't like to, to use it that much because it, it tends to just not understand context very well.

• 10:36Ed Zitron?

  And what does that mean in practicality?

• 10:39Speaker 05

  So it's, it's kind of just like, you know, understanding the, the existing resources that you've already built in your code base. So this is like the avoiding writing the same like sales tax thing twice. They, they tend to default to rewriting things. And they tend to struggle to.

• 10:55Speaker 05

  Reuse the same like style. So you know, an important thing in coding is maintaining consistent like styles of your code. So like there's, there's a whole theory and practice of actual like how your code should look like as like how the text should look, how you should avoid, what things you should avoid because there's, there's bad patterns, there's good patterns and you know, there's everything in between. Um, I think it's very similar to like, you know, when image generation AI kinda came onto the scene, you had a lot of people being like, all right, well, graphic designers are done. You know, they're out of a job. They don't need to do anything because I can generate a logo now and it like, it looks

• 11:38Speaker 05

  pretty good. Like I can generate a logo for my nail salon or whatever. And look at it. It looks great. But as soon as you try to generate a second logo that looks like the first one and you know, maybe has a slight modification or something like that, it, it's really, it's really bad at it. It's really bad at using the context. And you're like, okay, now I need a graphic designer. And as soon as you need, um, you know,

• 11:59Ed Zitron?

  You end up hiring one anyway.

• 12:01Speaker 05

  Yeah, exactly. And so, and, and you know, that's, it's very similar to engineers, you know, they, a lot of what you're doing is sort of working around context and avoiding redoing things and avoiding moving away from your existing styles.

• 12:16Ed Zitron?

  And what is the, the, the critical nature of styles? Is that because, is that so that people looking at your stuff in the future can say, okay, this is what they were going for? This is so that everything doesn't break?

• 12:30Speaker 05

  Yes, exactly. It's about consistency. It's about, um, so there, there are certain, you know, for example, JavaScript, which is the, the main language I work in, is almost infamous for basically allowing you as the developer to do all sorts of buck wild stuff that you should like never do coding wise. Like a lot of old patterns, a lot of recycled stuff, a lot of, it, it just, it just lets you, it's, it's a extremely varied language in the, the things it supports. And so what you want to do when you're writing in JavaScript is only use the good parts. You want to strategically avoid doing a bunch of bad things. And some of those bad things, there's, there's tools that will automatically detect if you're doing them.

• 13:20Ed Zitron?

  Alright folks, I'm back talking to you about Quince, one of my favourite clothing brands. And I finally pulled the trigger and picked up one of their Cuban-curved chains to go with one of their Pima cotton t-shirts. Gonna be a good look for me? Eh, you'll find out. All of this will sit beneath one of Quince's responsible down-hooded parkas, a ridiculously warm and comfortable coat that I wear constantly in the fall and winter. It can actually really take a beating in the wind, it's wonderful, and I can't recommend it enough. Now, everything at Quince is priced 50 to 80% less than similar brands, because they work directly with ethical factories and cut out the middlemen, so you can give luxury-quality pieces without the luxury price tag. I've been buying from them for years before they advertised, and I heartily recommend their stuff, even wearing one of

• 14:01Ed Zitron?

  their t-shirts as I speak. Give the gift of quality this holiday season. Download the Quince app for app-exclusive offers, or go to quince.com. Get free shipping on your order and 365-day returns. Now available in Canada and the UK too. And when Quince asks you where you heard about them, let them know it was better offline. It's the best way to support the show.

• 14:23Speaker 01

  Empower is all about helping you invest well, so you can go out and live a little. And their Free Empower Personal Dashboard shows you your full financial life in one place. You'll be able to see your net worth, monitor budgets, analyse your portfolio, set retirement goals, and more. Whether you're saving for something big, wanna treat yourself, friends or family, or meet long-term goals, Empower helps you see where you stand. You've worked hard for your money, and Empower can help you make the most of it. Download the Empower Personal Dashboard or visit empower.com. Not an Empower client paid or sponsored.

• 14:54Speaker 06

  OnDeck is built to back small businesses like yours. Whether you're buying equipment, expanding your team, or bridging cash flow gaps, OnDeck's loans up to $400,000 make it happen fast. Rated A+ by the Better Business Bureau and earning thousands of 5-star Trust Pilot reviews, OnDeck delivers funding you can count on. Apply in minutes at ondeck.com. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 15:25Ed Zitron?

  What are these bad... Is it just that Java... What's so special about Java that it allows you to do that?

• 15:31Speaker 05

  Yeah, so JavaScript, which is technically different from Java. Um, yeah, no problem. Basically, the reason JavaScript has all these bad parts is because it was created in 10 days as a toy language at Mosaic, which was the precursor to Firefox. You know, the first browser that really gained traction and set the internet on the course of what it is.

• 15:51Ed Zitron?

  I'm old enough to have used Mosaic somehow.

• 15:53Speaker 05

  Yeah, so JavaScript was created in like 10 days, almost like just as like a thing to test, like just something to try, like, okay, what if we put a coding language into the browser that people could just like ship with their websites? And in response to that, other companies said, okay, well, we need to support JavaScript as well. And so when Internet Explorer came out, they shipped their own version of JavaScript, which was slightly different. And so you have this engineer that was kind of built on like these like very hacked-together principles that was then rebuilt with slightly different principles. And then what you have now is 30 years later, sort of the conglomeration of all those things into one language.

• 16:36Speaker 05

  And it's improved dramatically in that time. But the nature of coding languages is there's a lot of backwards compatibility, like a video game, where you know, you want to be able to build a computer that can run a game from 1991. So you want the browser to be able to still run code largely that was written in 1995 or whatever. So a lot of that stuff still exists. And what they've done is introduce new patterns that can be better. And so there's a sort of an equal amount of like avoiding that. And then also just consistency. So there are times where there's two good ways, two right ways to do something, but you always want to do it only in one way so that anytime someone reads it, they see that pattern, they say, okay, I know exactly what this does.

• 17:21Ed Zitron?

  Right. And this fundamentally feels like something large language models are bad at because they don't know anything and they don't create anything unique either. They just repeat.

• 17:33Speaker 05

  Yeah, exactly. I mean, they are statistical inference models and so they are very good at generating what they think should come next based on probabilities. And you can, when you're training a large language model, you can try to push it in one way, push it in another way to say like, no, don't do that, do that. But like trying to do that on the broad spectrum of all things code is basically impossible. And so you're going to get things that default to the way that they were done on the internet and so much like, like I said, JavaScript is 30 years old. It's extremely, it's gone through a lot of dark times in terms of patterns. And so on the internet, there are just vast swaths of terrible JavaScript and

• 18:17Ed Zitron?

  Oh God, so it's just these models are trained on bad code as well as good code?

• 18:21Speaker 05

  Yeah, exactly. And, and you know, I'm sure they're trying very hard when they're training the models to filter some of this stuff out, but like trying to do a broad filter on hundreds of thousands, millions even of webpages of code.

• 18:35Ed Zitron?

  And how old was it again? Like decade?

• 18:38Speaker 05

  I mean, JavaScript's been around for 30 years. I just have most programming languages that are in use right now.

• 18:44Ed Zitron?

  Right, but this particular one seems particularly chaotic in how it's sprawled.

• 18:50Speaker 05

  Yes, JavaScript is a particularly thorny language, yeah.

• 18:53Ed Zitron?

  Why, so what you're describing, I'm not trying to put words in your mouth, is that this stuff doesn't do the stuff that everyone's saying it does. Like it's not replacing engineers. It doesn't even seem like it could replace engineers. Like it's just not, it doesn't do, in fact, maybe it's fair to say that it doesn't do software engineering.

• 19:12Speaker 05

  That's almost the perfect way to put it. It does coding, but it doesn't do software engineering. And software engineering is kind of this broader practice of like everything that comes together around coding. So, you know, some people really integrate LLMs deeply into their everyday work and they do similar work to what I do. So, you know, there are people who are primarily having like LLMs write their code and they can be good engineers. But they're intervening pretty constantly from what I understand and you know they're having to sort of redirect it make sure it stays on patterns and all that stuff and there's just kind of you

• 19:54Speaker 05

  know this is the reason that I don't really use L L M's that much is there's just a constant tension between the the lack of context they have and you know what you want them to do and constantly reviewing the code and making sure it's up to standards when I know I could just write it and you know it'll take about as much time than as it would take prompting and re prompting it's just I get better code that way and that's that's what I care about most.

• 20:19Ed Zitron?

  So do you think it's kind of a mirage almost the productivity benefits?

• 20:24Speaker 05

  I think it would vary a lot. I think yes broadly. I think that there are productivity benefits but like these like huge outsized like oh my gosh I'm galaxy brained right now I'm doing so much. I think that that is largely a mirage based on like extrapolation of like small wins. I talk about this a little bit in my article that you know there are one thing that I use L L M's for a lot and not a lot sometimes but that they are really good at is you know sometimes you're writing code and you're like I need to write a thing that I will only run once and I will throw in the trash or I need to write a thing and it uses this tool that like I don't

• 21:07Speaker 05

  have the time to learn but like I'd really like to have this tool like I'd really like to just like use this once and so you can you can vibe code something and not really understand what the code does you can run it once and not you know validate the output and make sure it works fine and you know I've saved in that time you know it might have taken me five hours to like learn how to use this tool properly and like learn how to use it with good standards it could even have taken me more and instead I spent 20 minutes.

• 21:36Ed Zitron?

  Wouldn't that be dangerous because you don't know how it works?

• 21:40Speaker 05

  Exactly, exactly why I like to only use it for these like one-time, like low, you know, so for example, I wrote some code the other week and I was basically, I basically refactored some existing code. So I adjusted how it worked a little bit and I realized there was a way that I could break some existing code with it, but I didn't have an easy way of checking across the entire code base. So we have tests in our code base that would, you know, catch issues, but there's always some code that isn't quite up to, up to par with tests. And so I had this idea that, you know, a simple, a really simple language parser could go through and make sure that this code was right, you know,

• 22:20Speaker 05

  something that is only like 30 lines of code. But language parsing is really complicated and the tools to do it are, you know, there, there are a lot of them and they're very well-supported and because language parsing is a huge deal. But it would, it would take me a lot of time to learn that. So I just, I just vibe coded a little tool. I said, "Hey, find me every case where I use this function and I call it like this in this entire code base." Or actually, I said, "Write a script that will find me that." And I looked at the code. I was like, "Yeah, that looks right. That looks right. That's right." I ran it. It said there was no issues in my code base, so I intentionally created an issue just to make sure it worked and it worked. It showed it. And so then I got rid of the intentional issue and I was like, "Okay, this is probably good." And I pushed

• 23:05Speaker 05

  my code and it turned out to be good. But that's, you know.

• 23:09Ed Zitron?

  That was the seams low stakes.

• 23:10Speaker 05

  Exactly. And it's, it's what I would say wouldn't scale. So if I really needed to start doing like language parsing constantly, like I was doing it daily at my job, I simply would have to learn. Like you said, you know, how do you know that there are any issues? I would have to learn the tools.

• 23:28Ed Zitron?

  Right.

• 23:29Speaker 05

  The time that I saved was by avoiding learning for, for this one thing. But eventually, like if you're going to make something your full-time job, you have to learn it because you can't fully trust the output.

• 23:42Ed Zitron?

  Because otherwise it isn't your job. Like you were just kind of mimicking.

• 23:47Speaker 05

  Yeah, and you know, you have to, you know, the LLMs make mistakes in occasionally extremely catastrophic ways. There's a thing called slop squatting. Have you heard of slop squatting?

• 24:00Ed Zitron?

  No, but please tell me. I love this term so much already.

• 24:03Speaker 05

  Yeah, so basically, you might have heard of like domain squatting. So this is where like,

• 24:08Ed Zitron?

  I think I know what this is and I'm very excited to hear more.

• 24:12Speaker 05

  Yeah, so like, you know, this was a thing where like in 1991, you're like, okay, I think the internet's gonna be big, so I'm gonna grab Nike.com.

• 24:21Ed Zitron?

  Right.

• 24:21Speaker 05

  I'm just gonna hold it. So that's squatting a domain. So slop squatting is basically where you,

• 24:29Speaker 05

  the LLMs, you know, they're statistical inference machines. They don't, they don't actually understand what they're doing. And so, you know, sometimes they will import software. They will,

• 24:43Ed Zitron?

  Is this when it looks on GitHub for something that doesn't exist?

• 24:48Speaker 05

  Yes, and it will just add code to your project. It's like import this thing. And it will install it and it'll say like, okay, this is the library that you want to use. And it's not right. It's either misspelled or it's, you know, for example, you would be looking for a library called left-pad and it would import something called left-pad with no dash.

• 25:12Speaker 05

  And so what people realized is that they can, you know, when the LLMs, they're statistical machines. And so they frequently, they do the same thing a lot. They make the same mistakes a lot. So you could grab that library, left-pad with no dash, and input code in there that works, that does the thing that the library is supposed to do, but also retrieves, you know, all of the secrets that are in your environment. Or like looks for crypto wallets or like production database passwords and stuff like that.

• 25:47Ed Zitron?

  And if you're someone that can't read code or can't read that kind of code, you would have no idea this is happening.

• 25:53Speaker 05

  You have no idea, and even if you're somebody who does read code really well, if you, you know, look at that, you said it's imported left, you're like, that sounds right, that looks right. Like unless you are really familiar with this library, and even if you are familiar, you might just like gloss right over it. So it's really, really, really dangerous. And like, this is the type of thing that could, you know, like maybe if the LLM is making you even twice as productive, you know, that doesn't mean much if, you know, there's a chance you could destroy your entire company with a catastrophic security breach, you know?

• 26:29Ed Zitron?

  Yeah.

• 26:29Speaker 05

  Leak all your database to this hacker.

• 26:33Speaker 05

  And so, yeah.

• 26:34Ed Zitron?

  Not good.

• 26:35Speaker 05

  Yeah, exactly.

• 26:36Ed Zitron?

  And is that becoming more prevalent?

• 26:39Speaker 05

  I haven't heard much of it like happening in the wild, but it's just one of those things that is bound to happen. Because again, these are just statistical models. They don't, they don't have the ability to really reason about the actual nature of the things that they're doing. They can try, you know, they can make a sub LLM call and ask the other LLM, like, does this look right? But then you're, you know, burning more and more tokens and...

• 27:03Ed Zitron?

  And also at some point, you are trusting the statistical model to measure a statistical model's ability to do its job.

• 27:09Speaker 05

  Yes, exactly. And, you know, it kind of devolves. Anybody who's used an LLM for coding knows that the deeper you go into like a single prompt, like the more back and forth, the larger the context window, the more garbage it gets. And so, like, as you have things like working off of other LLM input, which is effectively what you're doing in a large context window, you know, it's just, the LLM is sort of reprocessing the text that it generated and that you've added to it. It steadily gets worse the later in the context window. So basically, all of these sort of mitigations are, you know, they've made surprising progress with the way that like,

• 27:51Speaker 05

  you know, things like this don't happen as like just raw hallucinations of like, I think this library exists. They happen a lot less now than they used to, but they're just always, they're always a risk.

• 28:03Ed Zitron?

  And they're also always gonna be there.

• 28:05Speaker 05

  Yeah, pretty much.

• 28:06Ed Zitron?

  It's not really something you could, unless we invent new maths.

• 28:10Speaker 05

  Yeah, I mean, at least with the way we approach AI right now, which is based purely on language as tokens. And it can't really fundamentally like understand things outside of, you know, word probabilities.

• 28:36Speaker 01

  Empower is all about helping you invest well so you can go out and live a little. And their free Empower Personal Dashboard shows you your full financial life in one place. You'll be able to see your net worth, monitor budgets, analyze your portfolio, set retirement goals and more. Whether you're saving for something big, want to treat yourself, friends or family, or meet long-term goals, Empower helps you see where you stand. You've worked hard for your money and Empower can help you make the most of it. Download the Empower Personal Dashboard or visit empower.com. Not an Empower client paid or sponsored.

• 29:07Speaker 06

  Cash Flow Crunch on Deck's small business line of credit gives your business immediate access to funds up to $200,000 right when you need it. Cover seasonal dips, manage payroll, restock inventory or tackle unexpected expenses without missing a beat. With flexible draws, transparent pricing and control over repayment, get funded quickly and confidently. Apply today at ondeck.com. Funds could be available as soon as tomorrow. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 29:38Ed Zitron?

  All right, folks, I'm back talking to you about Quince, one of my favorite clothing brands. And I finally pulled the trigger and picked up one of their Cuban-curved chains to go with one of their Pima cotton t-shirts. Going to be a good look for me? You'll find out. All of this will sit beneath one of Quince's responsible down-hooded parkas, a ridiculously warm and comfortable coat that I wear constantly in the fall and winter. It can actually really take a beating in the wind. It's wonderful and I can't recommend it enough. Now, everything at Quince is priced 50 to 80% less than similar brands. Because they work directly with ethical factories and cut out the middlemen. So you can give luxury quality pieces without the luxury price tag. I've been buying from them for years before they advertised and I heartily recommend their stuff. Even wearing one of their T-shirts as I speak. Give the gift of quality this holiday season.

• 30:23Ed Zitron?

  Download the Quince app for app exclusive offers or go to quince.com. Get free shipping on your order and 365 day returns. Now available in Canada and the UK too. And when Quince asks you where you heard about them, let them know it was better offline. It's the best way to support the show.

• 30:44Ed Zitron?

  So where is this pressure coming from? Because it feels like it's everywhere. And you've got people like Paul Graham who are wanking on about, oh, the I met a guy who writes 10,000 lines of code. I think he said, what is it? Are we just finding out how many people don't know how coding works?

• 31:02Speaker 05

  I think a lot of that, yes. Like I said, it's exactly the same way as like people, you know, when the first image generation models were like, oh, great. We don't need graphic designers anymore. And then they, or you know, oh, great. We don't need customer support chat anymore. Because they fundamentally don't understand what those roles do, you know, they're, uh, they think graphic designer and they think image generator. But a graphic designer is a, is a human being that's, that's dealing with different stakeholders, dealing with people saying like, no, no, no. The, the logo can't have that or, you know, yes, the logo must have this. Yeah, yeah, exactly. And they're dealing with, you know, uh, different requirements. And then they're needing to make variations, which is something that LLMs are not always very good at. Or, uh, I should just say generative AI is not always very good at.

• 31:48Ed Zitron?

  Um, yeah. It's so strange.

• 31:51Speaker 05

  Part of it is, is, is people who, you know, like I said, you know, you get these like brief bursts where you're like, oh my God, I just saved so much time. And you extrapolate that like, so some of it I think is engineers who are just like, who are actual engineers who know how to code, who see these things. And they think like, I did this today and like, I must've been so much more productive as a result. You know, I saw this one thing happen, but they, they don't really actually measure in depth. What they produced and was it more than what they would have produced? There, there have been some studies to measure that and they haven't looked particularly good for LLMs. You know, if you, uh, actually compare people, you know, using AI versus not using AI.

• 32:33Speaker 05

  The results don't always look particularly great for AI. And, and one thing that's very common out of that is that people overestimate their performance.

• 32:41Ed Zitron?

  And I think that might be a problem across all, like, a lot of people don't. I, I also think my grander theory with all of this is a lot of people don't know what work is. Like, a lot of these investors and managers and such, and even people in the media, don't seem to actually know what jobs are and how the jobs work. And they think that things like coding is just, like I said earlier, yeah, just walk into work, I'll write 10,000 lines of code and I'll walk home. But now I can write 20 billion lines of code because that's all my job is.

• 33:10Speaker 05

  Yeah, absolutely. I mean, thi- this is, this is something I talk about in my article is that there's always this like degree of separation. And the people who are talking the most about AI coding aren't really coders and they're not really providing like detailed reproduction steps.

• 33:28Speaker 05

  You know, I, I know engineers who, who love using AI like every single day and they use it for like all of their projects. And I know really good ones who do that. And if I asked them like, hey, how could I like be more likely, be more like you? How could I be a better coder? One of the last things they'll say is probably, you know, start using AI more. It, it's really just a tool to accomplish part of their job. And so, so yeah, I think there's a lot of, you know, there, there, there's probably, you know, plenty of genuine people who just like, you know, they've never written a line of code in their life. They pull up lovable, they say, generate me an app that does this. And like they legitimately are like, oh my gosh, it actually worked. Like I, I can code.

• 34:13Speaker 05

  And they just, they just, you know, naturally, you know, they don't realize it. Um, they don't realize that like there is so much more to this actual practice. And, you know, they're, they're, they're just not in tune with the way that coding actually works.

• 34:29Ed Zitron?

  Yeah, it's a little bit sad as well because it really feels like a lot of this is just the, the point you've made about like the image generators. It's just this immediate moment of wow, look what this could do. Imagine what it could do next. And then you look at what it could do next and it can't like it looks like it can generate code, but it can't actually generate software. Like it can't, it doesn't seem to do the steps that make software functional and scale. Because there's these tendrils.

• 34:58Ed Zitron?

  From software into the infrastructure to make sure it can be shown in different places, or to make sure that it actually functions on a day-to-day basis.

• 35:08Speaker 05

  Yeah.

• 35:08Ed Zitron?

  And it's like, yeah.

• 35:10Speaker 05

  It's a pretty simple like curve. Like, you know, you start out and it generates so much code, that's like, pretty correct, really fast. If you start a completely bare project from,

• 35:24Speaker 05

  just like a lovable prompt or something like that. But as you go on, you know, it's like the curve kind of flattens and it goes down and it becomes less and less productive. And I think eventually, usually, like pure vibe-coded projects hit a pretty big wall, because you're just introducing so much code, and it's not consistent, it's not,

• 35:46Speaker 05

  it's not using shared tools. And so you eventually just end up with this, you know, they call it spaghetti code. It's, you know, code that is so interwoven and difficult to understand that like you can't actually see what's going on with it.

• 35:59Ed Zitron?

  Yeah, feels like context is the whole problem. Just because even not say, like, LLM-generated writing is dogshit. And I think it's worse than code, because code, code is functional in a way that I don't think writing has to be. Like, writing conveys meaning, but good writing is usually more than just, "I am entering this writing into someone's brain, so something happens," in the way that code is. But with writing, they share the same problem, which is, great writing has contextual awareness. It builds, it connects. There is an argument, or there is an evocation from it. In the case with software, it appears to be, if you don't know every reason that everything was done, and fully understand the reasons that were previous, and the reasons that are happening

• 36:43Ed Zitron?

  right now, you kind of will fuck something up naturally. Even if you do know how to code, if you just don't read any of the notes, if you, if it's not clear why things were done, things will break anyway, right?

• 36:55Speaker 05

  Yeah, and great writing, you know, it knows when to, you know, a great writer knows, you know, when they need to explain something, and when they don't need to explain something. You know, they know, you know, "I'm writing a trade publication, like, I don't need to explain how friction works." Or, "I'm writing a, you know, a public press release. Okay, I do need to explain how friction works," or, you know, whatever. So, it's, it's very... You know, these are the things that the LLMs are like, yeah, exactly, just like very not good at, you know, they can, they can generate stuff that looks good, but it's just the more you try to build on top of it, the more it'll, it'll end up restating like, you know, LLMs are really good at writing like the classic school level five paragraph essay.

• 37:39Speaker 05

  But everybody who actually writes anything at all knows that like the five paragraph essays that you wrote in high school are like terrible and like, nobody wants to read something structured as like, you know, premise, three argumentative paragraphs, conclusion, like that's, it's really bad, like unconvincing writing.

• 38:00Speaker 05

  And it's the same with, you know, LLMs are really good at making toys and like, like quick things that like are, are fun or, you know, maybe a little bit useful in certain situations, but really bad at writing things that, you know, like, like a book level type things, like, you know, AIs are horrible at writing books because they restate things and they lose track of what they're talking about and all of that stuff.

• 38:22Ed Zitron?

  And the more stuff it creates, right? The more it looks over, the more confused it gets.

• 38:26Speaker 05

  Exactly. It's, it's really, really similar. I, I think that coding has really just followed the same trajectory as like all of these other things where we're like, oh, we don't need copywriters anymore. Oh, we don't need designers anymore. We don't need graphic designers. We don't need this, that and the other. We don't need lawyers anymore. We can have an LLM PR lawyer. And you just very quickly realize that like these, these jobs aren't, you know, dumb factory, like producing, you know, pulling a lever type jobs. They're, they're about interaction.

• 38:59Ed Zitron?

  And that's not an insult to factory labor. That is very hard work, but it's not like a repetitive action that is always the same thing.

• 39:08Speaker 05

  No, not at all.

• 39:08Ed Zitron?

  In a simple way, yeah.

• 39:09Speaker 05

  And like a, a factory worker, you know, is, is doing a lot more than just pulling levers, of course.

• 39:16Ed Zitron?

  But it's not like just hitting a button, but I think people condense coding to this thing.

• 39:22Speaker 05

  Yeah, it is. It's very similar to like robotics in factories too, where you know, the promise is like, oh, you know, we'll just have a robot do this thing that a human does. But the human is doing a lot more than just, you know, pulling the lever. They're like observing the process. They're making sure that things are not getting broken or getting gummied up and stuff like that. So there's just, there's just limits to what machines can do when they're not actually intelligent. And that's just what it comes down to.

• 39:47Ed Zitron?

  Do you buy the, any of these companies like Google are writing 30% of their code with AI?

• 39:55Speaker 05

  It, it, the thing about like those numbers is that they are super easy, maybe not to game, but to, first of all, I don't know anyone who's actually like measuring this in a really like effective way. Because the thing about your like coding editor is that it's, a lot of people, I don't like to use the AI auto completion, but some people do, you know, there's, there's pieces where it'll, you know, you'll start writing some boilerplate and it'll like pop up, you know, what it thinks you mean to write and you'll just hit tab, the tab key on your keyboard, and it'll just like finish the line of code that you were writing. And maybe it was wrong.

• 40:32Ed Zitron?

  And that would be written in AI?

• 40:34Speaker 05

  Yeah, yeah, exactly. And it might even, very often what it produces is wrong, but it's like 75% right. So you're like, oh, I can just like use these keystrokes. I can save these keystrokes and I can just fix what it did wrong. Um.

• 40:47Ed Zitron?

  Oh, God, it really, it's like saying autocorrect wrote parts of your book.

• 40:52Speaker 05

  Yeah, it really is. And, um, you know, there are times where like, an AI does fully write the code for a feature. So people use, you know, uh, things where they can prompt an LLM and it will, uh, with like what they want the feature to be or what they want the bug fix to be. And it will go and it will write all the code and it will make the sort of, we call it a merge request, but that's the request for new code, um, uh, that then gets reviewed. It will go all that way. But the thing is, you know, it might have written the code, but somebody took time away from, you know.

• 41:36Speaker 05

  Their normal job would just be writing code to write a really good prompt to make sure it didn't screw it up and then re-interact with it. And so, you know, did it write the code? Yes, but did it do the task? Not really, because it needed somebody else to do some support work to make it even possible for it to do it. And that person could have... That person needed to be technical and needed to be able to say like, "Oh, you need to look in this part of the code base." And so you end up just getting the same type of like actual work from the actual specialist who knows how to code, the same amount of work. They're just doing something slightly different.

• 42:14Ed Zitron?

  Cole, it's been such a pleasure having you here. Where can people find you?

• 42:19Speaker 05

  For sure. My blog is colton.dev, C-O-L-T-O-N.dev. I don't post that often because I work full-time and I just post when something really gets to me, but you know, I might say some things here and there.

• 42:36Ed Zitron?

  And you have your excellent blog that I brought you on for.

• 42:38Speaker 05

  Yeah, yeah, that's my most recent one. You can feel free to check it out. I'm sure the link to that will be in the description. But yeah, it was great being here.

• 42:44Ed Zitron?

  Thanks so much.

• 42:54Ed Zitron?

  Thank you for listening to Better Offline. The editor and composer of the Better Offline theme song is Matt Osowski. You can check out more of his music and audio projects at mattosowski.com, M-A-T-T-O-S-O-W-S-K-I.com. You can email me at [email protected] or visit betteroffline.com to find more podcast links and of course my newsletter. I also really recommend you go to chat.wesyoured.at to visit the Discord and go to r/betteroffline to check out our Reddit. Thank you so much for listening.

• 43:27Speaker 07

  Better Offline is a production of Cool Zone Media. For more from Cool Zone Media, visit our website, coolzonemedia.com, or check us out on the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts.

• 43:59Speaker 01

  Empower is all about helping you invest well so you can go out and live a little. And their Free Empower Personal Dashboard shows you your full financial life in one place. You'll be able to see your net worth, monitor budgets, analyze your portfolio, set retirement goals, and more. Whether you're saving for something big, want to treat yourself, friends, or family, or meet long-term goals, Empower helps you see where you stand. You've worked hard for your money, and Empower can help you make the most of it. Download the Empower Personal Dashboard or visit empower.com, not an Empower client, paid, or sponsored.

• 44:30Speaker 06

  Cash flow crunch? OnDeck's small business line of credit gives your business immediate access to funds up to $200,000 right when you need it. Cover seasonal dips, manage payroll, restock inventory, or tackle unexpected expenses without missing a beat. With flexible draws, transparent pricing, and control over repayment, get funded quickly and confidently. Apply today at ondeck.com. Funds could be available as soon as tomorrow. Depending on certain loan attributes, your business loan may be issued by OnDeck or Celtic Bank. OnDeck does not lend in North Dakota. All loans and amounts subject to lender approval.

• 45:00Ed Zitron?

  Game's on a break? Don't doom scroll. Try rips on Underdog. Buy a digital pack, rip it open, and reveal a real, physical card. Football, basketball, even Pokemon. Packs start at just $5. Ship it straight to your door, or sell it back instantly at full market value. Max value up to $125,000. Download Underdog and get in on the chase. Deposit $5, get five free packs. Real rips, real cards, real cash. Only on Underdog. Unleash your dog. 18+, terms apply. Top grails are extremely rare.

• 45:31Speaker 00

  This is an iHeart Podcast. Guaranteed Human.