Why software keeps breaking? (Lost Episode)

[Sponsor] Sentry: Catch, trace, and fix bugs across your entire stack. Use code: prime for $100 in free sentry credits → https://trm.sh/sentry [Description] Ever wonder why even the biggest apps and systems seem to break out of nowhere? In this lost episode of The Standup Pod, we dive into the hidden chaos of modern software development, technical debt, and why keeping complex codebases alive is harder than ever. [Hosts/Guests] Prime: https://thestanduppod.com/contributor/theprimeagen Teej: https://thestanduppod.com/contributor/teej-dv Trash: https://thestanduppod.com/contributor/trash Casey: https://thestanduppod.com/contributor/cmuratori [Partners] Linear: https://trm.sh/linear Sentry: https://trm.sh/sentry [Socials] x.com/thestanduppod instagram.com/thestanduppod tiktok.com/@thestanduppod facebook.com/profile.php?id=61573286173150 [Other Places to find us] Spotify: https://open.spotify.com/show/01A062kejnXFkJE01bjN5J RSS: thestanduppod.com/feed.xml Clips: @TheStandupPodClips Website: thestanduppod.com

• 0:02Speaker 00

  Hey, we're doing a quick bonus episode of The Stand Up. It's gonna be short, it's gonna be hot, it's gonna be spicy. Casey is gonna give us kind of a thesis on the change of software and where things are going. And perhaps kind of his thoughts on the web world and on the programming world.

• 0:17Speaker 01

  Why does software always break?

• 0:23Speaker 00

  Uh, anyway, sorry. Casey's gonna give us a thesis on why does software always break. Casey.

• 0:30Speaker 02

  This is a hot new thesis. No one's ever seen it before, except I posted it back in 2021.

• 0:38Speaker 00

  It's just gotten more true since then, Casey. It actually has gotten more true and reposted.

• 0:45Speaker 02

  So essentially, what I wanted to try and point out to people, because I don't think that this is appreciated enough. We talked about it in the previous episode of The Stand Up that we did, where I was saying, like, all of, like, a lot of the things that I see people saying positively about AI coding, I don't necessarily disagree with. I just think they're not including this really important other part, which is that a lot of the things that people are talking about doing with AI are things that no one should have had to do in the first place. They're being done because we've created such a bad programming environment that nobody wants to interact with it anymore, right? It's always breaking, it's always changing, it's got way too many layers of abstraction, most of those layers don't work very well, there's way too much complexity, like, all this stuff. So yeah, it's like that, I mean, it makes perfect sense why someone reached for an AI

• 1:30Speaker 02

  tool, because why do you want to do it, right? And so I just want to talk about sort of a separate part of that, which is just the very unreliable nature of software nowadays, and especially builds. Like, it's like, okay, I got this piece of software that I wrote, and like, what are the chances that I could compile it again in six months or something? Or a year? Or two years, right? Or even, not even compile? What's the chance that it'll run still if it uses things like REST APIs with some web service, right? So, you know, I've got these different web services I'm using, so even if I don't have to recompile my thing, or even if it's an interpreted thing that runs and I'm keeping the same version of the interpreter or whatever else, it's gonna make these API calls out to web services and those services could change. So what I did, and hopefully we can put the graphs up as I'm talking about them, but

• 2:15Speaker 02

  I posted these on Twitter, they're very simple. It's just taking the fact that, look, if you assume, right, and I say this in the tweet stream, that if you take the chance that something will be, remain working after a year, is some probability. So like, 90% chance that this Twitch REST API that I'm gonna call, a 90% chance that they will have kept it the same a year from now, so that it will still work, right? My app sends this REST API call out to Twitch, it expects a certain response back, and they're not gonna change it in some breaking way, right? In a year. If we assume that we just have some probability, like 90% for that, that we can pick, we just imagine one, right? Or, you know, so we don't have to measure, just imagine in your head 90% or something

• 2:59Speaker 02

  like that, then the chance that your code remains working after X years is just given by P to the X times N, where N is the number of those calls to things that you have, right? So you can graph this. And what I showed is that if you had a 99% chance that every API call you use, 99%, right? Which is way higher than anything in the web world typically has after a year. But 99% across all tools, the graph still looks pretty bad, right? You look, and you look at it, it's like, okay, after a year, it's like, it still looks pretty darn bad. It goes down pretty rapidly based on the number of tools. So you can see the graph I show is like, one tool, two tools, three tools, four tools,

• 3:44Speaker 02

  five, just it goes down, right?

• 3:45Speaker 01

  I love that book. Great Dr. Seuss book.

• 3:48Speaker 02

  Yes.

• 3:49Speaker 01

  Great book.

• 3:50Speaker 02

  It's, he captured, Dr. Seuss was, a lot of people don't know, that he, the doctorate that he had was in computer science. It's very, you know, a lot of people don't know.

• 4:01Speaker 01

  He was software engineering. He was one of the few to become a professional engineer in software, as opposed to the rest of us, yeah.

• 4:06Speaker 02

  Yes, yes. That, that book, One Fish, Two Fish, Red Fish, Blue Fish, that's where Two Fish comes from. He did that cipher. Yeah, and, and along with Bruce Schneier. So anyway, um, so if you pick something less than 99%, like, the graph for just 90%, it's very depressing, right?

• 4:29Speaker 01

  Hey, Brian, don't forget, Skron Plannings later.

• 4:43Speaker 02

  No.

• 4:46Speaker 02

  Can't do this!

• 4:58Speaker 03

  Product manager here, we've decided to put you on call.

• 5:07Speaker 02

  I can't, I can't do, I can't do on-call.

• 5:11Speaker 00

  You're on-call?

• 5:14Speaker 00

  Lucky?

• 5:15Speaker 02

  What are you saying?

• 5:17Speaker 00

  With Sentry, on-call is so much fun. It has all of our errors, logs, spans, metrics, replays, and even releases in one spot.

• 5:27Speaker 04

  It has all the context we need to solve any bug. Even if half the time Sentry beats me to the fix anyway. Just 'cause software development's torture doesn't mean on-call has to be.

• 5:37Speaker 02

  Wow, that's amazing. Does Sentry also attend my meetings?

• 5:41Speaker 04

  Oh, that's the best part.

• 5:43Speaker 02

  No!

• 5:43Speaker 04

  Sentry can actually make on-call fun, but it can make it less scary. Everything you need to fix real issues in one place.

• 5:51Speaker 01

  Buckaroo. Got that 6:00 PM post-mortem planning, don't forget.

• 5:57Speaker 02

  Look at that graph. It's like, nothing's going to work. Even at 90% chance that it remains working over a year, not 90% chance it will break, 90% chance it will work over a year, it is dismal. Even with just one or two tools, it's horrible, but if you're talking about using, you know, seven, eight APIs, which is very common in software nowadays, forget it. You simply will not be able to use this thing if you update anything, right? And so I just wanted to underscore, like, this is not goo-good, right? This is not a very sustainable way to work on things, and it has incredibly bad knock-on effects.

• 6:40Speaker 02

  One of them is what we talked about, which is that no one wants to do this anymore. It's not satisfying to work in this world, because you're constantly just, you feel like you're drowning, I feel like, right? With all these, like, things, oh, this broke, this changed, oh, they changed the way React worked, oh, we're doing this now, oh, that, that web service doesn't even exist anymore, they, they canceled it, and now it's this other web service. That's not programming to me. That's like some kind of weird, like, manage, management feeling thing. It's like, it feels like you're a manager more than a programmer, because you're just trying to make this house of cards not collapse by, like, you know, constantly shuffling things around. And so, A, I totally understand where people are coming from when they want to reach for these AI tools.

• 7:21Speaker 02

  I totally get it, and I think that if I was developing software in this world, I would reach for them too. So like, that's why when I say, like, I don't use any AI tools right now, I'm like, asterisk, but that's because I'm working on very specialized stuff. I'm not having to do these things, right? And so it's very obvious to me why I have a very different opinion. It's not that I, like, if somebody, like, just don't understand the power of AI, it's like, no, it's like, I do actually understand the power of AI. I just think that a lot of the power of AI is really, it's only correcting pretty bad situations that we sort of created ourselves, that's mostly what it's doing for you in these scenarios. But also, I think this has so many bad effects and everything else. Performance, uh, security is the biggest one, but things are changing like this all the time.

• 8:04Speaker 02

  You just have so many exploit surfaces, and you have no idea, like, if you asked me to secure some system that's using all of this kind of, this way of working, I just have no idea, I'm like, how could I? Right?

• 8:17Speaker 00

  Here, Casey, I got, I got a good one for you. Let me just show my screen really quickly.

• 8:20Speaker 02

  Yeah.

• 8:21Speaker 00

  This happened just a little bit earlier, uh, when I was doing Chessy, the vibe coding, my vibe coding good times right here. Let me just go like this. Am I, I think I'm, I think I'm not mirrored, right? Yeah, I'm not mirrored. This was the thing that I got back from one of the Claude 3.5, which was, I asked it because I was like, hey, I need to log in, we're gonna use Twitch, and I need to be able to store, obviously, my session data, so I can make sure, you know, when someone makes an HTTP request, I know who I'm talking to. And so when I did that, uh, this is the message I got back, because I asked, like, is the session data secure? Like, if I just knew someone's Twitch ID and username, could I just, like, log in by them? And they're like, oh, yeah, totally, you definitely can. Let's add a JWT, like, this was gonna go out, and then someone could just spoof me, and then just, you know, and then just start...

• 9:06Speaker 02

  And it knew! It knew when you asked it, right?

• 9:08Speaker 00

  But I had to know to ask it, because I've written this problem once before, so it's like, I know what I'm looking for, and it's just hilarious about how, like, dangerous security issues actually are. Because if you didn't know this, like, how would you know to ask to do some sort of JWT or some sort of cryptographically signed thing to verify you're from the server and not from someone malicious? Like, you just wouldn't even know the aspect.

• 9:31Speaker 01

  Brian, I didn't get to see your initial prompt. Did you initially say make it secure?

• 9:36Speaker 01

  How did it know you wanted a secure service?

• 9:38Speaker 00

  I mean, okay, so again, that's fair. That's prompting issues. We can call that prompting issues that I did not tell it to make my login secure.

• 9:45Speaker 01

  But it is funny, though. It is very funny that it's like, oh, you want it secure? I can do that. I love that that is the...

• 9:53Speaker 00

  I got you.

• 9:54Speaker 02

  I'd like to raise a more serious point about that, though.

• 9:57Speaker 01

  Okay.

• 9:58Speaker 02

  Which is that, um, if you imagine how things work currently, they're already bad currently, but let's just take AI out for a second. If you imagine the way things work currently, there is actually one saving grace, which is everyone uses all these frameworks and all these different things. They like, you know, they pile all this stuff together. And it's a nightmare to secure, yes, but what is true about that? Well, you at least know that you are using this thing. And there are security researchers also looking at this thing. So when there is an exploit, you either know or could know. And when it's fixed, if you update, you get that fix, right? So if I'm using one of these, one of these things, I don't know what's a good example to pick here because I don't...

• 10:43Speaker 01

  4chan.

• 10:44Speaker 00

  We can just use, we can just use ExpressJS. ExpressJS just literally had this with the, or this was like six or eight years ago with the RegEx expansions. You could do certain RegEx expansions and have a RegEx DOS effectively. Single request, take down an entire machine.

• 10:57Speaker 02

  So that's gonna say like, I don't know what to pick because I don't know what would be fresh people's mind, but so something like that. So you take an exploit like that, it's like, that's bad. And that happened because you're using all of this other code that you have not secured yourself, and that's not a good thing. But it does mean that when someone finds it somewhere, you will at least know. And if you update your system quickly, you can mitigate that damage to some degree, right? If you imagine the alternative world where, you know, some OpenAI product is just crapping out exploits like that. Because in the system, there's certain things it just didn't know or learned improperly. That's like, when it compressed them down and it's got its weights. It never really understood how to secure this one particular thing.

• 11:40Speaker 02

  Everyone who asked for that thing now has that in their own codebase and there's no tracking. Because we have no idea how many people asked for something that happened to hit that part, right, of the LLM's production chain. And so unlike the ExpressJS, it's where everyone at least knows they got jumped. In this case, we don't even know where all of those exploits are. They're everywhere, right?

• 12:05Speaker 00

  Yes, and then you also have the, the kind of like the creeping problem, which, or the leaky abstraction, which is, everyone has that problem. A quarter of those become open source projects. The LLM learns from those open source projects. It re, you know, it like, it, it, it, you know...

• 12:19Speaker 01

  Like Donald Rumsfeld, you know?

• 12:21Speaker 00

  Like Donald Rumsfeld talked about this, RIP.

• 12:25Speaker 02

  So it's one of those things where it's like, I, I just really do... This is one of the things too, is like, I, I don't actually dislike progress in some Luddite way. Like, I like computers getting better and I like pushing the boundaries of what they can do. And people have this weird thing where they think if you're not pro AI, you're just like some kind of person who just doesn't understand or doesn't like progress. Like, no, it's like, the problem is, I'm not hearing anyone solve these problems. If I thought that these things were in competent hands, where like people were making reasoned decisions and they saw the train wrecks and they were gonna, they had ways of figuring out how they're gonna fix them.

• 13:04Speaker 02

  I would be much less worried. But like, a lot of the stuff I see with AI is it just feels like people who don't really know what they're doing, applying these things way too early. And I think the costs to software are gonna be really high. And, you know, these people are gonna be nowhere to be found, right? They're gonna have collected their huge paychecks from companies that never even made money in the first place, but just were VC funded. They're gonna take a bunch of that money and they're gonna be gone. And they're not gonna clean up the mess, right? So in my mind, it's like, there's only two ways this goes. Either the AIs get so good that they can fix this problem themselves. And just to show you how not anti-biased I am, I'm praying for that outcome, right? Because it's the only one that I see plausibly averting this disaster.

• 13:49Speaker 02

  Or B, they don't. They don't get that good, right? For the next 10, 20 years, no one can figure out how to make an AI that can really be good. And then we're cleaning up this mess. And that is going to be the nightmare to end all nightmares. Because it is just gonna be like, if you thought security was bad now, and it is, oh my God, dude. If you thought performance was bad now, oh my God, right? It is going to be an epic nightmare. So I just think, I, I wish people took this stuff more seriously and they're really not. And that's the part that really, you know, definitely gets me upset about it in that Casey rant way. And so I'm just like, ah, what if this doesn't work, guys? Like, what if it doesn't work?

• 14:30Speaker 02

  You're, you're just, you put all your hopes on this someday getting way better than this right now. What if it doesn't? I'm so nervous. So let's, fingers crossed that it works.

• 14:39Speaker 01

  Yeah, two things. Yeah, two things. Number one, Casey, since you like seeing computers push, you know, to their limits, you are a fan of JavaScript on the server then, right?

• 14:48Speaker 00

  True, true.

• 14:49Speaker 01

  There's nothing, there's nothing to bring that CPU temp up for a smaller amount of users than putting that JS on the backend, boys.

• 14:58Speaker 00

  But only one of the cores, TJ. Only one core.

• 15:00Speaker 02

  I would also like, can everything go through a MySQL query as well? Everything. I mean, every, like, there should be no data stored anywhere except in MySQL. Let's query every, every byte, every pixel of an image from MySQL.

• 15:11Speaker 00

  We call it our sequel at this point. Our sequel.

• 15:15Speaker 01

  Our sequel. Yes, it's our sequel. Uh, my serious, my serious point is, um, I, I heard it framed this way a while ago. Justin Keys, one of the maintainers of Neo Vim, was talking about. He's, he would be very interested to see, and like, this is more towards vision one of AI being good at solving these problems than, than not, is like, can we start seeing AIs reduce the entropy of a system?

• 15:42Speaker 02

  Yeah.

• 15:42Speaker 01

  Right now, they're very good, very good. We'll put quotes, very good. I mean, okay, 10 years ago, we would have considered it literal magic. To type something in and have a website come out of any kind.

• 15:54Speaker 02

  Correct.

• 15:54Speaker 01

  So, so like, so we'll say very good. I'm gonna say very good because it's like unfathomably good compared to what like my prediction for where we would get in my lifetime 10 years ago, right?

• 16:03Speaker 02

  The human language understanding part is like clearly light years ahead of anything we had 20 years ago.

• 16:09Speaker 01

  Yeah, and there's a bunch of other follow-up effects. But so it's like, okay, but so it's like, okay, it can add a bunch of stuff to my system. I need a new feature. I have a clearly scoped bug request. I have some idea that I'm like, I'm the driver. It is my agent, right? I think that's kind of where this, like, I'm sending it as my representative to go solve these things. Um, and it can like maybe do that. But it doesn't actually, like, if I just say, fix the mistakes, or like, I say, make it better.

• 16:39Speaker 02

  Make it more secure. Look at this codebase, make it more secure.

• 16:42Speaker 01

  Right. Sometimes it will pick some, up some obvious ones, which also you're sort of like, okay, but then shouldn't you have gotten that the first round? It is kind of like a strike against the LLM. You have encoded inside of you the secure pattern and you gave me the not secure pattern. That's stupid. I'm not writing the code anymore. Pick the secure one.

• 16:59Speaker 02

  To be fair, to be fair, that is also what a human would do. It has learned correctly. If you ask them to write it, they will write the instructions. Like, why didn't you write the secure one? It's like, oh, I didn't wanna.

• 17:09Speaker 01

  It took longer. That's like, I don't know, there was this paper a while ago where it was like, LLMs were like more tired in the wintertime because they had the time in the system prompt.

• 17:18Speaker 00

  Sadder answers. They had sadder answers.

• 17:19Speaker 01

  They had sadder answers and they worked less hard when it was wintertime because all the training data is like, oh, it's January. And everyone's like, dude, I hate work.

• 17:28Speaker 00

  Yeah, they also got more accurate on math answers. If you said, take a deep breath. They, their accuracy actually like skyrocketed because it was just like, well, because people will, because remember, LLMs are just reflection of written human behavior, right? Like, that's what it is.

• 17:43Speaker 01

  Error minimizing devices, right? So the next most likely thing of after try again and think smarter this time is to get a better answer.

• 17:52Speaker 00

  Yeah, could you try again? But hey, let's take a deep breath, like relax for a second. Could you answer it one more time? You'll get a, you'll, you'll literally get a better answer from most people because they're like, oh yeah, I feel better. Okay, yeah.

• 18:01Speaker 01

  Maybe they solved that, whatever. I don't really know. They're doing all this. But my, my general point being, I don't currently see them overall being a thing that I can let go and it reduces the entropy of my codebase. Which is that's, if it could do that, even just a little, we would be like way more on the track of your vision one where it's like, oh, we can just let this run on Chromium. We'll just spend a billion dollars.

• 18:28Speaker 02

  And just run it for, yeah.

• 18:30Speaker 01

  We're gonna run it for five, 500 million human years on Chromium. And like, in six months, it's gonna come out and Chromium's gonna be tight. It's no longer one gig per tab page. It's gonna be 800 megs, boys. 800 megs only to load that static site, right? And we'll be like, incredible! But that's not, I, I don't see that as a thing people are proposing of like, we are close to. I get the zero to one. I get like smaller features. I get like agentic things for different stuff. I've seen value in each of those. But like, I'm not seeing anybody being like, we just let this run wild on Chromium. It's a superhuman programmer. You know what I mean? Like, where's that?

• 19:13Speaker 02

  Yeah. I mean, that's probably because the direction originally of the research, right, is generative, right? So like, it's why it's called generative AI is because it's, you know, so it may, it probably has taken them a bit to course correct, to the extent that they even want to course correct. To, to do like, okay, what if, what if it's about refinement now, right? Although again, like reinforcement learning is, is kind of in that direction, right? So that is a, is a change. And so, you know, presumably they are kind of working on this stuff. Obviously, it's, you know. I, I don't work on AI, so I don't have predictions about how they're gonna get there. But, but anyway.

• 19:47Speaker 00

  I do wanna throw out, I do wanna throw out something also about what you said a little bit earlier, when it came to just like the security vulnerabilities and all that. I think one of the reasons why this will be the case is that we are also marketing a tool that gives the illusion of experience to people that don't have the nomenclature to understand the usage of the experience, right? I mean, it's the same reason, like, if you've ever chopped wood, the first time you chop wood, you almost hit off your legs. Because you like swing your axe and you realize your legs are too, like, you have, oh my gosh, I almost just hit my shin with my own blade. Like, you learn to stand different because you, it takes...

• 20:22Speaker 01

  I had a buddy who did that.

• 20:24Speaker 00

  Yeah, I know. It's, it's a very reasonable thing for a lot of people to do.

• 20:26Speaker 02

  Stumpy, yeah, you introduced me.

• 20:29Speaker 00

  And so it's like, it's, that's what I worry about is it's, it's not, you know, security, hopefully it will get better. I assume that all things will, it will be better in 10 years than it is today. We, I think anyone will agree with that statement. I'm not measuring how much better, all that kind of stuff.

• 20:44Speaker 02

  We just don't know how much.

• 20:44Speaker 00

  But experience doesn't get better. People will still be the same people. So if you're marketing to the same people, they will objectively build bad stuff and they'll objectively build insecure stuff. They'll do stuff that's crazy. They'll be like, hey, I need to be able to access my database for the client. No one, the L F N are gonna be like, hey, bro, that's a bad idea. They're gonna be like, got your back. Keys are down, client is downloaded, let's go.

• 21:05Speaker 02

  That's what they're gonna have to realize what they're, okay, here is the best possible end game for AI. All right.

• 21:11Speaker 01

  Okay, I'm ready.

• 21:12Speaker 02

  This is, this is the best possible end game. So it all, they get it all work, they, they get to the point that I want, right? They're like, okay, this thing is actually like a master programmer, right? Um, and, and even better because it knows more domains, right? Like it knows more domains than most master programmers are in a certain domain, right? But knows more domain, so it's great. We've got it, this is gonna be great, right? And then what they realize is they've still got that obsequiousness thing going where it's just always like, oh, yes, absolutely. Oh, I'll do that for you, master.

• 21:41Speaker 01

  Certainly.

• 21:42Speaker 02

  Right? You know what I mean?

• 21:43Speaker 01

  Yep.

• 21:44Speaker 02

  It has that kind of like, really like, like, you know, almost slavish kind of disturbing degree to which it's accepting your commands and doing what you ask. And what they realize is that being a dick was critical to master programming. You had to be able to look at the program manager and go, you are so stupid right now. You have no idea, right? You had to turn to them and go like, you don't understand Galactus's pain. You don't understand this thing. It's like, shut up and leave the meeting, right? And so what they have to do is they have to re-change that front fine end, fine tuning process they do afterwards to make the AI a dick. And then, then we have fantastic software.

• 22:29Speaker 02

  I love it.

• 22:30Speaker 01

  But the problem is.

• 22:30Speaker 02

  I want this future, AI companies, where are you? Do this for me. Make the AI dick programmer who changes the world, please. I will be so happy with that outcome.

• 22:42Speaker 01

  You heard it here, we're gonna cut that clip and it's gonna stop before Casey says programmer.

• 22:47Speaker 02

  AI companies, make me the AI.

• 22:49Speaker 00

  They've had that forever. They've, that's probably been there since 10 years ago.

• 22:53Speaker 01

  Oh, man.

• 22:54Speaker 00

  That already exists.

• 22:55Speaker 01

  That got different. Um, okay, one thing for, I wanna go back quick to Prime's thing about, um, the security. I don't, the one thing I'm, I'm feeling somewhat like less optimistic about it is people will be able to build more complicated ways more easily to break systems because of AI. Right, like.

• 23:17Speaker 02

  Oh, yeah.

• 23:18Speaker 01

  So like, not only do I think there's gonna be more services probably, it definitely in, um, I don't know about percentage wise, but certainly in absolute terms, more insecure things on the internet. That, I think that's pretty much undeniable because there's just gonna be so many more things on the internet. Then the second thing is it's gonna, the cost of creating software in this world goes dramatically down. Okay, well, what, what happens when costs go down? People make more of it. Malware, hacking things, DDoS bots, all these, you know, and so it's like, there is something where I, I don't even know that we can say for sure, oh, security is gonna be so much better in 10 years, but people making bad soft, like evil software, bad software, they're already doing that.

• 24:00Speaker 01

  But evil software is going to be more because it's gonna be cheaper, right? And it'll just be like, so I don't know. So it's just something like.

• 24:07Speaker 02

  That's a scary thought actually, because if you think about it, it's like, well, what would you have to do to make an AI system that was good at producing more secure software? Well, we have to write a counter agent that's looking for exploits and we're gonna run that. And that's gonna be part of the feedback loop where we train this AI. Well, the AI training sometimes takes months to run, depending on how, you know, serious this thing is. We, that means whatever AI agent for finding the things that we can write, we have today. We won't have the AI can deploy those things for a little while, but the people who are like exploiting the exploits, they will have that system for finding the exploits today.

• 24:49Speaker 02

  So the cat and mouse game just got, so the, the, the, uh, I guess you don't know who the cat and the mouse is, but who, the exploit finders are always at an advantage. Because they will always have the AI system for finding exploits before the people who have the one that can correct it. Unless, again, there's some really revolutionary change in how these systems are made, right?

• 25:08Speaker 01

  And to circle back to your first point, Casey, of the more things you have in your software stack, the more difficult it is to change anything because it's more likely to take down your entire system.

• 25:16Speaker 02

  Yeah.

• 25:17Speaker 01

  People are still running Windows like XP.

• 25:21Speaker 02

  Yeah, yeah, yeah.

• 25:22Speaker 01

  Like right now, you know, like in mission critical scenarios, they have Windows XP or like the joke I was saying for Fortune was like, they got owned by some like 15 year old buggers. What was it, Prime?

• 25:35Speaker 00

  Like. I, it was some PHP vulnerability, I can't remember what it was.

• 25:39Speaker 01

  Some ancient PHP vulnerability.

• 25:41Speaker 00

  It was deprecated like 4.2, before I started using PHP. It was like so old, it was deprecated.

• 25:46Speaker 02

  I didn't even know PHP ran on Windows XP in those days. What did it?

• 25:49Speaker 01

  Yeah, who knows? I don't know.

• 25:50Speaker 02

  I guess you didn't have to install it. I was just like, I've cra, it's crazy that they were, those were contemporaneous because I was, I always think Windows 7 is PHP or something.

• 25:57Speaker 00

  I believe I did ZAMP server, X A M P P. Anyways, it doesn't matter, TJ, your point.

• 26:01Speaker 01

  That's crazy. But my point though is like, there, people don't update when it's even available to them. It's not like, oh, well, the hacker guys got the new stuff today and the new, the like fix comes out next week. So everyone's up to date next week.

• 26:15Speaker 02

  No.

• 26:16Speaker 01

  Not even close, not even close. Like, so that part is a little bit like, I, just something to be wrestling and grappling with. And once again, like my point through most of it is like, yo, if you know things about software

• 26:34Speaker 01

  and you think more software is going to exist, that is a nice combination of skills to have. I don't know exactly what software development will look like in five years, but my general like thought process, just from first principles reasoning is if you know about software and you know lots about it and you're good at it and your prediction is much more software, those, that is a good combination of things to have. You will be valuable. It might not be hidden keys inside of Neovim. I don't know, maybe Neovim will be dead in five years and it's all, uh, has the brain control thingy from Tesla with the implant, you know, whatever, from Elon, right? And we're, that's the only way we're coding. Sick, but knowing more things about software is still better because I'm gonna say, use a J W T instead of store this in plain text cookie on the front end, right?

• 27:19Speaker 01

  Like, that will be better. It will be better.

• 27:23Speaker 00

  All right, so I also have one more thing that I wanna point out with all of this, all of these things, especially targeting people that are, you know, no coders to make code like things. Is that how I learned how to code was that I first started off and they said, okay, this is an if statement. Um, somewhere between 1999 to 2005 is when I started kind of doing like basic exploration of code. Uh, and so here's an if statement. Okay, this is an if statement. Okay, this is a while loop. Okay, this is a while loop. Okay, I want you to print out a house. Okay, I want you to print out a diamond. You get to print out four diamonds and now you'll notice it gets really annoying. I want you to be able to do it by different sizes. Here's a function, here's how you can make a print diamond function, you know? And you go through all these things and you slowly go, okay, okay, yeah, okay.

• 28:04Speaker 00

  You build up this kind of picture of how code executes, you learn to debug, you do all these things. A lot of people that are vibe coding, I'm curious how discouraging it is to get dropped into a next JS app with Superbase with, you know, Auth Zero with like 900 things and you have to start by debugging a request response?

• 28:25Speaker 02

  Yeah.

• 28:25Speaker 00

  And you're like, what's a server? And you're like already into some sort of like crazy amount of difficulty where it's like, I started by drawing a, a diamond. Right, like, like that starting point is vastly different.

• 28:38Speaker 01

  It was literally the draw the owl, but you had the middle steps.

• 28:41Speaker 00

  I had all the middle steps. They're literally given the owl, be like, draw the owl, right? It's just like, that's really, really, really hard. And so I'm actually curious about the success rate of somebody going through and being dropped in hyper complex projects comparatively to, hey, you're young, you have this free time, we're now putting you through this school, like maybe it's high school time. Hey, let's draw, let's draw a diamond. We're gonna draw a diamond together. We're gonna use Q Basic or some basic language, right, Lua? And you're just gonna do the simplest kind of form of doing things. I'm just, I'm curious what that does to somebody. Uh, cause I know there's gonna be a bunch of success stories. There's gonna be people that are super stoked about programming, they're super stoked about building a product and they will figure out a way, no matter what system you give them.

• 29:26Speaker 00

  But I wonder like, overall, does this actually help make programmers or does this actually hurt the learning process?

• 29:33Speaker 01

  There's also, I think, gonna be a bunch of success stories of people who hate programming, but were able to make whatever their business product thing is without really having to know anything. And that, like, is certainly gonna. gonna happen and you can say like maybe it's a net negative for programming or something like that or for the web although probably most of these people are not building like foundational technologies so hopefully like it does but like

• 29:58Speaker 02

  that is gonna happen

• 30:00Speaker 01

  right yes they're gonna make their own website they're gonna get to they're gonna make uber for cats um and like they're gonna get to make uber for cats and they finally don't have to recruit their other friend and tell them I've got the idea you do the code we'll split it fifty fifty right they'll just be like I'm gonna do the website

• 30:14Speaker 02

  the Winklevoss twins would have owned Facebook they wouldn't have needed Mark Zuckerberg they would have owned Facebook

• 30:20Speaker 00

  yeah but would they have had Justin Timberlake say drop the the it's cleaner

• 30:24Speaker 02

  yeah

• 30:25Speaker 00

  you know like that's the thing is I don't know would you have had that

• 30:28Speaker 01

  yeah

• 30:28Speaker 02

  drop drop the the Facebook meta it's just cleaner

• 30:32Speaker 01

  it's just

• 30:33Speaker 02

  that's what he said they didn't listen to him at the time Zuckerberg realized later

• 30:36Speaker 01

  took a little while

• 30:37Speaker 02

  took a little while to sink in how Justin Timberlake does

• 30:41Speaker 00

  I will also give the inverse which is that you can also ask AI any question and you can repetitively ask stupid ass questions over and over again and it will it will repetitively in the obsequious way I'm not sure how to turn that word obsequious this is

• 30:59Speaker 00

  whatever that word is for saying subservient I know the word I just don't know you know that that term the lowly term uh it will repetitively be like certainly I would love to help you no matter you know unlike stack or yeah stack overflow mods you will not be marked as a duplicate or opinion based you will actually be given a nice full answer every single time so maybe maybe in the end it does actually help more people achieve their coding dreams I don't know I I know I wanted to make sure that people don't think I'm just hyper negative on all those things I'm just I don't understand how this affects the new people or how it affects learning because you I also had no shortcuts when my teacher said build a uh resolve a maze recursively I had to learn recursion at that point

• 31:42Speaker 00

  like there was no other option I had to learn it I couldn't just get an answer I had to figure it out which is like there's something there that is also very special and I don't I don't know where the balance is

• 31:52Speaker 02

  I do think that there might be an argument there for like can we make an AI that's been trained not to really give you full answers right for educational purposes so it's like

• 32:02Speaker 00

  the rabbi GPT

• 32:04Speaker 02

  kind of right like like one that's gonna sort of it's gonna give you a hint to help you get unstuck or to help help point you in the direction that you need to go but it's not going to just tell you how to do the thing because it wants you to learn right and I assume that that is doable you know if you spend uh time training it to do such a thing because obviously they train it to do very complicated things already they have you know post their post initial learning phase stuff is very complicated at this point so I'm assuming that this would if if someone put their mind to it this would be very doable or maybe someone already has and that does sound useful as a learning tool right because you can't a lot of people don't have the ability to ask you know a a great programmer who's sitting next to them or something they don't they don't have that opportunity so

• 32:48Speaker 01

  yeah the two that I know for sure like boot dev shout out promo code by the way for me and prime if you like that

• 32:54Speaker 00

  yeah share the code

• 32:56Speaker 01

  they've got a little like AI helper thing and it's got special prompts for like each lesson and a bunch of other stuff like that right so it's like it can help you when you get stuck on a lesson

• 33:05Speaker 02

  yeah

• 33:05Speaker 01

  and it's like not supposed to be like here's the code right I mean like I'm sure you could prompt objective blah blah it'll give it was like

• 33:11Speaker 00

  yeah

• 33:11Speaker 01

  okay but it's helping you to learn so it like is at least its aim its error minimization is is towards that the other one that I've seen is called like synthesis school or something like that which is like a bunch of AI tutor things but they build it into like a bunch of lessons and so like this is I definitely think uh I I've said this before too I think like people are underestimating in the like learning phase if you are motivated to do it LLM can be very helpful at doing that now you have to make sure you're not getting gaslit into believing some function doesn't exist or but

• 33:48Speaker 01

  for like basic CS fundamentals it it's got all of those books loaded in

• 33:53Speaker 00

  yeah

• 33:53Speaker 01

  direct it's can probably you could ask it what's on you know like page thirty seven of algorithms you know and it'll it'll pull out it probably knows you know what I'm saying like it's seen it so many times on the internet so for like basic CS stuff to get you far on a bunch of these basics like dude you can be asking it you can be doing questions you can be like explain that again explain it you know like in a way that I would understand if I like really like I don't know math like can you relate this to a physics example for me because I understand physics but I don't get computer science like people are definitely sleeping on that aspect of helping them out but it's you gotta do it yourself so that's kind of the

• 34:30Speaker 00

  yeah but that's where that's where the danger is because even if you're semi desiring to learn because it really is a desire magnifier so it's like it really reveals your ultimate desires was your desire to simply get the thing done or was your desire to learn

• 34:44Speaker 02

  right

• 34:44Speaker 00

  and if you if you don't have your desires correct it will or at least in your head what you think of yourself versus in your actions it will it will make it happen

• 34:52Speaker 01

  revealed preference

• 34:53Speaker 00

  revealed preference that's the term I'm looking for

• 34:55Speaker 02

  that can be hard like this is why I think like having uh an AI that's specifically designed for this and you only subscribed to that service would be helpful for people because I don't know about you guys but like if I want to eat more healthy food the easiest way is to just only buy the healthy food so I don't have like bad food around the house right

• 35:14Speaker 00

  yes

• 35:15Speaker 02

  it's much harder if like I buy a bunch of like cake that I love or something and I'm just supposed to not eat it right like

• 35:21Speaker 01

  just only eat one slice of wheat Casey come on

• 35:24Speaker 00

  I'm just like I want to eat the rest of the cake she's like I bought natural popsicles for the kids and I'm like damn I love strawberry you know like it's just very hard for me not to want to eat them you know

• 35:33Speaker 02

  this is what I'm saying for reals though and so I feel like the AI is a bit of a problem that way which is why it'd be nice if you had a service that was like open AI or somebody right they have this one that's like education only and it all it's just it's specifically designed not to give you answers very quickly like it's like I'll I'll dribble out some stuff I'll give you some other stuff uh even has like this maybe you could bake the cost of time in there it's like if you haven't been working on this for a couple of days I'm just not gonna tell you anymore right you have to spend some time trying it yourself right I could see that being very helpful for people because you know inaccessibility is best willpower is second best right so if you can have it you know if you could have that that would be cool and I think that would help bring out those learning abilities of the LLM so that people aren't

• 36:17Speaker 02

  sort of like too tempted to just ask it look just tell me how to do the frickin diamond right or whatever

• 36:22Speaker 01

  yeah just give me the code

• 36:23Speaker 02

  yeah

• 36:24Speaker 01

  give me the code

• 36:25Speaker 00

  and I the thing is is I also can't blame people for doing that I would totally do that too

• 36:28Speaker 02

  this is why I said like have the AI not do that is better even though it's less you know it's it trains your willpower less but willpower is hard it's hard for everybody and

• 36:37Speaker 01

  also if like if you get the experience of actually solving it for yourself a few times then you're like oh it actually was kind of rewarding

• 36:45Speaker 02

  right

• 36:45Speaker 01

  so like it really can help you right you're like oh the first if you start working out and then like suddenly your times start going down on like you know how how fast it takes you to row two hundred meters right and you're like sick that feels good like initially I wasn't thinking it would feel good I didn't see any change at all at the beginning you can get there like same thing can be happening for some of these too where like use in your brain it's getting connected like oh working hard can pay off interesting interesting

• 37:12Speaker 02

  yep

• 37:12Speaker 01

  believe it or not chat believe it or not

• 37:15Speaker 00

  I like how this was gonna be a super short quick episode we are probably like an hour in at this point so

• 37:20Speaker 01

  we're thirty six minutes in uh and that's why I said we're gonna stop and start a new recording so you guys you guys on YouTube you can like it like it like the video right now subscribe leave a comment saying Casey thank you for this bonus episode

• 37:32Speaker 00

  I never ask people I never do a call you know calls to action so hey like the video

• 37:36Speaker 02

  so he has TJ do it

• 37:38Speaker 00

  press the subscribe button TJ streams by the way Casey has computer enhance they have slam dance that

• 37:44Speaker 02

  and the bell what about that bell you gotta click that bell

• 37:47Speaker 00

  oh man get that bell dog

• 37:49Speaker 02

  look at that bell there

• 37:50Speaker 01

  YouTube needs to do why does that bell not make a sound when you hit it you know what I'm saying like you get that little that little Pavlovian response

• 38:00Speaker 02

  yeah

• 38:00Speaker 01

  for smashing the bell ooh that would be nice

• 38:02Speaker 02

  because someone at YouTube is still trying to figure out which Gemini prompt to type in to get that to happen and it hasn't happened yet

• 38:07Speaker 01

  yeah yeah yeah and the sounds are wrong it's because they're like I'm not gonna do it until two weeks before my review cycle so I can have a good review cycle I can promote it to L two and then I can trash this project to get back to L three and then after that we can just delete the whole down vote button and then I'll get promoted to a VP of up or down vote systems and then we can close down YouTube because it's a Google product

• 38:30Speaker 00

  boom yeah C suite that's what we're talking about

• 38:34Speaker 01

  C suite material boys

• 38:38Speaker 00

  alright well hey that was fantastic

• 38:41Speaker 01

  that's the end of the episode goodbye everyone

• 38:43Speaker 00

  bye later

• 38:43Speaker 02

  take it easy everybody