Casey Muratori: Surprises In Computer History And Where Bad Code Comes From

Casey Muratori is a video game developer and programming creator also known for his talks about the history of software and computing. We talked about surprising parts he found while digging through computer history, where bad code comes from, and his career story. • My ergonomic keyboard project I mentioned, you can follow along here: https://read.compose.llc/ • The Kickstarter page for it: https://www.kickstarter.com/projects/ryanlpeterman/compose-simple-ergonomics-beautifully-done Podcast links: • YouTube: https://youtu.be/jHLbL1Eg4gM • Apple: https://podcasts.apple.com/us/podcast/the-peterman-pod/id1777363835 • Transcript: https://www.developing.dev/p/casey-muratori-surprises-in-computer Thank you to this episode's sponsors for supporting my work: • Jira by Atlassian: Get more work done with your favorite agents and models all in one place, check them out at https://jira.dev/ • WorkOS: makes your app Enterprise Ready with easy to use APIs to add SSO, SCIM, RBAC, and more in just a few lines of code, check them out at https://workos.com/ Timestamps: (00:00) Intro (01:08) Digging into computer science history (04:08) What shocked him (16:23) Dijkstra was depressed (26:17) The personal side of goto considered harmful (35:09) The anatomy of a 35 year mistake (50:12) Clean code horrible performance (58:33) How to write high performance code (01:01:18) Where bad code comes from (01:06:37) Why design docs before code is a bad idea (01:09:20) The only unbreakable law in software engineering (01:15:57) How he got into programming (01:21:16) Why he didnt work in big tech (01:30:44) Should you work at a startup early on (01:34:52) What video game engineering is like (01:39:57) Why preventing recursion is reasonable (01:43:24) Is vibe coding bad for the industry (01:50:17) Technical reading recommendation (01:54:27) Advice for his younger self (01:56:54) Outro Where to find Casey: • Website: https://caseymuratori.com/ • Newsletter: https://www.computerenhance.com/ • X/Twitter: https://x.com/cmuratori • GitHub: https://github.com/cmuratori • YouTube: https://www.youtube.com/@MollyRocket • Bluesky: https://bsky.app/profile/cmuratori.bsky.social • Twitch: https://www.twitch.tv/molly_rocket • Handmade Network: https://handmade.network/m/cmuratori • Wikipedia: https://en.wikipedia.org/wiki/Casey_Muratori Where to find Ryan: • Newsletter: https://www.developing.dev/ • X/Twitter: https://x.com/ryanlpeterman • LinkedIn: https://www.linkedin.com/in/ryanlpeterman/ • Threads: https://www.threads.com/@ryanlpeterman • Instagram: https://www.instagram.com/ryanlpeterman • TikTok: https://www.tiktok.com/@ryanlpeterman Referenced in this episode: • The Root of the Root of All Evil: https://computerenhance.com/theroot • The Big OOPS: Anatomy of a 35-Year Mistake: https://www.youtube.com/watch?v=wo84LFzx5nI • Edsger Dijkstra's "Notes on Structured Programming": https://www.cs.utexas.edu/~EWD/transcriptions/EWD02xx/EWD249/EWD249.html • Donald Knuth's "Structured Programming with go to Statements": https://doi.org/10.1145/356635.356640 • Edsger Dijkstra's "Go To Statement Considered Harmful": https://www.cs.utexas.edu/~EWD/transcriptions/EWD02xx/EWD215.html • Ivan Sutherland's Sketchpad thesis: https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-574.html • Clean Code, Horrible Performance: https://www.computerenhance.com/p/clean-code-horrible-performance • Where Does Bad Code Come From?: https://www.youtube.com/watch?v=7YpFGkG-u1w • The Only Unbreakable Law: https://www.youtube.com/watch?v=5IUj1EZwpJY • Dijkstra letter's reference: https://medium.com/@acidflask/this-guys-arrogance-takes-your-breath-away-5b903624ca5f#.1rdj838x6

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  The amount of stuff that you learn about what was going on at that time is just mind-boggling.

• 0:06Ryan Peterman?

  This is Casey Muratori, video game developer and sometimes computer historian through his popular talks. And I asked him all about the stories in computing he uncovered. Dijkstra was depressed at that time?

• 0:18Speaker 00

  He literally says, like, in this period, I was very depressed because of these reasons. Look at the correspondence. They didn't have the ability to do like pithy Twitter replies, so it was just on paper.

• 0:30Ryan Peterman?

  What drew you towards video gaming?

• 0:33Speaker 00

  There's a bunch of things I could say about that, but they're probably all BS.

• 0:38Ryan Peterman?

  Here's the full episode.

• 0:44Ryan Peterman?

  There's so much, you know, AI doom content out there. I'm hoping that we can just talk about interesting things in software engineering and just see where it goes. So I'm not going to ask anything that's, uh, you know, all this AI doom. You're gonna, you have two years left to be a software engineer, so.

• 1:04Speaker 00

  Fantastic. I will try to keep the AI mentions to a minimum.

• 1:08Ryan Peterman?

  There's this famous quote by Donald Knuth. The big part of it is premature optimization is the root of all evil. And I remember I've, I've heard that before, but I, I don't think I know the full history behind it. And, you know, why is that significant?

• 1:25Speaker 00

  So I guess we can split that into two parts, like what's the history behind it and why is this significant? The significant part is probably the easiest to answer because it's, you know, for whatever accident of history, it's something that people really remembered, like it's stuck. And so the reason that it ended up being significant is people keep repeating it and they keep interpreting it or reinterpreting it. So the effects of that phrase, far beyond what anyone may have done with it in context at the time, or what they meant, or any of those things, or the history. The effects of it are like still felt to this day. Some people, uh, I think probably correctly, you might say,

• 2:07Speaker 00

  interpret the term with a fair bit of nuance and, and say like, oh, well, you know, it can mean a number of different things, or it's trying to capture something subtle or whatever. Other people are very blunt about it and just think like, oh, it just means I don't have to think about the performance of my code till like the end of the project, right? Like, like premature just means any consideration of performance is not worth it. Eventually we'll see where the code is slow and we'll fix it or something, right? And so, uh, it's, it's a significant phrase for that reason, at least to me. You still hear it to this day and it's at least, you know, depending on how you want to account for it. It's at least something that's 50 years old now.

• 2:47Speaker 00

  And so that's an incredibly lasting impact for a rule of thumb, or however you want to look at it. So, uh, on the history side, this was really, uh, when I decided to, to do a talk on this, I knew I wanted to do another computer history talk, um, because I'd given one before at, at, this was at something called the Better Software Conference. It's, you know, there's been two of them so far. And at the first one, I did a history talk and I wanted to do another history talk. And this is going to be like, I'm probably only going to do two of these in this second one. I wanted to do it on the history of this phrase because when I started looking into the history of this phrase a while back, I found that there was just a lot of stuff there that I didn't know.

• 3:27Speaker 00

  And this is true, I mean, I guess I would say every time I've ever looked at computer history, every, every time I've go, I, I refer to it as dumpster diving, but it's not, it's not like, that makes it sound derogatory. Uh, I just think, but I think of that as kind of what I'm doing, but it's not really like, it's actually a very enjoyable experience, uh, to go back through history and try to pick, pick through what happened. But, um, so to do this talk, I basically went and tried to do a, a pretty detailed, you know, analysis of like, how did this phrase end up being put in print? You know, what were the worked examples at the time? What were they thinking about? Why might it have been said?

• 4:02Ryan Peterman?

  What did you learn in doing all that research about these famous computer science phrases?

• 4:08Speaker 00

  The thing that shocks me, it shocked, it both times, even though I've now done it twice, the amount of stuff that you learn about what was going on at that time that you just had no idea is, is just mind-boggling. And, and I would point out that this is for a talk, you know, I'm not a historian. I don't do this for a living. I'm not at a university somewhere doing computer history research. So what I think of as a pretty thorough read to the history that I did for the talk is actually somewhat shallow. Like if you had time and the resources, you could go access some of these people's personal papers, make appointments to see things that are not really in the public record so much and so on. So even just with publicly available things that you can get,

• 4:52Speaker 00

  the amount of stuff that I found was just really kind of, uh, intriguing to me. And so, um, I'll give like a, a kind of a brief overview of sort of what, what I thought was the story of this kind of phrase in a way. I think there are kind of two, I guess I would say there, there are sort of two threads that are good to understand, that come together, and this is what I tried to portray in the talk as well.

• 5:19Speaker 00

  The first thread is this idea, um, of a thing and, you know, we didn't really do an intro to this, to this podcast, but, um, so I'll just, I'll say it now. I'm a big fan of your show. When, when you said you want to be on the podcast, you didn't have to tell me what the show was. Like, I've already watched it. I, I love it. And one of the people you had on, for example, was Barbara Liskoff. And in that interview, she actually refers to something called the software crisis, right?

• 5:47Speaker 00

  And I think you prompted her a question about this as well, right? So a lot of people don't really know about the software crisis. They don't know that that was a term. They don't know this something that happened because it's kind of ancient history now. It was in the sort of late 60s, early 70s. This was a thing. And what this was is originally, you know, computer hardware in a sense that you or I can't really conceive of it in practice because it was so far before our time, right?

• 6:17Speaker 00

  That basically computer hardware originally was very simplistic. And so the things that you were going to do with it could be described very easily. And really all you were trying to do when you programmed a computer was just translate a fairly straightforward definition of some computation into like machine instructions. The kind of thing that we would kind of think about today as like hand optimizing assembly code was really just all computer programming kind of was, right? It was just like, okay, we're gonna, we're gonna take this problem description and translate it in.

• 6:50Speaker 00

  But during the 60s, computing power was advancing to the point where people could consider doing much more significant things with it. They could run payroll systems. They could do analysis of, of data sets and they would want to be able to be flexible and change the sorts of computations they were doing and all these other sorts of things. They were starting to have graphical displays, uh, you know, um, Ivan Sutherland's Sketchpad is like in the early 1960s, right? And so what they found was at that time programmers were not really ready to make that transition. They didn't think about things the way we kind of all now take for granted where it's like, oh yeah, you know, I built this library for loading JPEG images. And now when I want to load a JPEG, I just call load JPEG and it's done.

• 7:31Speaker 00

  And, you know, all of these things that we take for granted, that all had to, that all had to be a cultural shift. The idea that there were going to be these, um, abstractions and these ways of, of sort of breaking down a much more complex system into parts that we could manage, that actually was a transition that had to happen. And so the software crisis was kind of this period where people were starting to say, the methods that we're using for programming will not scale to these larger problems that we want to do. We've got to figure out something, uh, to do about it. So that's one thread of history that's coming. Another thread of history that's coming that's, you know, it's a little bit more specific to Donald Knuth, who was the first person to really put this into a worked example.

• 8:15Speaker 00

  Like he's kind of who I would credit with the origin of this phrase, even if there are people who might debate that fact. And to me, it doesn't really matter who said the phrase, like the phrase lives on in it on its own. So, you know, I, I, I wouldn't, I wouldn't even argue with someone if they wanted to sort of, uh, make other claims about it, but just in general.

• 8:34Speaker 00

  He was going like, like at Stanford at this time, he was sort of learning about the fact that you could do these profiles, like taking actual execution profiles, figuring out where time was spent in the execution of a program. And using that to guide your decision-making about how, where are you going to spend time developing the software? Again, something that even today, performance is maybe not as much of an emphasis as it maybe should be in software. But even today, with that in mind, most people understand the concept of an execution profile. They understand that you could go into Krobe and see like a network profile of how that's working. You could, uh, see a flame graph of, of, you know, where time is being spent or all these sorts of things.

• 9:18Speaker 00

  Again, you have to remember, if you rewind the clock back far enough, these were all brand new concepts, right? A sampling profiler, a block brace profiler, uh, you know, a, a binary instrumentation profiler, all these sorts of things were not just tools that everyone assumed that they could just go have and, and had seen at some point. Some people may never have even realized that was, was a thing, right?

• 9:41Speaker 00

  So that's another thread that's happening. And those two things converge effectively in 1974, roughly, to get us this statement. Knuth is very much into this idea of structured programming, which is sort of a solution, a proposed solution to the software crisis. Maybe a proposed solution is the wrong way to say it. It's a certain set of ideas that are trying to be employed to improve the situation, let's say. He's very much into this. We can talk about what that is, a sort of a separate tangent if we want to later. He's very much into that. He's thinking about that.

• 10:21Speaker 00

  He's, he's thinking about this idea that programmers can't just be about writing all of this very tightly optimized assembly language code. And that's all we're going to spend all our time doing. We have to be thinking bigger picture. We have to be picking and choosing our battles. We have to make concessions to maintainability. You know, if this particular routine is not really accounting for hardly any of our runtime, we should not be obfuscating it with all of these like hand unrolling the loops and all these things. Remember, again, underlying all this, they don't have like massive optimizing compiler kind of passes like we have with LLVM now, right? They don't have all of this extra stuff. So if you wanted something unrolled, you were going to unroll it yourself. If you wanted some kind of loop, hoist a loop invariant out, you probably had to do that yourself.

• 11:06Speaker 00

  Like a lot of that stuff that we take from today, again, not in there. So that sort of this, he's very into structured programming. He's starting to see these execution time profiles. They do a thing at Stanford in the 1970. Actually, he does it with some students where they take a look at FORTRAN programs and they instrument them to see where the time is being spent. Not in exactly the most rigorous way that you might imagine. That's another thing from my talk that I kind of go into. But he's sort of saying like, look, we, we as programmers need to start taking the decision about what to optimize more seriously. And he kind of has a fairly nuanced take on it. If you actually read the section of his 1974 paper,

• 11:49Speaker 00

  Structured Programming with Go-To Statements, where kind of the first time this was really put to print that I know of, he does, he does say it in an ACM Turing lecture a little earlier, but he would have submitted, like, based on my timeline, he would have submitted the manuscript for this publication prior to the lecture. So it was the first time that I know of that he actually committed it to paper would have been there. But he may have said this a lot in lectures. He kind of obliquely refers to that in the Turing lecture, that maybe he had said this around, you know, around town, so to speak. So, you know, where this was first uttered, I don't think anyone knows. But point being, when he says that phrase, kind of, it's in this context of saying, look, we, we need to be very concerned about efficiency.

• 12:33Speaker 00

  We shouldn't be leaving performance on the table when it matters. But at the same time, we need to know that it matters. If we don't know that it matters that we're going to apply this optimization, then, you know, that's a premature optimization and it has real costs. It, it's going to make our maintenance and debugging of this program worse for all the same reasons that it would make it worse today. Like if we go into something and we start applying these like optimizations to it that are above and beyond the kind of structural things that we might be doing to that code otherwise, that's creating a cognitive burden, right? That other people are now going to have to deal with. And you're going to have to deal with in the future. So I don't want to put too many words in Knuth's mouth, certainly.

• 13:15Speaker 00

  But in general, that's sort of the way that this comes together is those two things. This idea that we need to be, you know, be doing things to solve the software crisis, have a more structured approach to programming, care about maintainability and debuggability, composability, abstraction, all that stuff. But also at the same time, we do care about optimization. So how do you dovetail those things? Preempture optimization is the root of all evil is kind of like this phrase he uses. I would, I would say if I had, if I had to imagine, his way of sort of reminding himself, think about all this before you make a decision, right? That would be kind of my take on it, right? And where it comes from. So hopefully, I don't know, that's very long, but it's a very expansive question. So hopefully I've kind of wrangled most of the stuff into there.

• 14:00Ryan Peterman?

  It sounds like this is really an engineering trade off where the software crisis is about the maintainability and the, the complexity for people to actually write and handle their code. And then the trade off between performance and I guess, you know, code that is maintainable. And it sounds like this Knuth quote is, you know, be mindful of where you pay that cost.

• 14:29Speaker 00

  Yes, and it's always important to understand the context in which this was said historically. So I want it to be historical talk. Because who your audience is, is a large determiner of what the root of all evil is, right? If you're talking, if you're talking to a bunch of people who don't think about optimization at all, then you don't really need to say this phrase to them, right? And so if you fast forward to today, when a lot of people don't really take optimization into account at all during their programming, or very, very little, this phrase doesn't make very much sense to someone who does care about performance. It's just like, why would you ever say that? It just kind of seems like it encourages people to not be mindful of the performance of their program.

• 15:14Speaker 00

  But if you rewind to that time and you understand that, well, a lot of the program, you know, they had the opposite problem. A lot of the programs at that time, what, what programming was, was coming up with all these crazy little assembly tricks to like shave a cycle off here or there, right? And if you're talking to an auditorium full of people who are really into that sort of thing, then it really can be the root of all evil, right? It's like, you guys are spending all your time on this, but we have this really big other problem and that is not the right trade off, right? That is, you've, you've swung, you know, like, calling it trade offs is a great way to say it. You've swung the pendulum way too far in one of those directions at some point.

• 15:57Speaker 00

  And, and so, you know, to some degree, while I would, while I would highly dispute the, the description root of all evil today as a good thing to say to somebody or that it leaves a good impression. When we rewind to that time period, I think it makes a lot more sense. Because when you look at what people were actually spending their time doing, where their priorities were, I think that maybe that wasn't so much of an exaggeration. I mean, obviously it's a hyperbole, but it's less of one.

• 16:23Ryan Peterman?

  In your slides, there were so many quotes and snippets of really kind of digging into the true history. Was there anything you found that surprised you when you were doing the research?

• 16:33Speaker 00

  I mean, constantly, absolutely constantly. And I mean, first of all, I'll just say that there's, as a meta point, I'll, I'll give you a specific thing that surprised me. That's kind of funny, that's, that's more about like a technical thing. But first I want to talk about a broader picture, which is, one of the things that really hits home for me whenever I do these is the personal aspects of it. Like, these people were like friends and competitors and like all these sorts of things. And it's all kind of lost to us in a way when we think about computing history in like a sterile way.

• 17:15Speaker 00

  Like, you know, when I read up on what Dykstra, Dykstra was the person who kind of kicked off the structured programming. No, I'll call it a revolution. That's a bit too probably grandiose, but kicked off that train of thought, say. He wrote a thing called Notes on Structured Programming. It was a manuscript or a monograph. I don't, I don't remember what they called it. They're going to get like a typewritten, like it looks like exactly like a typewriter written thing, right? He, when he wrote that, he was like very depressed. Like he was having a lot of trouble in his life at that time. He had, he had gone through this, this situation where they had done what people today now recognize

• 17:59Speaker 00

  as like pioneering work in distributed computing. Like he and these other people at the Einhoven Technological University. I don't, I apologize, I can't say the proper name for it. It's, it's, you know, it's well beyond my pronunciation capabilities. But point being, he had done this really foundational work in distributed computing and published it. And people today now recognize it as that. It is like undisputedly a massive contribution to distributed computing. He did it with a part-time team of people at the university who this wasn't their job. It was a mathematics department they were in. And, you know, as a result of that, basically they sort of just got dissolved. Like the Department of Mathematics, I guess, like he wasn't really specific about it.

• 18:43Speaker 00

  But in his notes, like the Department of Mathematics was just like, they disbanded the team. It was like this, you know, this isn't math or whatever. I don't know, they just had a pessimistic view of it. So, and he was really depressed about that. And he wasn't sure what he should do with his life. And like, you know, what's, what, what's the deal here, right? And he was having this sort of, I don't want to call it an existential crisis because I don't want to, again, put words in his mouth. These are historical figures and, you know, I can own, that's why I try to use quotes to sort of show you like what they said. But like that's, it's just so relatable when you go through the history this way. It's not just some random guy who wrote some math down in a paper. It's like people were really struggling with this. And some of the most important aspects of computer history come out of these amazing human stories. And I found that absolutely fascinating.

• 19:26Speaker 00

  So I'll just put that out there as one of the biggest rewarding things about looking into this history, if you ever do it, is if you can go find the actual writings of the people, like things outside of just their technical papers. It's just fascinating and it's so much more relatable and fascinating because you feel the story. It's not just this abstract computation, computer science thing that happened. So that's one thing.

• 19:55Speaker 00

  Well, the other thing I was gonna say is like, when you read through this stuff, I just go through piles of documents and I'm just reading them and seeing, you know, does this fit into this story, you know, should it be part of what I'm telling? Or is it extraneous? Is it something that, you know, is interesting perhaps, but not actually part of it? And one thing that I found, when the VOD of this lecture or talk goes up, I'm going to include this in the notes because I thought it was so. It didn't make it into the talk. I found a thing, like a thing from Tony Hore. So, so Charles Antony Richard Hore, who again is another massive figure in computer science, right? Hore, Dykstra and Canuth are like, you know, the three amigos and like they all write to each other, right?

• 20:40Speaker 00

  And they're very important people in computer science history. So I found a thing by him that was like a thing he did not decide to pursue. It's like this note where he's talking about, you know, he's talking about this thing that he's thinking of doing. And there's just a handwritten thing on him from later on in his life when he was, I guess categorizing these documents. And he just writes down like, I decided not to pursue this because, you know, I talked about it at this conference that I was at. And Peter Nowar, who's like, again, another famous figure in computer science history. If you've ever heard the, if you've ever looked at like context free grammars and parsing, you've probably heard Bacchus Nowar form. He's the Nowar in Bacchus Nowar form.

• 21:23Speaker 00

  If you've ever heard of the language Algal, he was like a major, you know, figure in, in standardizing that and writing up the standard and all this stuff, right? So anyway, Horace like, yeah, I, I proposed this thing. I went through it, said it and like, and Peter Nowar was like, ah, making multi-pass compilers is easy. I just wrote a nine pass one. So I was, you know, I decided not to pursue this, right? So what's the thing? I read through the thing and like, maybe I'm just over reading it with the benefit of hindsight, but he pretty much describes static single assignment form like, like SSA, which is a very standard compiler technique developed in the eighties, but this note is from the sixties.

• 22:06Speaker 00

  So it's like, in my head, I'm like, did Peter Nowar accidentally set back like computer, like, like compiler science by 20 years by like telling Tony Hore, or 15, let's say, by telling Tony Hore, like, this is not important, when actually it was, would have been like very, very important. So you find stuff like that all the time where you're like, whoa, what is this? I saw another one too, just one more I'll mention. I can't remember now, I'm sorry. Again, your brain kind of turns to mush when you try to dump this many documents into it for a talk, but there was a pretty interesting thing I thought where Margaret Hamilton,

• 22:47Speaker 00

  so the, the person who, you know, managed the Apollo, she was, she was, you know, one of the core programmers originally on the project and then, then was like the man, like the manager of the whole OS, like the whole like real time software that ran the Apollo 11. Well, all the Apollo flight computer stuff, right? Which, I mean, most people today, Harold, is like a very, very significant, like real time systems, like achievement in that era, right? Like everyone pretty much agrees with that. She had to write like a defense. I think it was in the ACE, the Communications of the ACM, maybe it was in the animation, they say, I can't remember the venue.

• 23:27Speaker 00

  Because like, it had been pointed out as like a failure because people didn't understand the context that like, the reason that the Apollo computer had to like trip those alarms was because it had been used improperly, like it was used in a configuration that it wasn't supposed to. And it actually, rather than crashing, went into backup modes and successfully landed, like it, it worked, like it, it actually was a triumph of like fault tolerant engineering. And she had to write like a defense of this because people at the time were saying like, oh, that they screwed up. Like this is an example of, of why you wouldn't want to do engineering this way or like, you know, don't write the code this way. And so, again, stuff like that always makes me think like, the more things change, the more they say the same. But it's exactly what would happen on Twitter now, right?

• 24:11Speaker 00

  Like, like some very successful thing that people did write, they would just get like slandered and say that it was done wrong and there would be like a flame war and then they'd have to come out and have a thing, you know? So anyway, those are just some examples, but there's so much great stuff in there. I highly recommend anyone who likes this kind of thing. You won't be disappointed if you go, if you go dumpster diving, as I call it.

• 24:31Ryan Peterman?

  When you said that you saw the human stories and that Dykstra was depressed at that time, what did you see that made you realize that?

• 24:42Speaker 00

  One of the nice things about Dykstra is that he's kind of a pretty straight shooter. And so I didn't actually have to do any interpretation. He literally talks about like being depressed and like he has like a, there's a paper, not a paper, like a thing that he did, that's like a retrospective on, that's like handwritten by him later on in life. And he literally says like, in this period, I was very depressed because of these reasons. Now, he could be wrong about the source of his depression, right? So I don't want to claim that, that just because he said it, it's definitely true or something like that.

• 25:20Speaker 00

  But one of the amazing things, I think, one of the, the really cool things about academics, especially that era, is they just produced a voluminous amount of material. And so we don't have to guess that much. In the private sector, I think it'd be a lot harder. Like if you wanted to know what people at, you know, maybe, you know, Lockheed or something were thinking or doing at that time, I imagine it's much more difficult just because, you know, things are classified or they're not academics, so they're not necessarily going to write them all up, you know? So a lot of the internal stuff, when you're written up, academics don't really have that. They don't have an incentive or a prohibition on writing up everything they do. So they don't just have to have like a public facing statement of what they did or a public

• 26:05Speaker 00

  facing paper of what they did. And then, oh, there's this extra secret stuff. The academics don't have that restriction. They're sort of, they benefit from being as forthcoming as possible about what they've accomplished usually.

• 26:17Ryan Peterman?

  I saw in the, in the slides that you looked at go to considered harmful, that famous. Do you have any sense of the, the personal or people side of that?

• 26:27Speaker 00

  So you're talking about the original like Dykstra letter to the editor that's like, Edsger Dijkstra, go to statement considered harmful, this, okay. The complete story is actually relatively simple. He, according to him, he was at a, at a conference in Tennessee where he was talking to Brian Randell, who's another computer science guy. He, he doesn't quite get the same level of name recognition as a Canuth or something like that, but he has a lot of papers at that time, like you can go find him. He's not an obscure, he's not an obscure figure, I wouldn't say to anyone who reads the history. You'll see him come up.

• 27:10Speaker 00

  He's talking to Brian Randell and some other people outside. This is exactly like the same thing that happens at modern conferences. There's like the talks and then there's all the stuff that goes down at the bar, right? It's, it's very much that. They're outside, they're talking. And according to him, he's basically giving the same sort of example of a problem with the go-to that he gives in the paper, right? This idea of enumeration, we can talk about that later if, but it's sort of a side note. He's sort of saying like, this is kind of a problem with, with go-to and one of the reasons that maybe it's not such a good idea. And the people who were listening to him were like, you know, Brian and all the others

• 27:53Speaker 00

  were like, you should pub, like you should publish that. That would be helpful because, you know, there's these arguments going on about whether go-to is good or bad. Like that was kind of happening at the time already. Since, since around 1959, even, I think there'd kind of been a little bit of that had been kind of growing. This idea that maybe go-to statements weren't, were the best idea as a way to structure your programs. And so he does, he goes back to Einhoven and he writes up this same thing he was saying, basically, and he sends it to, to Communications of the ACM for publication. Nicholas Worth, who is like the creator of Pascal, right? A very prominent figure, also was very heavily involved in Algal and all this sort of stuff,

• 28:36Speaker 00

  like, you know, language designer guy. He is the editor who is in charge of like getting this, getting this thing published, I guess. I'm not exactly sure how things work at the Communications of the ACM. He doesn't want to wait to have it published. He doesn't want to take the time to like go through the referee process or whatever. I don't know, like at that time, what the requirements were for publishing a article in Communications of the ACM, but I'm sure that it involved a lot of procedure. It wasn't just like, oh, hey, I'm, you know, I'm the creator of Pascal. I mean, although at that time, he wouldn't have been the creator of Pascal yet because Pascal comes later, I think. But, but either way, he's like, hey, I'm this important guy.

• 29:18Speaker 00

  I'm just going to put this whatever I want, because they said that's not how it worked. And so he decides that in order to get it published more quickly, he's just going to publish it as a letter to the editor. Because then there's no, like any, you know, anything goes there as long as the editors are fine with the content, I assume. Like it doesn't have to be refereed, doesn't have to have any kind of review, doesn't have to, et cetera, et cetera. So he turns it into a letter to the editor and he takes the name of the thing that Dykstra sent, which was a case against the go-to statement. That was what Dykstra wrote at the top of the thing as the title. He changes it for a letter to the editor to Edsger Dijkstra, colon, go-to statement considered harm, or considered harmful, right?

• 30:01Speaker 00

  So it wasn't, it wasn't even Dykstra's plan to like have it maybe be that confrontational, but that's what happens. Now, this is not well received, to say the least. The, according to Canuth, again, I'm just trying to say who said what here. According to Canuth, Dykstra said he got letters, like, like angry, like threatening letters, like in much the same way you would today, right? Like got, got people who were very abusive and, you know, telling him off. And I'm sure part of that is because by all accounts, Dykstra himself was also somebody who liked to push people's buttons.

• 30:46Speaker 00

  That's well acknowledged. I think Alan Kay is probably the best source for this. He talks about this, he liked Dykstra and they apparently got along well, but you know, he, he's often said that Dykstra was someone who kind of leaned into the being sort of brash about stating things about programming and, and kind of relished that position. So it probably didn't help that he was already sort of known a little bit in that way. But in general, that's, that's how it went down and.

• 31:16Speaker 00

  Like I said, that wasn't the initial foray against go-to statements by any stretch of the imagination, but it just kind of, maybe you call it the straw that broke the camel's back. It was like a, it was like a flashpoint, might be the way to say it. And so, and then the title, of course, that Nicholas Worth picked is quite the doozy and kind of clickbaited everyone, you know, as I call it into that. So.

• 31:43Ryan Peterman?

  It's funny, yeah, because I mean, the social media, it's similar patterns, right? People have very explosive first lines and then the comments are full of all this hate and stuff. In their case, I'm guessing this is all papers. So you submit a letter to that, it's a physical thing and people are reading, maybe a newspaper. So I don't know.

• 32:05Speaker 00

  Yeah, it's like a periodical, like it comes, you know, as a bound. I mean, I guess there's, all the things around us here are hard bound, but it's like, usually was soft, it was a, you know, maybe a perfect binding, kind of bound thing that has, you open it up, table of contents and letters to the editor, right? And yeah, like, you can go find, there are, there are a few, if you go look at people's personal papers, which like, like I said, if I did this full time, I'm sure I could find out way more stuff than I did, right? But some people have gone and looked at the personal papers and occasionally some of them have been able to digitize some of those and put them online that we can

• 32:43Speaker 00

  look at. And you can find, for example, online right now, if you search for it, there is a, there's a back and forth, letters, personal letters between Dykstra and a guy who was kind of into functional programming and promoting that. And like, you can look at the correspondence and it's kind of, it's a little, it's a little flame worry, like, like it really, like, that's what they did. They, they didn't have the ability to do like pithy Twitter replies. So it was just on paper. And yeah, and Canuth did something similar. It wasn't really a flame because he had a, at least when reading, it comes through as a tremendous amount of respect for the authors of the book Structured

• 33:26Speaker 00

  Programming, which is Dykstra, Hor and Dahl. He, you know, one of the things that he published was a series of open letters to them, reviewing the book and talking about the things that he didn't find compelling in it, right? Like, very respectful. So it wasn't, that one wasn't a flame war, but that's what they, yeah, that's what they had to do because they didn't have social media. So, you know, Edgar Dykstra's, he wrote like all these things called EWD with a number. It was, it was his initials, basically. And then a number, and that's like this like serialized list of all the things he wrote. And he like labels them this way. Like, it's not like some historian character after the fact. It's like, oh, I'm doing,

• 34:09Speaker 00

  he's like, I'm doing EWD 937 now or whatever, right? It's hilarious. But anyway, several of those have been put online that are not necessarily technical. Like, for example, his trip reports. You can go read those and you can read about like, oh, like I went and, I went and stayed at like such and such's house and like, we went to dinner or whatever, right? So you can find like some nice personal anecdotes in even the publicly available stuff. But in terms of like a real, like, heart-to-heart conversation, I didn't have access to anything like that, that wouldn't have just been in a paper more or less normally. So, you know, that's, it's a shame. One of the problems with this stuff is so interesting,

• 34:52Speaker 00

  but it's not a job to do. Like, it's like, if somehow it was a job, I probably would almost take that job. Like, like, being the person who crawls through and tries to like redocument this whole thing, but you know, computer historian, no one's, no one's hiring. Like, that's not, no one's interested in paying for that. So.

• 35:09Ryan Peterman?

  You have this other talk and the title was just so catching. It was the big oops anatomy of a 35-year mistake.

• 35:18Speaker 00

  Yes.

• 35:18Ryan Peterman?

  What is that 35-year mistake?

• 35:21Speaker 00

  So this was a, this was a kind of funny thing that happened to me that I then did a historical lecture on. I, I had worked on systems in the past that were basically like editors, like, you have to, like 3D graphics editors, like get the multi-select things and move them around. And there's a bunch of like, architecture things you have to learn to be able to write that kind of code. And, and certain kinds of problems you have to solve, like a very basic one that I would point out, that I, hopefully most people can relate to, would be if I have a bunch of things on the screen, some of which have a color. So maybe, maybe this is a drawing program and I've got some text and I've got

• 36:03Speaker 00

  some shapes and I've got some strokes, you know, some, some hand drawn stuff, whatever. And I want to be able to, to select a bunch of them. And I want the user interface to present to me which things I could edit on these shapes. So I want to be able to edit the color and I want it to apply to all of them, right? This is just a basic architectural problem that you have to solve. If you're going to write one of these programs and you want it to be any good. Now, oftentimes, you will see programs where the person didn't solve this problem. And you can't do that. It's very frustrating, right? So, a good program, someone has solved that architectural problem in some way. And I was watching, I just happened to be watching, because sometimes I do, like, I like to watch old videos of

• 36:44Speaker 00

  computer science pioneer stuff. And I was watching a demo of Sketchpad. I'd seen it before, but I was watching a demo of Ivan Sutherland's Sketchpad. And Sketchpad, for those who don't know, is a groundbreaking computer graphics program. It's done with a light pen, and it was done in the early 60s. And you could draw things like you do in a modern CAD program. And you could do stuff that even a lot of modern CAD programs don't really offer or only offered recently. I think in the talk, I said in 2007 was the first time that like AutoCAD got some of these features. You could do stuff like, say, these two shapes, like this line has to be perpendicular to that line, constrain it. And then you could just do whatever you wanted with the drawing and it would like re-solve

• 37:27Speaker 00

  for like making that happen. And these are very rare in programs, even to this day. Like, that's just not a common operation you would see in a drawing package today. So, I was looking at this and I was like, how the heck did he solve this architectural problem? Like, how did he do this stuff in the early 1960s? Like, in assembly language, with no editing tools, no debugging tools? I mean, this would have been like, you know, either punch cards or one step away from punch cards. Like, it would have been hard to develop this stuff. And when I went back and I went through his thesis and I looked

• 38:10Speaker 00

  at the code, it's actually documented how he did it. And I realized like, oh crap, this is actually like an entity component system, basically. Like, what we would now consider sort of state-of-the-art of real-time architecture for doing these kinds of operations, meaning cross-cutting, like, operating on this kind of property across many things at once that are all kind of different. And this was something that object-oriented programming, I'm editorializing now, a bunch of people will complain that I'm saying this, but this is just my opinion. Object-oriented programming and the pedagogy around it really struggled with that problem for a long time because they were teaching, despite people

• 38:55Speaker 00

  will now claim otherwise, I documented very completely in the talk. So, I, I don't think they have any leg to stand on. But the pedagogy at the time was all about build a domain model. The domain model looks like your hierarchy, basically. Like, I've got employees and employees, I've got contractors, and I've got derived, like, is this a full-time employee or a part-time employee? Like, that was how they suggested these things should be implemented, right? And that kind of architecture doesn't work for these kinds of operations that I'm talking about. Entity component systems do. And generally, that kind of sort of rotated architecture, I might call it, does work for these. And so what I call the 35-year mistake is the

• 39:38Speaker 00

  fact that, I, very ironically, in my opinion, Sketchpad sort of showed in its architecture how you could solve this problem and how you could solve it in arguably an object-oriented way. I think if you squint at it, a person who does object-oriented programming could easily make an argument that you could do an object-oriented version of this. You don't have to violate any particular principles to do it. It's just not the hierarchy domain model way that people were conceptualizing it through a lot of the, you know, 80s and 90s, let's say. And even unfortunately, to this day, but, you know, I think a lot of object-oriented programmers don't do that anymore. Ironically, that was in Sketchpad.

• 40:20Speaker 00

  The DNA was there and we could have had it right away because Ivan Sutherland figured it out. But weirdly enough, the whole idea of this sort of inheritance hierarchy with lots of encapsulation around it.

• 40:34Speaker 00

  Grew in part out of Sketchpad because like Alan Kay looked at it and said like, "Oh, the interesting part of this is the fact that you could take a shape and not know what it was and just have this like draw circle call on it or something." That was the part they took away from it, which is much less interesting in my opinion, and not that architecturally useful in most cases. So in kind of this amusing way, Sketchpad contained, if you know, I'm editorializing, it contained what I consider to be a good architectural idea, but the takeaway from it was a bad architectural idea is how I would say it. And saying bad architectural idea is a little bit strong because as I say at the talk, I think there are times when you do want to think, like a very high level, you might

• 41:17Speaker 00

  want to think about things in kind of the way that, you know, small talk thinks about them. So I don't want to say bad idea in the absolute sense, but bad in the way it ended up being, you know, applied, let's say.

• 41:29Ryan Peterman?

  Why is that class hierarchy not work for this problem?

• 41:33Speaker 00

  It's pretty easy to understand, actually. So when you think through an inheritance hierarchy that's based on sort of what you see in the real world or what you're trying to model. So you say like, the typical example, and it's brought up tons of times by people who wrote the OOP literature, like Bjarne Strustrip, like the small talk people, is the idea is, keeping it with a Sketchpad example, I'm going to have a shape. From the shape, I'm going to derive like triangle or circle. And the encapsulation is around that thing. So a circle has a radius, but shapes don't have radiuses. Shapes don't necessarily know what that is, right? So it tends to get deferred down.

• 42:16Speaker 00

  And the encapsulation boundary is drawn around that sort of derived class. That's like the most derived version of whatever this thing is, right? The problem is that creates a lot of headaches when now something that wants to work across a lot of shapes needs to actually do something intelligent with modifying, say, the scale of something or wanting to change its color because it doesn't really have a way to know what's going on in these highly encapsulated things.

• 42:49Speaker 00

  So typically what you end up having to do in those situations is you then have to take this inheritance hierarchy and kind of in a sense, throw away the benefits of it. You end up having to make at the top, all of these accessor iterator things that allow you to probe into that like lower class and say, "Tell me all of the colors that you might be using and a name for them," right? Or, and so in a sense, what you end up having to do when you want the architecture to work is you have to rebuild the Sketchpad version on top of the inheritance hierarchy version, which doesn't really do anything for you, right? And it's not to say that there isn't some benefit that you can derive from having, derive is kind of a pun here, I guess, that you can derive from having this idea of saying,

• 43:34Speaker 00

  "This thing is like this other thing. Please give me some of the implementation of it." That's not necessarily a bad thing in practice. The problem is when you teach people that this is how it's going to work, like you just make this hierarchy and then your architecture is just supposed to work. It really ill prepares them for the reality that like, that's actually not going to solve most of, most of the problems that you have are not going to be solved that way. And again, it just comes from this fact that typically when we're working with computer programs, most of the time, what we need to do is create cross cutting operations that work with a lot of the data that's inside these derived classes. That's actually the thing that we want to do. And it's really cumbersome and typically a bad fit for the problem space to put that

• 44:17Speaker 00

  into some virtual functions that are sitting at the top of this very sort of tall hierarchy. It's usually much better to do, again, even if you're doing object oriented program, you don't have to stop doing object oriented program, you just have to think about it differently. You have to think about what the objects are differently, where you're drawing encapsulation boundaries. It just helps to think about it differently and say, "Actually, the more important things to model are what are the things that stuff is built out of? How do I build a circle out of objects, right? What are those things? A radius property, a center property. Those maybe should be the objects that I'm thinking of as more first class citizens." And this inheritance hierarchy stuff about what the objects are that I see on the screen, like triangle, circle, that's a distraction.

• 45:02Speaker 00

  That's not the proper way to model the problem. And again, I don't think I would be pissing off any object oriented programmers by saying that today because I think a lot of them use architectures that do think about objects in that sort of rotated sense and not in the domain model hierarchy sense. Not that there aren't still people who are doing that, but you know, it's not the only way to go nowadays, not the only way it's promulgated.

• 45:28Ryan Peterman?

  Because I guess my immediate thought would have been, in that case, the base class has the notion of a color or some shared thing and we just operate. If you're a shape, then we assume you have a color and we do. But you're saying refactoring and putting up in there is suboptimal and there's, you want all these subclasses to have some handle to some radius property or something like that, is that?

• 45:55Speaker 00

  What you just described is what tends to happen in systems that are architected this way. You end up with your derived class not really being very much of anything because in order to edit stuff, you had to migrate it all up to the top. And there was actually terms for this in the old days, I don't know if they're still used, like fatty base classes, right? Or things like that. It's like, it just ends up, your base class has like every type of property that any derived class might ever have. Like you have like, get primary color, get secondary color, get tertiary color, right? Because it's like something needed three different colors or whatever. And most things don't use those colors and all these are the sorts of things, right? And so there's certainly nothing wrong with running a program that way. It's just, again, why did you bother with the hierarchy? Like you're not getting any actual encapsulation benefits from it because you just forced

• 46:40Speaker 00

  everything in the base class anyway. So why didn't you just make that be your thing? Just make one class called shape and be done, right? So in terms of the ECS style of implementation, I don't necessarily want to advocate for it or not because I don't actually personally use ECSs or anything like that, but they are very common now. The idea there is to just turn it on its side and say, "Look, typically what we want to do is do things like, okay, there's stuff that has like physics on it. Like, you know, I'm doing this thing and I want to have like physical simulation of entities." Well, the physics system is the thing that probably knows how that should be stored. So rather than having that be data in my derived class that is encapsulated around

• 47:22Speaker 00

  the derived class, instead, why don't I just have the physics system knows what physics is? And then if you want to make something that participates in physics, you just have a handle for whatever this thing is, like a handle for my shape. That handle is valid in all the systems. So I can go look up the physics properties of the shape and gather it when I want to actually do something with it. But generally speaking, the encapsulation stays around the system. So physics properties are defined by the physics system because hey, that's who knows how physics should work. And I just ask about my physics when I need to do something with it. And this kind of gets you out of that problem. And then now you can just compose things. "Oh, I want to make something that has several physics elements in it."

• 48:06Speaker 00

  Fine, I can just have multiple physics handles if that's what I need or something like this. I can very flexibly like merge these things together. I want something to not have physics. I just don't ever insert the, you know, I don't ever ask the physics system to have a valid, you know, piece of data for this handle in it and it won't simulate any physics for it, right? Whether that's the world's best architecture or not, I'm not going to argue for it or against it. It's just, it's a lot better than domain model hierarchies, that I will say, right?

• 48:34Ryan Peterman?

  OpenAI, Anthropic, Cursor and Vercel all use this product to make their lives better. And the problem it solves is when you're building SaaS or an AI product and you want to sell to other companies, there's all these requirements you need to meet. There's SSO, there's SCIM, there's RBAC, there's audit logs. These are all things that take time to integrate, but aren't the main focus of your app. WorkOS is an API layer that lets you meet all of these requirements in just a few lines of code. So let's say you have a new SaaS product and you want to sell to other companies. WorkOS will solve all of these critical feature gaps for you. You can check them out at workos.com to learn more and get started.

• 49:16Ryan Peterman?

  And I appreciate them for supporting my work and sponsoring this podcast. Jira by Atlassian isn't just for tracking work anymore. Now you can pick your favorite AI agent to assign tasks to, and they'll get access to the rich context that's already in Jira. When the agent is done, it surfaces a pull request. That way you can get more done with your favorite agents all in one place. Learn more at jira.dev, that's J-I-R-A dot D-E-V. Appreciate them for sponsoring the podcast and back to the show. Earlier, we talked about the trade off between, I guess, performance code and maintainable code. And I saw you had this one video that said, you know, in quotes, "Clean code, horrible

• 50:01Ryan Peterman?

  performance." I feel like this is on the same topic. What's your, your take there on, you know, because you also put clean code in quotes. What does that mean?

• 50:12Speaker 00

  So it's a, it's a fairly subtle topic, right? And that's why the quotes are there because the first thing that I, you have to point out is that clean code is clean code, object oriented programming, a lot of these phrases, they mean different things to different people. And so if you're going to say that you're, you know, this code is clean, it's pretty hard to find two programmers who will agree on exactly whether it is or not, right? Like, like they all have like different ideas about what is clean and what is not, right? So the reason I put that in quotes is I was talking about a very specific thing, which is like literally like the book Clean Code, like the thing where there's like, here are the things that we would recommend that you do, right?

• 50:54Speaker 00

  So in that particular case, what I was talking about is there's a certain set of ideas that is advocated in Clean Code as like, these are the things you should do. And they are like, you should have, everything should be kind of, I guess I would say, dynamic dispatch, or at least you shouldn't, code shouldn't know the types it's operating on, right? Saying it's dynamic dispatch is maybe, it depends on the language, whether that's going to be true or not. But when I write code, I don't know what types I have. It's just, I don't know, like some base classes is what I'm operating on, and they could be anything, right? That's thing one. Thing two is functions should be very small. In fact, the, the numbers are kind of weird. They're like four lines or five lines.

• 51:36Speaker 00

  Like, when you go look at the actual things that are claimed in some of this literature, it's, it's really strange. You're just like, gosh, that's very small, right? And I go through some of those things, and I say, look, if we were to actually follow these, we get into a really bad state. Because a lot of the languages that people are going to be using to implement this stuff, like C++ for example, which even would be a fairly good case in a lot of, at least it's a compiled language and so on. In a lot of cases, these things are kind of a recipe for disaster. If you're, if you're handing out types where you can't know the type at compile time, or you can't know the type, or it's, or it's, you know, going to be in a different module. So, you know, unless you have like a really aggressive link time code generation, it's

• 52:20Speaker 00

  not going to be likely that you could figure this out. And you're saying, oh, the function should be very small, and then of course, they're going to be virtual, because they're on these sort of types that you don't know what they are. This creates a really toxic combination for the compiler. Normally, you can have, I mean, you can have small functions, if you want them only four lines long, if the compiler can know that it can just inline that, right? If it can see clearly what you're doing, and it can just merge those things in, we don't have a problem. If they're behind a virtual function, so it doesn't know, like, it can't guarantee that it is that type, it can't do that. And so when you talk about all this stuff, what you're, what you're giving up, people, I think also, because the video, I mean, the video was a very short video, it's part

• 53:04Speaker 00

  of like a series. I think people sometimes get the wrong idea that I think that virtual function calls cost a lot. They do cost something, but, you know, depending on how you want to look at it, they actually don't cost that much. Because, you know, it depends on whether it's predicted correctly or not, but in general, like, the cost of virtual function is less than you would think, a lot of times. The actual reason that it slows things down is because the compiler cannot merge the code together, right? It can't inline stuff and remove and reduce the waste, like, that's the actual problem. So if you just want to call a virtual function, yeah, if it was a really, really hardcore optimization scenario, then you don't want anything probably calling a virtual function in general, because there is some cost to that. Just calling a function has cost.

• 53:48Speaker 00

  But that wasn't the really bad part of it, right? So it's that. Plus, there's an additional thing, which is that it can't unroll and go wide on things. If the compiler can see everything you're doing, it can use SIMD, it can like widen loops to operate on multiple things at once, it can unroll those loops, there's all these things it can do. If it's got these virtual functions, it's just like, that's it. I don't know, I can't make decisions about that, I have no idea what this thing on the other end is doing, right? So that was kind of my point on that. And again, I, it's not meant to be assailing the idea that your code should be easy to read, or easy to maintain, it's just more like, these guidelines seem really bad. And also, I don't think we need them for code to be maintainable. I, I don't have trouble maintaining functions that are 30 lines long, or 50 lines long.

• 54:33Speaker 00

  I don't find that five is a magic number or something, or that has to be really short. And also, I find that it's usually pretty easy to write code where I know what the types are, that those can be determined at compile time. Like, I don't think that results in code that's particularly hard to read.

• 54:48Ryan Peterman?

  When I was in college, very, and very early, I remember people would recommend that book, so, oh, you should read Clean Code. So my understanding that you wouldn't recommend people who are software engineers to read that.

• 55:01Speaker 00

  I guess the way I would categorize it's a mixed bag. So I wouldn't, I wouldn't actually say that I disagree with all, all of the things in the book, though. I mean, some of the things are things that I definitely do myself, like, giving variables like easy to understand names, is something that I think would be, and that's in that book, is good advice, right? And so, kind of more, I wouldn't, I wouldn't necessarily say do or don't read a book. What I would say is like, I think there's some things in this book that don't, that aren't addressed properly. Like, I don't think you want to tell people these rules of thumb, and not tell them about these other problems, that was exactly what I was pointing out there.

• 55:41Speaker 00

  And so usually, it's more that, it's like, I find that oftentimes, there is this tendency, I guess I'll say, to pretend that we don't have to talk about performance, and we can just say like, in fact, premature optimization is the root of all evil, bringing it back to there. I feel like there's this temptation, and very prevalent practice, of sort of taking that idea to mean, we don't have to talk about performance when we're teaching people things at all. Or like, when you write the book Clean Code, you don't have to talk about performance at all, you can just include a thing that's basically like, hey, you know, there's performance issues, but most of the time, it's not a problem, or something like that.

• 56:26Speaker 00

  Which is, you know, or if you look at refactoring, the book Refactoring, very popular, it literally says exactly that in like the opening thing. It's like, you know, there, there might be performance concerns, but, you know, they're usually okay, or something, right? And it's like, that is not true. I just think that's fundamentally not true. I think these books should include detailed discussions of the actual performance problems that you are very likely to hit if you take some of their advice. Because it's not that it means you can't do those things, but, I'll quote you back to you, it's about trade-offs. There is always a trade-off that you're making, and sometimes, if you understand the trade-off, you will make it.

• 57:08Speaker 00

  You'll say, hmm, I know this is going to cost, this may be a serious performance problem for us, but I think that's okay. Like, I think it's not, like, our performance won't suffer to the point where it's a problem for the product. I understand what that is. There's a huge difference between that, and just going like, eh, we don't worry about performance till the end, right? Like, there's, it's completely different, right? Those are two different engineering approaches. And so what I try to do is encourage people to put that analysis back in. Understand the performance trade-offs you're making. Understand how much it might cost in the future to like, make these fixes. And I, I don't really do any AI stuff, but my sort of feeling is, more so now than ever,

• 57:53Speaker 00

  I feel like that's got to be pretty important. Because you're instructing these AIs to do what they're going to do. And I feel like it would be a bad idea to not know about performance trade-offs, because, especially if an AI is going to be doing your bidding and structuring the code the way you want to structure it, or doing whatever, it seems like a very straightforward part of the process to include performance stuff in those instructions, right? Like, that would be a natural thing that we would want to understand and do, right? So I feel like, really, I've been just trying to get that more into the conversation, and I feel like it's, it's as relevant now as it ever was, and arguably maybe more so, I don't know, but I could be wrong about that.

• 58:33Ryan Peterman?

  What are some things that are, in your mind, that are those high-value performance things to consider, that don't cost a whole lot to think about?

• 58:43Speaker 00

  So to me, awareness is the number one thing, right? Understanding roughly the performance characteristics of the hardware that you're working on, which includes things like, you know, if there's a network, like, what does that look like, and so on. It's really about awareness more than anything else. Because a little bit of awareness can go a very long way. If you understand the basic concept that like, you know, a, that network latency versus network throughput are different things, you can make upfront decisions about structuring code, such that you batch things properly, it's so on and so forth. If you don't understand those things, you could get very far down a project, only to

• 59:26Speaker 00

  realize that everything you designed and all the way it works is all very serial, and really just cannot be accelerated over a network at all. Like, it's never going to run reasonably, or something like this, right? And so I think the, the lowest hanging fruit is just to get some education in performance. Like, get some education in how to think about the way a machine works, and what makes it fast or slow, what it struggles with and what it doesn't. And just to keep that in the back of your head. Because at the end of the day, if you have that knowledge, I think you're very unlikely to make the kinds of architectural decisions that will be hard to undo later, right?

• 1:00:08Speaker 00

  And so that's really, that's really the majority of it, I think. And that is by far the highest impact, lowest cost thing you can do, is just do that training once. Because once you have it, it's with you forever. Once you understand how to think about performance, it can always be there, and at any time, you can sort of have that alarm bell of like, hmm, this, I don't see, you know, you're always kind of looking like, I don't see the path towards this running well. And then, so that's your key to stop and go like, ah, maybe we need to rethink like, how we were making some of these decisions. As long as you can see that path, you're pretty, you can, you can delay most, like, optimization work, as long as you can see the path, like, here is how we will optimize

• 1:00:53Speaker 00

  this, you're in pretty good shape. Because if you're making those trade-offs correctly, you're not going to paint yourself into a corner where there's nothing that you can do, other than scrap and rewrite, right?

• 1:01:06Ryan Peterman?

  You had this one video titled, it said, you know, where does bad code come from? And so, you know, I guess, yeah, my question to you is, what's the answer to that? Where, where does bad code come from?

• 1:01:17Speaker 00

  Yeah, that's a good question. My feeling on where bad code comes from is that, for everything that I've seen on projects in the past, the bad code all seems to come from roughly the same source. And that is not dealing with the actual thing that's happening, right? There's a lot of ideas in computer science about sort of doing upfront design and making a lot of decisions without ever really implementing anything, right?

• 1:01:54Speaker 00

  And I have found that universally that leads to the worst kind of code. And the reason for that is, you know, I hate to be pessimistic and I hate to be pessimistic about my own ability, but I'll just say for me personally, I am not able to correctly hold all of the details for most complex software in my head at once. There's just a lot of stuff going on at like when, when the actual, you know, instructions hit the CPU, there's a lot going on down there that's very hard to keep all in your head. And so when you approach an upfront design, typically unless the problem is very, is

• 1:02:39Speaker 00

  like stupidly simple, right? When you approach something with upfront design and you think you're going to get all of this architecture worked out ahead of time, you forget some important things. You make decisions without realizing, oh, wait, there's this thing that has to happen there that means that this isn't really the right way for these pieces of code to interoperate. And so then you, you know, you end up seeing these hilarious APIs or something where it's like, you're just like, how did this end up being the way that you do this, right? Like I have to create all these objects and I have to connect them together with these things and I have to create this weird like filter graph thing and then I have to call compile on it, but only like you end up with these like insane things and you're like,

• 1:03:23Speaker 00

  all I wanted to do was call this one thing that said like low pass filter this buffer. It could have been one function call, right? It's like, how did you get there? And the answer is because you weren't actually like dealing with the actual problem and looking at what the actual code looks like, uh, when you want to actually, uh, when you actually want to solve the problem in practice. And so my idea where bad code comes from is usually just, it's that it's this failure to engage with the actual reality of the situation. Maybe there are people out there, I certainly have never met any, but maybe there are people out there whose brains are so expansive that they can actually hold all that in there and they can therefore just do it upfront.

• 1:04:04Speaker 00

  But other than for various sort of constrained problem spaces, I've never seen that work. And, and you know, there, like I said, there are sometimes when that's the case with to transrupt, like if you're just doing like, I'm trying to do this particular mathematical operation, that might be a case where you can work it all out on paper because it's very specific like what it is. It takes these N inputs produces this output and I can work, right? But when you're talking about architecture, like, hey, it's a web browser, right? Some complicated thing. The details are all that matter to me, right? It's like, it's where all the complexity lies. And if you try to do the design without reckoning with them, it just leads to bad results in my experience.

• 1:04:41Ryan Peterman?

  So how do you avoid writing bad code in that case?

• 1:04:45Speaker 00

  I, I try to start with the actual solutions to the problems and work up from there. So, um, one way to say it would be instead of top-down programming, bottom-up programming, right? As much as possible and top-down programming isn't really the same as designing upfront, but you know, I'm just saying like, as a contrast there. Try to start with the things that you know, you need, like, it's like, okay, if I'm, if I'm building this thing and it's got to have a rasterizer, all right, I'm going to start making a rasterizer. I'm going to see what kinds of stuff that does and how I would like, what's the most straightforward way to call this and use it? Okay, let's make an API out of that, right? Like build it out of steps where the abstraction comes from actual working code that gets

• 1:05:27Speaker 00

  abstracted, rather than pushing the abstraction down from the top where I say, this is how I will abstract the rasterizer. And now I go write the rasterizer and the thing that uses the rasterizer only to find that that is not a very good way to use a rasterizer, right? Like, so to me, starting with that and working upwards is the best way to ensure that you will get an architecture, uh, that is at least good for one thing. Now, if you want it to be good for multiple things, you probably need a couple different, like I want a few different varied things that would use this rasterizer that I will kind of work on together and I'll make abstraction decisions that work across all three of them. That's a good way to make a more reusable API that works for, you know, a lot of things.

• 1:06:12Speaker 00

  But I never want to just sit down and say, hmm, how will I design an API for lots of people to use a rasterizer without actually writing any of it? It's like, no, no, no, like, and I definitely can't do it. And I, I question, I would question someone who says that they can, unless, one caveat, they've written a lot of them before, right? If you've written times before, then you've sort of done the thing that I'm talking about already and you might remember, oh, it's this way, right?

• 1:06:36Ryan Peterman?

  I think there's a lot of common advice, especially at these, these larger companies where there's this processes to, you know, write a design doc and kind of, uh, you know, put together everything in advance, present it before you write any code. And it sounds like you're saying that that's a terrible idea.

• 1:06:57Speaker 00

  I think that's an absolutely terrible idea. Now, that doesn't mean that I wouldn't be okay with that process with like a slight modification, right? If what you're doing during that time is writing test code in the way that I'm saying, so we're going to write a little experimental rasterizer, we're going to do those things, we're going to start, and then our document that we're producing is here is what we have determined is a good API based on these experiments. I have no problem with that. That's really, that's following my procedure pretty much to a T. And, you know, I don't tend to produce documentation because I work, you know, in, on smaller teams. I don't have to have a thousand person org know exactly what it, what I'm doing there or whatever.

• 1:07:39Speaker 00

  So I wouldn't produce upfront documentation normally in a, in a case like that, but if you, if you wanted to, that seems totally reasonable, right? And that like is communicating our research into how this should be structured, uh, out to a wider audience, but we still did the due diligence of determining that it really does work in practice. It's not just our guess about what it will be.

• 1:07:58Ryan Peterman?

  I see. Okay. So you're saying kind of this, uh, pair design, pair prototype, kind of, you know, like you just build as you go, but it's just so you get a sense of the reality of the design.

• 1:08:10Speaker 00

  Yeah. And I think I would be very comfortable with the team that wanted to work that way. They're like, look, we want to produce, we want to document this thing before we start developing it. That's just how we feel more comfortable, you know, with how we're going to do it. Maybe it's a very large org, maybe we have some very good reasons for that. Then I would say totally fine. It's just, I want, I want to hear, you know, if I'm going to be comfortable, I want to hear that that process is not, we're just typing on paper. We are actually testing these API, uh, design decisions and they come from looking at actual usage code that we have made and that we have implemented at least, uh, experimental versions of, so that we know they really do account for all the details that we're likely to encounter in practice. That's what I want to hear, right?

• 1:08:55Speaker 00

  I don't want to hear like, ah, we thought it through. Like, I'm like, did you? Did you think it all the way through? Cause I've seen a lot of times where people thought they did and they didn't, including myself, like that is not, that is not a, I always think it all the way. So it's like, no, this comes from a personal place of, I forgot the thing, like I forgot this important thing and I made a really stupid design, right?

• 1:09:13Ryan Peterman?

  There's another video you had is titled, "The Only Unbreakable Law."

• 1:09:19Speaker 00

  Oh, yeah.

• 1:09:20Ryan Peterman?

  What is the only unbreakable law in software engineering?

• 1:09:24Speaker 00

  So yeah, that was, that was a lecture I did, um, where I was trying to think of like, if I was going to say one thing about architecture to people, right? If I was going to say one thing about architecture, what could I say that isn't probably wrong? Cause you think about a lot of our ideas about architecture, it's like even the stuff I just said to you, who knows what we're going to be thinking 10 years from now. It's like, they're not really laws, they're just, they're observations that we've had that maybe this is a good way to do things, but it's not really a law, right? So, uh, the only thing I've seen that feels like a real law to me is the thing I cover in this lecture, which is Conway's law.

• 1:10:05Speaker 00

  Uh, and this is a, a, it's not a real law in the sense of like the laws of gravity or something like, it's not a physics law, like a physicist would laugh at calling it a law. Um, you know, it's probably not even a theory. It's more like a hypothesis at this point, right? But in terms of things that I would place money on being validated as a law at some point, if we ever had the means to do so, this would be one of the only software engineering things that I think, uh, would qualify.

• 1:10:41Speaker 00

  And it is that if you would like to organize a series of, like a set of people to work on something, or in the modern parlance, let's say agents. So it could be, it doesn't have to be a human. It's any system that has sort of its own internal processing, like a human brain does or like a computer does. If you need to partition a problem into a set of workers in this way, where the communication between those workers is slower than their own internal computation ability, which we would all agree is true about humans, it's true about two computers running an AI model as well, right?

• 1:11:25Speaker 00

  The amount of time it takes to send information back and forth to them is significantly slower than what like the GPU can be processing directly on it on its own, right? In systems that look like that, when you start to tackle a problem, in order to get the benefits of that parallelization, you will have to make decisions about who is working on what, at least some decision. For example, if we're making a car and you and I are the two people who are going to make the car, I'm going to design the body of the car, you're going to design the wheels, right? Just a decision someone could make. Once you've made that decision, you have now locked in the fact that the iteration

• 1:12:08Speaker 00

  on the design will be slower across that boundary than it is interior to either of the two parts. So for example, your ability to improve the design of the wheels on their own and my ability to design the body of the car, uh, and improve that on its own will be much faster than our ability to design the interface between the wheels and the tire, like, or, or it's not even just the interface, the degree of harmony between the designs. So the degree to which your wheels complement my body design and my body design complements your wheels, that there's, that they all are considering the trade-offs together, right?

• 1:12:48Speaker 00

  And Melvin Conway's paper, which lays this out, uh, this is a very early paper. It's, you know, it's in the sixties at least, it was in the fifties. It's, it's way back when. That lays this out, points out the consequence of this is that products or any, and when I say product, I mean, anything that comes out of one of these design processes or development processes, products will, will have a similar structure to the organization that produced them. You will be able to see in the product, evidence of this, because the degree to which things are able to harmonize is restricted across that boundary. So when you look at the object, it will have that boundary visible in some way.

• 1:13:33Speaker 00

  Right? It won't be quite as good at the interface between these two things, or in the way that they work together, as the things are themselves inside, like to themselves. Right?

• 1:13:48Speaker 00

  And it gets sort of flattened, like one way the, the rule is pithly stated is like, products look like the org chart. Right? But that's not exactly what it says, but that's kind of a downstream consequence. And boy, do you ever see it in the real world. So, you know.

• 1:14:06Ryan Peterman?

  I've heard that in, in the context of these big tech companies, like, almost like it's a, like a bad thing to quote unquote, ship your org chart or basically, you know, the product you put out there is, has these inorganic.

• 1:14:21Speaker 00

  Yes.

• 1:14:21Ryan Peterman?

  You know, things, it sounds similar, what you're saying.

• 1:14:24Speaker 00

  Yeah, it's, and I think like, part of the reason I think it is kind of an unbreakable law is, I think it's somewhat unavoidable. It's more something that you just have to be aware of and mitigate to the degree that you can, like, you just have to understand, look, that lower communication time, uh, whether it's humans or computers or whoever is doing this work, that lower communication time, uh, or I should say that, that lower communication bandwidth is what I mean. That just means that we will, like, where we draw these lines has consequences for what we ship. And so, we really want to try over time to align those boundary drawings

• 1:15:08Speaker 00

  to places where it will have the least bad impact on the result of the product. And, you know, that's true for org charts. Like I said, I suspect it will also become true for AIs and things like that, where it's like, you will want to partition these problems in ways that respect that, um, that line drawing, because that will be less good than the thing that can be wholly solved, you know, by one, one unit, whatever that unit is. And so to me, uh, it's a very helpful construct for thinking about things. It's also a very good explanation for why you see something somewhere. You're like, why isn't this just integrated in this? It's like, because they were two different teams. You know, it's like, I'm sorry, like, I'm, like, I'm sorry, that's just the answer.

• 1:15:51Speaker 00

  And, you know, it, it costs a lot more for those two teams to have integrated this. That's not how we work. Right?

• 1:15:57Ryan Peterman?

  I, I'd love to hear more about, I guess, your career and how you got into programming.

• 1:16:04Speaker 00

  I started programming when I was really little, because my father worked at Digital Equipment Corporation, which is actually a very important company in that era, no longer exists, right? This was the, they would, they would be sort of like, almost like a case study in how not adapting to a change in technology, uh, leads to your demise. They were one of the most important companies in the world for computing. If you've ever heard of a PDP 11 or a VAX in computing history, that's them. Right? They made these things that were foundational computers in computing history, gone today. Right? They were, uh, part of them got absorbed by Intel, part of them got absorbed by Compaq, but they're, they don't exist. So he worked for that company.

• 1:16:51Speaker 00

  And so we always had computers in the home and I learned to program when I was little. And it's just sort of always what I wanted to do. I really enjoyed it. And so I ended up, uh, getting an internship at Microsoft when I was pretty young. And I met some people there. I came out to, I, I should say, I grew up on the East Coast, so nowhere near Microsoft.

• 1:17:15Speaker 00

  I met some people there. I ended up coming out to the West Coast, really early on, this would be in like 95. So, yeah, what, didn't feel early at the time. It felt late in the computer history, but nowadays it's like, oh, wait, 95? Oh my God, like before, before everything was on the web. Right? Uh, I ended up, there was a web though, it was, it was nascent.

• 1:17:35Speaker 00

  I ended up, uh, coming out and, and working sort of in the game industry. And that's what I did ever since. And I've mostly always worked on game technology. I'm famously like, absolutely terrible at understanding game design. People who watch my stuff know this about me, like a very, very bad, I've tried to make games a couple of times that I'm just, I just cannot do the design side, I'm awful at it. But I really enjoy the engine stuff and I've, I've, uh, feel like I, I'm okay at it and I've been able to contribute to projects. So, in general, most of the time, if you've used code that was written by me, you probably used it in the context of like, a video game that was using technology that I wrote.

• 1:18:17Speaker 00

  And so, you know, an example would be, um, I worked at Rad Game Tools on a character animation system. I wrote the entire thing myself. That is used, well, that's not entirely true. I wrote the entire thing myself, except for the texture compressor, which Jeff Roberts wrote. There was like this texture compressor that you could use, um, as part of the pipeline for the exporting and stuff like that. And it was a pretty cool project at the time. I'm really proud of it. The first version was awful, uh, as it is, because I was pretty new at the time. The second version, I thought we did a really nice job. And it was made at a time when people didn't think you could do licenseable game technology of that kind, because it was too hard to integrate into things, like you couldn't get the performance or whatever. And we did a lot of things that I think were pretty innovative at the time and we

• 1:19:01Speaker 00

  were able to make something that was very successful. And it's, uh, I mean, we released the first version of that in 99. It's still in use today, largely unchanged, from the architecture that I guess was the second version I did in like 2001 or something. There are a few changes to the architecture that people have made over the times, but it's largely unchanged. And, I mean, like, I found out recently Baldur's Gate 3, which was a big game, um, from Larian, came out. I had no idea. It turns out they still use, they use it, right? It's in their engine or whatever. So, I'm very proud of that product, because I think it, it, it was in tons of games and was a very hard problem to solve, and I think we solved it pretty well. It's largely irrelevant today. I would say it's still in some people's like, engines over time, but like, it's not

• 1:19:46Speaker 00

  the kind of product you would make today, because nowadays, engines are monolithic and you license them as a whole typically, right? Like, you, you wouldn't be getting like a character animation library. It's going to be something that's like built into Unreal Engine or built into Unity. Right? So it's not the kind of product you would probably consider making today. So that, that was mostly, um, like in terms of things that I've done that people might have actually have experienced or used, uh, like folks at home. If you've played games, you may have played something that I had a hand in at some point. But only the technology, not the design. Uh, I also worked, uh, a little bit on The Witness, um, which was a game by Jonathan Blow, uh, and, and team, uh, that I thought was absolutely fantastic.

• 1:20:29Speaker 00

  And I did some work on, uh, I did some work on the walk system there that I was, I was pretty proud of. I thought it came out pretty well. There's, there's some parts of it that are really janky. Uh, I don't think I did a very good job of the actual implementation of it, but the design was pretty good. Let's put it that way.

• 1:20:43Ryan Peterman?

  Early in your career, so you worked at Microsoft and then, and then you went to go.

• 1:20:48Speaker 00

  I was, I was only ever an intern. I never, like, I mean, I guess that's technically working there, but I, I didn't ever work there as like an employee.

• 1:20:55Ryan Peterman?

  As a, someone who's writing code, I mean, there's a lot of different paths, but there's one, one path I imagine is you go and you work at one of these big tech companies and create their tech products, I guess. And then video gaming seems like another path, and I think there's other paths as well, of course. What drew you to, you know, going towards video gaming and, you know, not continuing down a Microsoft path or something like that?

• 1:21:25Speaker 00

  I think that there's a bunch of things I could say about that, but they're probably all BS. The truth is probably that I'm probably more, uh, affected by the people around me than I would like to admit. I mean, I guess I don't have a problem admitting it now, but I mean, at the time, I probably wouldn't have said that, right? I could envision a, a, an alternate past where I did stay at Microsoft and, you know, try to get a job there and work there, instead of being just an intern and then going off and working at a different place.

• 1:22:00Speaker 00

  The reason that didn't happen was because of the people who I worked with there when I was an intern. I fundamentally really like, like nowadays, I fundamentally really love doing things like, you know, analyzing assembly code, or looking at exactly how a microarchitecture is working and these sorts of things. If I had gone to Microsoft and had just happened to be under some people who were doing that kind of work and had like, taught me how to write like, device drivers in assembly language or something like that, I, I might still be there today. Right? That's not what happened. The capsule summary is, the group that I was supposed to be in, the way that they

• 1:22:42Speaker 00

  did internships at that time, was that a set of interns, say, uh, three or four of them, would be, uh, underneath a particular manager who was going to like, be managing those interns. So, there were a couple of us who were supposed to be reporting to, to this guy, whose name I won't mention, just in case for some reason, it's ancient, I probably wouldn't care at this point, but, we're supposed to be reporting to this particular person. And literally the week before we arrive, he has this massive flame out with upper management, leaves, like, just walks out of the building and has not been heard from since.

• 1:23:25Speaker 00

  So we show up and it's me, there was like me, this guy named Rudy, a guy named Rajiv. And we're just interns, we show up, we're like, hey, how's it going? And we we report to this test guy, this S D E T guy named Scott Latham, really nice guy.

• 1:23:48Speaker 00

  And we're like, we're supposed to be like reporting to like a program, like a, you know, a software or like what's going on like this is one of the guys in the test org is like, yeah, that that guy's gone, right? Like, he's out of here, right? And and me and this other guy, Bruce Johnson, are gonna like, take care of the interns. Because, eh, right? We don't really know what you're gonna do. The first thing I think Bruce Bruce had me do was like an A and I cursor loader. There was, there's this format, I don't know, ancient history now, but there's this format for animated cursors on Windows. If you ever see the stupid little walking dinosaur or the like, this is, no one uses these anymore, I don't think, but they were this thing that were in was in there. He had me write a parser for loading A and I files, right?

• 1:24:33Speaker 00

  Like, it's just meaningless stuff. So my experience there was pretty lame. Like, I was like, this is kind of dumb. Like, I don't really want to work here, right? But it could have been totally different. Like, if I had been on some team that had like, really inspired me to like, learn the stuff that I didn't know, I didn't know assembly language at that time, really, at all. I think the only thing I'd ever written assembly language was a joystick pulling routine for DOS, because it's the only way you could pull the joystick in DOS, right? And I think that's really it. If I'm honest about it, I think most of it is that I think when you're, even if you're a very brash youngster, which I was, and even if you think that you're very independent, I think you definitely gravitate towards people who you see doing things you think

• 1:25:17Speaker 00

  are impressive or, you know, technologically interesting. And that is just not the experience I had at Microsoft. Now, there were some people who I, like, the people I went out to go work with, were people in other parts of Microsoft who were leaving Microsoft to do like a startup company, right? And those are the people that I was really like, impressed with. Like, those are the people that I wanted to hang out with. So like, if I just look at it in arrears, like, if those people had just been staying at Microsoft and doing something, maybe I would have gone, right? And I think that's the truth of it, as best I can determine at this point, anyway.

• 1:25:51Ryan Peterman?

  So you moved across the country to go and work with those people at a startup?

• 1:25:56Speaker 00

  Yes.

• 1:25:56Ryan Peterman?

  That's feels pretty risky to do, you know?

• 1:26:00Speaker 00

  It was, especially because at that time, a startup is not really what you think of as a startup today. Like, startup is just like some scrappy people in like, a crappy office doing stuff. Like, nowadays, you think startup, it's like, well, we've got $10 million in VC funding and we, you know, like, have free sodas and what it like, in the break room and a massage, a masseuse comes in periodically or something like this, right? Like, I don't know what the what the now version of a startup is, but it's very different, right? So, yeah, it was, it was pretty risky and I don't really know why I did it. It was mostly just because I hadn't had really good experiences with education.

• 1:26:38Speaker 00

  Like, I didn't really want to to do, like, college or like, I didn't want to do any more education. Like, I wanted to actually work, for whatever reason. And this seemed like a, this was just the easiest way to do that. And I don't really regret it, honestly. I've been able to learn most of the things that I would have needed to, like, if I was gonna go do a PhD or something, like, I've ended up doing work that I that is the sorts of stuff, like, I've been lucky enough to be in positions where I could go spend, you know, a few months writing like, this, you know, this particular like linear least squares solver or something like that, like, the

• 1:27:20Speaker 00

  kinds of things you might have done as like a thesis or something, or, or, you know, that walks somebody down the witness, those sorts of things are the kinds of things you would have done had you decided to get a master's degree or something like that. So I'm fortunate enough that I've kind of been able to have that part of the education. Because not everyone gets that opportunity. Like, you may never get, you know, if you, if you don't do a master's thesis or you don't do a PhD thesis, you may never really get the chance to like, really do a deep, like, work on a, on one problem and like, learn a lot about it, read a lot of papers, do, you know, maybe make some novel contribution in some very small way, usually, right? But, and so that, that I was just lucky enough to do.

• 1:28:03Speaker 00

  So I don't really regret not doing something like that. But I think, you know, had I not had that, those opportunities and subsequently, I could see being, I could see being regretful about that, because that is something that I do enjoy doing. And if I'd never had the chance, I would have been sad.

• 1:28:18Ryan Peterman?

  So you, and you went and you traveled to this startup? Because there's, there's no, there's no funding, it sounds like. So, yeah. What did you, it's just a group of guys, just, was there pay or it's just?

• 1:28:30Speaker 00

  Not really, no. It was, it was pretty rough. And I, it didn't last very long, right? I ended up going to work at an actual game company shortly after that called Gaspar Games. And then shortly after that, I started working at Rad Game Tools, which I stayed at for quite some time, and where I did like that character animation system and stuff. So it was, it was pretty quick into not doing that. But it was still a pretty educational experience for me. And also, what I will say is, at the time, so I worked with there a guy named Chris Hecker, who I have to give like, basically, complete credit to for teaching me basically about like, reading technical papers.

• 1:29:11Speaker 00

  Like, before that time, I just thought math was kind of stupid and an annoying thing you had to do like in class, right? And I definitely didn't know anything about like, reading a SIGGRAPH proceedings, really, right? And I may have been like aware, I mean, I was aware of SIGGRAPH, I knew what it was. But I don't think if you'd handed me, you know, that binder, I would have known what that was or what to do with it, like, you know? I think like, the biggest takeaway that I got from that, other than startups are hard and maybe don't do them unless you have a lot of people, a lot of funding and aren't really that much on the line, like, maybe.

• 1:29:51Speaker 00

  But the biggest takeaway that I got from that, that was positive, was just a much deeper appreciation for math, a much deeper appreciation for research, how to read it. That's where I learned to use like Sightseer, which would be like, kind of the precursor to like, Google Scholar or whatever, crawling references and all that stuff. In a lot of ways, you could say, if I hadn't had that experience, I bet I wouldn't have given the two talks that we've talked about for most of the interview. Because what are those talks? They're me crawling every reference, right? They're just me going back and back and back and back and looking at everything that I can find. And that's something that, if you, if no one ever conveys to you the importance of

• 1:30:34Speaker 00

  reading the scholarship and being aware of what's being done, and also teaches you how to read a tech paper and how to parse through it, I don't know that you get that, you know, I don't know that you get that.

• 1:30:44Ryan Peterman?

  I remember early in my career, I think there's this, I guess, common path of like, these, you know, big companies, and then there's this thought of, oh, me and a couple buddies, let's go, let's go build something. Now, now with your experience, looking back, if someone out there is young and thinking about starting, would you say, go that common path? Or would you say, take the the chance?

• 1:31:10Speaker 00

  You know, it's a really tough question. And I think that the advice, like, the advice is that you have to think about what it is that you want to do every day. Like, so, in my mind, the things I regret are the times when I've had to do things or chosen to do things that I wasn't really that happy doing every day, right? And so I think you just have to optimize for the experience you want to have. Because a startup might work out, it might not. It's always a risk, right? You may go to the big company and, you know, it may be cool, there may be interesting things there, or there might not be, like, who know?

• 1:31:54Speaker 00

  Like, you don't really know, you're making a kind of blind decision.

• 1:31:59Speaker 00

  And so I think you kind of have to optimize for what do you want your day to day to be like? What do you want the experience to be? And you want to like, keep a running tally? Like, you want to be aware of it. Like, if you make a decision and six months in, you are not liking what you're doing every day, then you need to get out of that, right? Like, that's my opinion.

• 1:32:22Speaker 00

  So I don't know. I think most people probably, if they sit down and actually like, clear their mind and aren't, aren't engaging in too motivated a reasoning, but just for themselves, go, what do I envision working at the startup will be like? What will I actually be doing every day? Like, am I gonna like that? Is this gonna be thrilling? Like, trying to make this thing work? And like, being kind of on the edge? And, you know, uh, do I want the, the kind of war stories of like, the things we had to do to pull off the demo or whatever, right? If all that sounds exciting to you and you want to have that experience, then you should do that thing. And, and, and I would say like, really, I really mean that.

• 1:33:05Speaker 00

  Like, I don't even think you should take into account, like, even like, let's say you know the startup's gonna fail. If that's, if you want to have that experience, then you need to do it, right? It's just like anything else. It's like going and backpacking across Europe or playing guitar at a nightclub. It's like, you may know that these things are not profitable. It's just, is that the experience you want to have? Are you gonna look back on that six months or a year or five years or whatever the amount of time you're thinking of committing to it? Are you gonna look back and say, I'm glad I did that? Or are you gonna be like, that was the, you know, those, that's time that I would have rather spent some other way, right? And so I think most people can probably, if they're honest with themselves, at least make a pretty good guess. And that guess, if they're honest with themselves, is gonna be better than any advice I'm gonna give them.

• 1:33:49Speaker 00

  Like, my guess about what you should do is gonna be worse than yours. So you should just make that determination. Because another way to look at it is like, you know, that the big company side of it, it's like, maybe you just want to have some security. Like, maybe when you're starting out, like, I mean, I'll just give some examples. Like, maybe you want to, uh, spend a lot of time dating. You want to, you want to, uh, find a partner and you want to have a relationship and you want to have a family. Maybe that's more important to you. Maybe the startup, yeah, it might be fun. Maybe it would be more money if it worked out or whatever. But like, maybe the stability is, is actually gonna be something that, that you would, that you would really benefit from. Because the sorts of things that you imagine being fulfilling in your life are not all around what you're gonna be doing in computing.

• 1:34:34Speaker 00

  I think you can know, have some guess about those things. When you, if you just let yourself have some space to think about it and don't engage in too much motivated reasoning about it, right? And, uh, and I think that would be the best way to make a decision if you're gonna make one. That's my, that's my feeling about it, anyway.

• 1:34:52Ryan Peterman?

  We talked a lot about video game engineering and I have no idea what goes into video games. If you were to just boil it down to the, the big pieces that you need for a video game, what are those software components?

• 1:35:07Speaker 00

  So the biggest thing that's different about a video game is that the sort of original mental model that you're taught, that you're taught in programming, which is like the, uh, standard I O, like, I get some input in, I process it, I put some input out, is like, not how it works, right? So the biggest mental shift is just like, oh, the way that a video game or a simulation of any kind, right? Works is, I need to have some world state and I'm constantly updating that world state on a regular interval. Everything is always happening. There's no, I wait for input and then I do something, right? Everything is always going because it is real time, like, the time is going, um, forward.

• 1:35:52Speaker 00

  So the components tend to be built around this idea. And it depends on what level of sophistication you end up getting into. But in general, you need a way of storing that world state. So some kind of, we usually call these entities, um, like the things that make up a world, right? So some way of modeling, what is in the world? Where are things in the world? What is their state? What are they doing right now? So you need something that does that. Something that is in charge of advancing that state. This is a mixture of several things. It could involve physical simulation. It could involve AI, uh, not necessarily like large language model, like the modern notion of AI,

• 1:36:32Speaker 00

  but like pathfinding, uh, making a decision between whether I should attack the player or not, like, that sort of AI, right?

• 1:36:42Speaker 00

  So we have the, the world, some way of storing the world state, some way of advancing the world state, like an update step that may involve lots of things like that.

• 1:36:50Speaker 00

  Then some way of presenting the world state, so a renderer, right? And this is something that typically, uh, has, uh, I guess I would say it's, it's traditionally been a very important part of a game. Because the visuals are often something that is a selling point for games. People produce trailers, certainly in the AAA space, we're trying to show you how cool the new lighting looks and the whatever. So that is a pretty big component of games oftentimes. You go look at an indie pixel art game, it might be much smaller, right? Because that's a much smaller part of the problem now. That's generally what, what, you know, the shape looks like.

• 1:37:34Speaker 00

  There's a lot of little pieces there. Typically nowadays, we would have some way of asset streaming, right? The, the renderer needs to have things like textures loaded, models loaded, things like that. We need world data, physics, collision models, these sorts of things. They might be too big to fit in memory or we don't want to spend a lot of load time to load them upfront. We want to stream them in as they're necessary. So there's typically like, this thing that's sitting there, constantly grabbing things off of disk, caching things, pulling them in and out of a cache that's being used, uh, by the renderer, by the physics and so on. So that's another common component that's new, didn't used to be there. There's gonna be an audio and music system, obviously, that's in charge of like, you know, when sounds are triggered in the world, ambient sound effects that are happening in the world, music that's playing, continuity.

• 1:38:20Speaker 00

  Again, all of that's interacting with the entity, with, you know, the world state to know which ones of those things are happening. The update step, which will be triggering things in that sound system. And of course, the, the asset streaming to load what sounds are being played or load the music. So we typically have that. And then, um, you know, I'm trying to think, not, I'm trying not to leave out any major components. In a modern, uh, context, there's often networking. So we want to have a way for multiple of these game clients to communicate with a server. So typically what that means is that world, that, uh, state of the world is now, might be provisional. It might be sort of a predicted model of the world that's not the real model of the world.

• 1:39:01Speaker 00

  The, the simulator is actually, the authoritative simulator is actually running on a server somewhere. And I am merely communicating with it to find out what the world state is. And then, because I don't want to wait, the late, I don't want the latency of like, going all the way around, I am predicting the motion of things forward in time based on the last information I have, right? So that's another kind of way that things tie in. If you want to start doing things like competitive multiplayer and all these sorts of things, right?

• 1:39:31Ryan Peterman?

  Oh, that makes a lot of sense. Because like, I, I used to play video games and then like these MMO art, you know, online games. And when I start to lag, everyone continues forward.

• 1:39:43Speaker 00

  Yes.

• 1:39:44Ryan Peterman?

  And then my internet catches up and then everyone jumps to where they actually supposed to be.

• 1:39:50Ryan Peterman?

  Okay, that makes a lot of sense. I pulled some of your top tweets. I thought that might be interesting to kind of discuss.

• 1:39:56Speaker 00

  I'm sorry.

• 1:39:57Ryan Peterman?

  So, you know, someone said NASA does not allow recursion in their code. How crazy is that? And then you said, not even slightly crazy. And it went really viral. Can you explain why that's not even slightly crazy?

• 1:40:12Speaker 00

  When you're writing functions in a programming, in a procedural programming language like we have, and that like NASA is probably using, you are able to use the program stack for storage. I mean, that's what it's there for. It's what local variables are. I call a function, I get some storage space on the stack. The compiler did that for me.

• 1:40:34Speaker 00

  It also saves the return address. So when I make a function call, that program stack is keeping track of where in my code I was, so that when the thing that I called is finished, I get back there, not some other point, right?

• 1:40:50Speaker 00

  Neither of those two things are things you couldn't implement yourself, right? They're both things you could do. You could break the thing up into pieces. You could have ways of remembering what they were, like a state machine. You can have your own stack that you put data on and access it, right?

• 1:41:06Speaker 00

  So the only thing recursion really does is it allows you to leverage the fact that someone already wrote that code for you, and it may be more convenient to use because it's built into the language. Now, there are some things, if you really want to get super technical about it, there are some things that happen at a CPU level when functions are called, but we can put those aside for now. If you want to talk about them after we talk about it. So the reason that I don't think it's crazy to go like, we're not going to use recursion to implement an algorithm, is because if you're doing that, you're kind of just YOLO swagging that there's room on the stack for whatever it was that you were doing, right? And you can't even really check unless you're going to do some kind of weird, like we could

• 1:41:50Speaker 00

  sort of do a thing where we go like, okay, let's try to determine how many more iterations, how many more recursion depths we have before we hit the end of our stack. We can do those calculations, but it's like eh, like, and also if the compiler changed something about the layout, it wouldn't be true anymore, and so on and so forth. So it's like, so if I'm NASA, and I'm like, hey, I don't want my astronauts to crash into the moon. It seems much more logical to say like, don't use recursion, just figure out, whatever this thing was that you're going to do, turn it into a loop, keep a stack, and make a state machine for it. We can reason about that much more clearly. We know exactly how far it's going to take. It doesn't matter what compiler it gets, it would work the same way every time. And it will know the bounds precisely, right? Very sensible to me, right?

• 1:42:35Speaker 00

  And again, you assume at NASA that they're not doing it for their health. They're doing it because they have, you know, hard constraints on the problem domain where someone's life is at risk, and at, or at a minimum, many millions of dollars in equipment is at risk if you screw up, if, if you stack false, like if you, if you get something where you'd recurs to any times and hit the end of the stack, that is not just a, oh, I rebooted the computer, or the, or restarted the software, right? So, so I don't know, hopefully that makes sense.

• 1:43:11Ryan Peterman?

  Andre Karpathy had this famous tweet that kind of coined the phrase vibe coding.

• 1:43:16Speaker 00

  Yes.

• 1:43:17Ryan Peterman?

  And then you said, if you thought software was bad today, buckle up because it's about to get a whole lot worse.

• 1:43:23Speaker 00

  Yes.

• 1:43:24Ryan Peterman?

  It's been about a year and a half since that tweet came out. The tweet came out of February of 2025. Would you say that that was an accurate prediction?

• 1:43:35Speaker 00

  To be clear, I think it, the whole lot worse part is predicated on something that may not happen. And that is that the idea that we just kind of type some stuff into a computer and ship it to prod, right? Basically like, hey, could you make me a thing and then publish it, becomes a common way of doing things, right? For example, also done by people who maybe don't have a computer science background. I think it's fair to say, and that I wouldn't be, uh, being sort of overly dismissive of AI at this point, to say that a person who is well-trained in computer science, using an AI

• 1:44:16Speaker 00

  to make code right now, can make substantially better code than someone who doesn't know anything about computer science, who is just given, you know, Fable and types some stuff in, right? The difference is rather dramatic, I would say, from everything that I've seen.

• 1:44:33Speaker 00

  So part, part of my, uh, concern that I was trying to express at that tweet was like, if the idea is like, we're just going to type stuff in and we're not going to really be checking the code, you know, someone who knows computer science is not really going to be looking at it. Um, worst case scenario, it's literally just like some random person in marketing somewhere who has no idea what programming is, just types some stuff in and crosses their fingers, right? I think we're in for a world of hurt, right? Now, it's a race, so it's hard to say, because it's basically a race of how good can you make the AI versus how much adoption does it get, right? It's a curve, right? It's like, if you can make the AI good enough, that the people who are adopting it at a

• 1:45:17Speaker 00

  particular rate are always using an AI that's good enough for what they're adopting it for, we wouldn't expect software to get significantly worse. If those curves go the other way, we're in a lot of trouble, right? So we're, I feel like right now, we're, we're almost kind of teetering on this knife's edge. It's, it's really, to me, it feels like a footrace of like, improving AI so it can be more autonomous and make better decisions without your, without you needing to make them for it, versus the capability level of people who are using it and the degree to which they're paying attention to its output. It's like these two curves that are just like, ah, you know, like, what's, what's going to happen? I don't have a prediction. I don't know where we'll be in a year.

• 1:45:59Speaker 00

  Uh, obviously, for all of our sake, I'm hoping that the AI curve wins, uh, because I agree, like, I understand certainly the perspective of people who maybe just don't like AI and don't want there to be AI. I can understand the, uh, wanting it to fail. I, I understand that, right? Um, and I'm no fan of AI myself, so it's not like I, like, I'm going to criticize someone for taking that position. But at the end of the day, if you're talking about something that tons of people are using, you're going to kind of want it to be good. Like, I think at this point, given the level

• 1:46:40Speaker 00

  of adoption of AI, I really don't think it's, would be great if it stopped getting any better right now. Like, if this was as good as it was going to get, I think that might be bad.

• 1:46:51Speaker 00

  Certainly six months ago, I think that was true, uh, and I think it's probably still true today. So I think ideally, if you want software to not be terrible, you have to kind of still be hoping that six months from now, the AIs are again significantly better than they were, right? Like, that, that is the only way out of the current situation as I see it, right? Uh, I don't know if that's fair, but that's my, that's my sort of feeling on that.

• 1:47:21Ryan Peterman?

  One of your other top tweets, it was, you know, Shopify put out this internal memo, and you just, you know, you replied, you know, sloppify.

• 1:47:30Speaker 00

  Yes.

• 1:47:30Ryan Peterman?

  So I guess it's because in this tweet, it's leadership pushing the adoption curve maybe harder than the capabilities of the AI in this case.

• 1:47:39Speaker 00

  Yeah, uh, although I also just liked the pun. One of the things that I think is most unfortunate about the AI adoption, as I've seen it, is just the, because people think that it's going to be this major, um, I guess if I had to categorize the way it appears that companies are reasoning about it, they're assuming that if they don't get in early, it will be a big disaster for them, right? Like, like, there's a tremendous, like, they don't just think, oh, well, we can just wait until the AI does what we need it to do and then start using it. They're like, no, we have to do it now, like, even before we know whether it can really do the thing that we want it to do, or whether we know whether the outcomes will be good. Everyone has to do it right now. Let's do this, right?

• 1:48:22Speaker 00

  Um, and I understand why they want, why they're going about it that way, because they think that that is critical, right? They obviously believe that's very important. Um, and to me, that's just, that's just kind of terrifying, because as with any technology, the sane way to do it is to measure its capabilities, see how well it is able to solve problems that you have, see if it solves them faster than the way that you were doing, doing it, and, uh, put it into a workflow at such a time as you've determined that it is a net positive. That's just the same, like, that's what you do with any technology, right? And you'd probably have like your team of people whose job it is to assess this thing, and they're out there YOLO swagging it, right?

• 1:49:03Speaker 00

  They've got 3,000 agents working on this cluster, talking to each other, and doing, you know, God knows what, right? And, uh, so there's going to be that, and someone's going to be doing that, but that is, should not be every org, right? Like, you wouldn't just be like, everyone needs to use a ton of tokens, right?

• 1:49:21Speaker 00

  So yeah, like I, I do have concerns about that. I don't think that, that the way AI adoption was done was, was the best way for quality in software. But I would temper that statement with just the obvious fact that like, we were not exactly a five nines industry to start out with, like, software quality was really pretty low rolling into the AI era. So I always try to just also caveat most of the things that I have to say that might be critical of a particular thing happening with AI, with just the fact that like, look, it wasn't particularly great beforehand either. A lot of this software was pretty low quality. And so you can't,

• 1:50:06Speaker 00

  some AI things may, may make things worse, but it's not like software was amazing and the AI showed up and ruined everything. That is completely ridiculous narrative that is not true at all.

• 1:50:17Ryan Peterman?

  Do you have any top technical book recommendations for any engineers that might be listening?

• 1:50:23Speaker 00

  Uh, no, I would probably use this opportunity to try to pitch reading technical papers. I think something that really the industry could use more of is people being more aware of what's happening. Both like the historical papers, but also just current papers.

• 1:50:45Speaker 00

  It's daunting at first, to be sure. If you don't tend to read technical papers in your field, you will probably find it confusing. You won't know where to find them. You won't know how, how to approach them. It will seem to take too long to read them because you don't know how to like skim them properly and determine whether it's worth your time to investigate a particular section of a paper and all that stuff. But if you're willing to spend a few months of just like, I, at night I look at a paper, you know, or on my lunch break, I look at a paper, you know? If you're willing to spend a few months of just doing that, you will get your bearings and you will start to know where the good papers are in your field. You'll know how to find them. You'll know what, where they tend to be published.

• 1:51:30Speaker 00

  You'll be able to read them much more effectively. You'll be able to know how to spend your time on them. And I think in all, but probably a few small fields that probably exist somewhere that maybe people don't tend to write much papers in or something like that. It's tremendously valuable. And I think it's so valuable that I read papers in disciplines I don't even do. And I find it extremely rewarding. I, I read security research papers all the time. I don't even work in a field where there are security research implications, like games don't really do much of that. They tend to be run like very sandboxed and they don't, you know, sometimes there are, but they're not that kind of thing. They're not like a, you know, web server authentication protocol or something like this.

• 1:52:13Speaker 00

  Uh, and I just find it incredibly rewarding because there's just so much good stuff out there that you can learn. So I would say that would be it. Instead of a book rec, I would say, find a paper, try reading a paper.

• 1:52:24Ryan Peterman?

  How do I go and find that, you know, first paper or someplace to get started that will be valuable?

• 1:52:30Speaker 00

  For most people who are watching, because they, they might be like generalists or, you know, work, working in web dev or in just a tech, general tech org at a, a big tech company or something like that. I would say like, pull up the proceedings of Usenix, U S E N I X.

• 1:52:51Speaker 00

  Look through it for a paper that sounds interesting to you and try reading that paper. Or oftentimes there's a awards, I think Usenix has them where it's like best paper of the conference. Try reading the best paper of the conference, see what you think. Cause that's like a, that's a collection of papers that's usually about like operating system stuff and, you know, systems design stuff. So it's going to be something that most people can relate to. It's not going to be really esoteric. Like if you were to open up the proceedings of Siggraph, for example, it'd be like, oh, uh, you know, neural networks for cloth simulation or something. You're like, okay, like, this is not, I can't relate to this because I don't do graphics or whatever, right? So Usenix might be a good place to start.

• 1:53:36Speaker 00

  But in general, yeah, like if you had, another option would be to go to scholar.google.com, which is their search that just specializes in like papers. And type in a topic description that you find interesting. So like, um, if you wanted to learn, you know, maybe, maybe you were very interested in consensus algorithm, Paxos or something and just like, I don't know what that is, or I, it came up at work and I haven't really ever looked at it. You can just type that in Paxos and there'll just be a list of papers and they'll say a thing like cited by. You can see how many citations they have. That's usually how influential that paper was, like, I mean, to a certain extent. So, you know, those are some ways you could get started and find something that might interest you.

• 1:54:20Speaker 00

  And, you know, it won't be for everyone. You may bounce off it, but it'd be, that'd be my recommendation is something to try. You, you might, you might find it interesting.

• 1:54:28Ryan Peterman?

  And then, yeah, last question for you is if you could go back to the beginning of your career, when you just entered the industry and give yourself some advice, what would you say?

• 1:54:37Speaker 00

  So I think the advice I would have given to myself was to get into low level programming earlier.

• 1:54:46Speaker 00

  I didn't really learn how to like properly analyze and/or even really write assembly language code until probably like,

• 1:54:58Speaker 00

  gosh, 2010 or 15 or like, I mean, it's recent. Cause I'm pretty old and, you know, it, it's like last 10 years or so or something, right?

• 1:55:12Speaker 00

  And, uh, and I feel like I always, I always wanted to know how to do it and just never seemed to. And like, the, the advice I would have given to myself, I was like, just, just go like down the hall. Like what I was interested in, I was like, go down the hall and like, be like, grab some, be like, show me how the heck you write this. So like, just, just show it to me. Like, right? I think one of the problems is, uh, a lot of people, especially when they're young, they are afraid of appearing like they don't know things. And I think that can be a real impediment to learning. I'm sure it was for me. And I probably like, just didn't want to like literally say like, I don't know, I don't understand any of this stuff. Like, can you explain it to me?

• 1:55:56Speaker 00

  And, uh, and I think that's one of the most useful things you can do. Like, I don't understand this. Please explain it to me. It's very useful and most people will be very happy to do that if they're not a dick. Like, most people will be like, oh, sure. Like here, you know, um, now they might not be good at explaining it. There are plenty of engineers who are like really good at something and suck at like telling you how they do what they do. So you won't always get a great explanation, but sometimes you will. Sometimes you'll find the type of person who can explain very clearly how it is they do what they do. So if just keep asking, you'll get, you'll get the good explanation. Nowadays, I wouldn't really need to give myself exactly that advice because the internet has so much great information on it.

• 1:56:41Speaker 00

  I could have taught myself. That did not exist at the time.

• 1:56:46Ryan Peterman?

  Awesome. Well, thank you so much for your time, Casey. I really appreciate it.

• 1:56:49Speaker 00

  Thank you so much for having me. Like I said, I love the show. It was, it was an honor to be invited on. So thank you very much.

• 1:56:54Ryan Peterman?

  Hey, thank you for watching this podcast. If you liked it and you want to see the show grow, please support with a comment or a like. Also, if you have any recommendations for people you want me to bring on, please drop a comment. Guests like Barbara Lizkoff, Mike Stonebreaker, Mark Booker, these were all people that I brought on because someone left a comment. On another note, aside from the podcast, I'm working on building the ergonomic keyboard that I wish existed. Here's a glance at the prototype. It's a split keyboard. So there's two sides. This is in the case, but yeah, we launched on Kickstarter and we hit our goal within eight hours of launching. I really appreciate it if you were one of the people who grabbed one of the early units. We're now working on the long journey of building the tooling now. And so if you still want to pick one up,

• 1:57:40Ryan Peterman?

  I've left the late pledges open on Kickstarter. So you can grab one there. I'll put a link in the description. Thank you again for watching the podcast and I'll see you in the next episode.