OpenAI Eng & Dev Tools Founder: How Software Engineering Is Changing | Charlie Marsh

Charlie Marsh is the founder of Astral, the Python devtool startup that was acquired by OpenAI. I inteviewed him about how software engineering is changing and learnings from starting his own company as an engineer. • My ergonomic keyboard project I mentioned, you can follow along here: https://read.compose.llc/ • The Kickstarter page for it: https://www.kickstarter.com/projects/ryanlpeterman/compose-simple-ergonomics-beautifully-done Podcast links: • YouTube: https://youtu.be/Iw65FD4MGgs • Apple: https://podcasts.apple.com/us/podcast/the-peterman-pod/id1777363835 • Transcript: https://www.developing.dev/p/openai-eng-and-dev-tools-founder Thank you to this episode's sponsor for supporting my work: • WorkOS: makes your app Enterprise Ready with easy to use APIs to add SSO, SCIM, RBAC, and more in just a few lines of code, check them out at https://workos.com/ Timestamps: (00:00) Intro (00:40) Origin story (06:04) The front page of Hacker News (14:35) Why he chose Rust (20:10) Full codebase migration from Zig to Rust (28:40) LLM generated code and open source (35:34) Performance optimizations (44:54) Optimization with AI and combating slop (01:02:08) Learnings as an eng starting a company (01:17:55) Top technical talk recommendation (01:18:56) Advice for his younger self (01:22:00) Outro Where to find Charlie: • LinkedIn: https://www.linkedin.com/in/marshcharles/ • GitHub: https://github.com/charliermarsh • X/Twitter: https://x.com/charliermarsh Where to find Ryan: • Newsletter: https://www.developing.dev/ • X/Twitter: https://x.com/ryanlpeterman • LinkedIn: https://www.linkedin.com/in/ryanlpeterman/ • Threads: https://www.threads.com/@ryanlpeterman • Instagram: https://www.instagram.com/ryanlpeterman • TikTok: https://www.tiktok.com/@ryanlpeterman Referenced in this episode: • Python tooling could be much, much faster: https://notes.crmarsh.com/python-tooling-could-be-much-much-faster • The coolest PR he's ever seen: https://github.com/astral-sh/uv/pull/789 • Andrew Kelley’s data-oriented design talk: https://www.youtube.com/watch?v=IroPQ150F6c • Ruff: https://github.com/astral-sh/ruff • uv: https://github.com/astral-sh/uv • ty: https://github.com/astral-sh/ty • Salsa: https://github.com/salsa-rs/salsa

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  It's such an interesting time to be building software because things are just changing so fast.

• 0:04Ryan Peterman?

  This is Charlie Marsh, founder of Astral, the Python dev tool startup that was acquired by OpenAI. And I interviewed him about how software engineering is changing.

• 0:14Speaker 00

  Now, it's like when you put up a PR, I actually have to review it really closely because you're not writing it anymore. It's like the agent. The cost of putting up a plausible PR has gone to zero, while the cost to review has remained the same. If they're building so heavily with agents, to what degree do they need to understand different parts of the code? I do think it would be really hard to be an early career software engineer right now.

• 0:38Ryan Peterman?

  Here's the full episode.

• 0:44Ryan Peterman?

  What was the space like when you were first getting into building these dev tools? Why did you think it could be better?

• 0:51Speaker 00

  Before I started Astral, I was at a computational biology company and I was the second engineer. I had no bio background and no ML background. And so I was building all the software systems to do machine learning. And so that was all Python. So I was learning Python, then we had some Go and some Rust. And so I just worked across lots of different ecosystems. And I think Python at the time, when I was in that position, we were trying to do a lot as a small team. So we had a very, I mean, not compared to large code bases, but we had a comparatively large code base. And it was really a small group of us that were software engineers trying to serve the rest of the team, whether they were like machine learning researchers or scientists.

• 1:36Speaker 00

  So I had worked across a bunch of different ecosystems, seen a lot of the tooling. And now I was working on Python. And we kept trying to get lots of leverage out of the tools, whether it was the type checker or the linter or the package manager. Things were scaling up. We were a small team. We wanted to have really good static analysis, really good tooling. And we kept running into limitations around it. I mean, I think there were some, I kind of looked at the Python ecosystem and I just didn't see. The same amount of experimentation that I saw in other ecosystems, like in the web ecosystem. There were a lot of things going on that seemed pretty crazy. And there was a very intense focus on, I don't know about very intense,

• 2:20Speaker 00

  but there was more of a focus on performance. And that started with ES Build, and then, which was in Go. And then you had SWC and Rust, and then Bun and Dino and all these other tools that came around. And this idea that you could have native tooling for JavaScript was like, became very accepted, especially as applications became bigger and people were doing more and more stuff on their machine. And that wasn't really happening in Python. And so that, to me, was kind of like the key question was like, why can't we have that? Because I was seeing all this stuff that was happening in web. And I had done some Rust, I'd done some Go. I'd seen these different tool chains and how they work,

• 3:04Speaker 00

  what the experiences, like the user experience, right? Mostly in those cases. And then I saw Python and it was like, we had a smaller set of tools. There were lots of interesting ideas from other ecosystems that I didn't see represented. And they were really all written in Python. And so that, to me, was kind of like the interesting question. And it was the first, it was sort of like the first thing I asked when I started working on this, like when I released Rough. The title of the blog post was something like, "Python Tooling Could Be Much, Much Faster." And for me, it was really like, this is a hypothesis. Like, could Python tooling be much faster? And then I built a prototype. And then that prototype was like, yes, I think it could be.

• 3:49Ryan Peterman?

  You wrote a post kind of like exploring mypy or publishing your results using mypy? And the date on that, like nine days later, you you published the post, that one that you mentioned, like it could be so much faster.

• 4:03Speaker 00

  Oh, that's funny. I didn't even realize that. Yeah, those were so close together.

• 4:06Ryan Peterman?

  Did you build that that proof of concept in those nine days?

• 4:10Speaker 00

  Yeah, I wrote that. I wrote that blog post about basically type checking at scale, because I was trying to think of what are things that I learned that people haven't written much about. And then I started with building a linter because I thought it would be much easier. I think it actually is a lot easier than building a type checker. We're now building a type checker and we have a type, we have a type checker. And so I can, I can say that I think building a type checker is much harder. But I started with a linter because I thought it would let me like prove a lot of the same ideas, but in a smaller form factor. Like building a startup or or just building like a tool, you want to try to like get something into people's hands that they can actually use and that actually proves that value like as quickly as possible.

• 4:54Speaker 00

  And also that lets you iterate very quickly. And a linter weirdly ended up being like kind of the perfect form factor for it because it it has like a pretty simple core. But it's, you know, then you have like tons and tons of rules, right? And so it's like extensible in a lot of different ways. But people can get value very quickly from a small core plus like some number of rules. And so we were able to like get something out there and then like iterate and like ship like more rules and more functionality and people could like use it alongside other tools. Like it was useful immediately, which I think is extremely helpful as opposed to something like, I mean, actually, the type checker would be a good example.

• 5:36Speaker 00

  Like a type checker that's like 75% done like isn't very useful.

• 5:43Speaker 00

  Same with the package manager, like these things have to be, I mean, there are ways like, I think I would challenge those as like blanket statements. But the point is it's harder to ship something iteratively that's like still useful to users. And I think that's like one of the key properties in actually like building momentum as like a tool or something else.

• 6:04Ryan Peterman?

  When you posted that writing, what was the reception?

• 6:07Speaker 00

  I mean, the blog post itself, like I published it and then it was on Hacker News and it got tweeted about a lot and it just created some, you know, some amount of.

• 6:20Speaker 00

  Excitement and interest around it. And those things, I mean, especially being on the front page of Hacker News, I have a lot of thoughts about. I mean, it's certainly not, it's like some percent luck and then some percent skill. And it is like a helpful thing, but not a requisite to success. And it also doesn't like guarantee success just because you got something on the front page of Hacker News. Like, it's great, it's great if you can, but there's a lot of randomness to it. I do think, though, that, well, maybe a couple of things. So, one, after that happened, like, some people started paying, actually paying attention to the project.

• 7:05Speaker 00

  And so I really tried to like capitalize on that by, basically by being like very engaged. And so when people would file issues, I would like aggressively try to like acknowledge the issue as quickly as possible, fix it and ship a release. Like, if you can do that, like, within one day, it's like a very, very powerful loop. Like, you gain supporters and like create more momentum around the project. And there were also a couple, like, high-profile people in the ecosystem that started paying attention to the project, like Sebastian Ramirez, who did Fast, who does Fast API and, you know, a bunch of other things. And he's been a great supporter of our projects, too. But like, early on, he was like, "Oh, this is interesting. I'd like to use it." And I was like, I basically asked myself, "Well, what would it take for him to use it?" And then I just like tried to do all those things.

• 7:50Speaker 00

  So I tried to capitalize on that. I think the other is, though, I do think a lot about like developer marketing. And it's sort of like a dirty word or like a dirty expression. Like, I think, like, I think a lot of engineers like want technical products and tools to just like win on their merits. Like, the best technical solution should just be the one that like wins and grows. But I actually, I mean, I have this sort of like stupid hypothesis that there's all these like, there's like thousands of like really amazing projects on GitHub that like basically don't know how to market themselves and so never, like, never get discovered and never go anywhere. And I don't actually know if that's true.

• 8:35Speaker 00

  But it's like sort of how I feel about a lot of technical work, where it's like, it's actually worth looking at a project like Rough.

• 8:44Speaker 00

  And when I think about like what to put in the blog post, or this extends to like what to put in the read me, or like what to put in the, you don't need like a million emojis and like, and like a million screenshots of like everything. What you need is like, okay, someone's gonna like, like land on this page and I have like 10 seconds to get them interested in what I'm doing and help them understand like why it might matter to them and like why it's useful. And so like that was like, that's like the key idea that I try to bring basically to everything. I mean, it's a little bit depressing because it's like, okay, I only have like 10, but like people, people really don't have like, like this is like the attention economy, right? It's like, it's same thing with a blog post.

• 9:27Speaker 00

  Like ironically, I used to look at a lot of the OpenAI blog posts when I was at Spring and we were trying to, we were doing like machine learning in bio and we were trying to publish material to like explain what we do. But, but we had all these interesting insights. Like we looked at the OpenAI blog posts, especially like the DALL-E ones, like the older ones, and we were all, we were like, wow, that's an amazing blog post because if you read none of the text, you still understand what they did and you're still impressed by it. Like if someone just goes on the page, they leave with an impression and they have some understanding. And I was like, wow, that's like a really amazing thing. And so when I write blog posts now, I'm like, if someone just lands on this page,

• 10:10Speaker 00

  I have to be prepared for the, for the large number of people who will only read maybe the TLDR, maybe look at the first image, maybe read the headline. Some people will read the full thing and so I should care painstakingly about every word and everything it says. But most people will read like almost none of it. And so again, sorry, it's a little bit like, it's a little bit depressing, but like, I do think it's worth thinking about your audience and like, how do you, how do you explain to them like why they should care about what you're doing in a way that's honest and genuine? Like not, like not like deceptive, but you do have to think hard about how do you communicate like why something matters to people very quickly?

• 10:47Ryan Peterman?

  Could you give an example of like what you did in your case?

• 10:51Speaker 00

  I think like for Ruff specifically, we had a really good graph of the benchmarks that we put in the blog post and that like went. It was like on Twitter and like that, that was like a priceless graph, basically. Sorry, not in terms of like monetary value. I just mean in terms of like attention, right? It was like, oh, that's like, that's like really obvious, like what's happening there? I think graphs of benchmarks can be like a really powerful thing. All benchmarks are bad and wrong in some way. And so there's a lot like, there's a lot more to like say about that like, but, but I do think like having a visual hook that explains to people why they should care and like what something is.

• 11:33Speaker 00

  And having a really strong tagline, like I would have to look back at what we had, but we were like, okay, we're going to be like compatible with your existing stuff and much faster. And that's kind of it. And it's like, okay, well, if it's compatible and it's much faster, like that was like, I was like, why should you care? Like it can be orders of magnitude faster and it'll give you the same experience. You try to convey that like as quickly as possible.

• 11:58Ryan Peterman?

  Right. And even the, I mean, that blog post header, I mean, Python tooling should, could be much faster, should be much faster.

• 12:06Speaker 00

  Yeah, yeah.

• 12:06Ryan Peterman?

  That instantly.

• 12:08Speaker 00

  Provocative, I guess. Yeah. I mean, I'm pretty against like clickbait or like trying to be misleading in like anything that we do. Like we, we, you basically want to, want to balance like creating like effective, like effectively communicating like the most interesting pieces to people. But you have to do it in a way that like you believe is like true and authentic. Like you could lie about a bunch of stuff and like post interesting things that are like clearly provocative, but like, I don't know. To me, that's just like, why, why would I ever do that? That's like such a losing battle.

• 12:43Ryan Peterman?

  What, what do you think about, I mean, so you have that graph. I'm, I don't know if the, I see this a lot though, where the axis is not zeroed.

• 12:53Speaker 00

  Oh, yeah.

• 12:53Ryan Peterman?

  So it's, you know, it's, it's all within the range of like 50 to 60.

• 12:56Speaker 00

  That's just chart crimes. Yeah.

• 12:57Ryan Peterman?

  Chart crime.

• 12:58Speaker 00

  I wouldn't do that. Yeah. Yeah. I do think like benchmarks are pretty complicated and controversial though. And maybe most people don't even see this controversy, but like I do, cause it's like, it's just very hard, like performance is very nuanced and.

• 13:20Speaker 00

  You know, like, like in, you know, you have to think about like, okay, with caching or without caching, those are like pretty different. Or like, depending on the project that you're running on, the performance characteristics will be like totally different. And like, we see this, like in UV, this is really hard. Because like, sometimes, you know, UV is really fast. But sometimes your package install is actually bottlenecked on like a C++ build that's like totally out of band. And it's like, okay, well, so how do you like accurate? It's like, I mean, it's sort of impossible to like accurately capture like all of the different scenarios. And so, but it's also like a disservice to users like not to

• 14:04Speaker 00

  try to communicate to them like the importance or like a real representation of like, what will the difference in performance look like? And why does it matter? And so you have to balance those things, which I think is actually quite hard. And a lot of people tend to get it wrong. And maybe we've gotten it wrong a few times too. I'm like, I'm happy to accept that. But I do think it's more complicated than people think. But it's also, I think, a huge disservice not to include like some way to try to convey to people quickly, like what things feel like.

• 14:35Ryan Peterman?

  I think one of the things I'm most interested in is like, you, you mentioned that graph. I see that graph and like, you know, the engineer in me is like, how, how did you make it so much faster? And one of the things in the tagline is Python Dev Tooling Written in Rust. And so I guess one of the first thing I'm curious about is why did you choose Rust? And what are the pros and cons of that?

• 14:57Speaker 00

  I would say I largely chose Rust for that project because of hype. Like, I don't think at the time I was that informed about a lot of the technical trade offs between those different ecosystems. And I hadn't really done a lot of systems programming. And so that's, that's like the honest answer is like, I was seeing it in a lot of places and it was growing a lot and people said it was like, it appeared to be sort of like an accessible way to write. My impression was that it was an accessible way to write high performance software. And, and so I started for that reason.

• 15:34Speaker 00

  In hindsight, I think it was an amazing decision and I wouldn't absolutely not do it differently. And I think it was like the best choice for what we're building. And now I have a lot more experience writing Rust. But, but it is funny because like I, I do sort of think that if I had tried to write this in like C or C++ I think I probably would have given up because I find those, even now, I find those ecosystems much more like intimidating, hard to, hard to access, hard to learn. I have often felt that like one of the great or like underrated advantages of Rust is basically like

• 16:18Speaker 00

  the tooling because like you drop in and you use cargo and that's how you do like everything. And if you clone, this isn't always true, but like, you know, there are exceptions as projects get sufficiently advanced, but like if there's like a crate out there that we use and maybe I want to submit a PR to it. I'm like extremely confident that I can clone it and figure out how to run and build it without having to do like any work, like being able to just clone and run like cargo run or cargo build or cargo test. That's actually like really amazing, especially for someone like me who was like new to systems programming and I didn't want to have to figure out like my whole like C++ like tool chain and like build system, like all this stuff. Rust was like opinionated about all of these things.

• 17:01Speaker 00

  And so there were a lot of things that were hard to learn, but, but I was allowed to like, basically, I was allowed to spend my time focusing on the things that like actually should be hard to learn as opposed to like all the other, basically all the bullshit that like you don't want to spend time on. Like, how do I get this thing to actually like build and run and blah, blah, blah? I think it has been an extremely good bet for us. Like it's scaled very well with the project and I, I mean, we at OpenAI too are like, we use a lot of Rust and they're like betting heavily on Rust. And then, you know, me as someone who builds software for, builds developer tools, like tries to build

• 17:45Speaker 00

  software for people to build software, I'm like very bullish on Rust.

• 17:50Speaker 00

  Like it's grown. Enormously, and I think it will actually just like keep growing enormously. At the same time, like I said this at the start, I've never been someone who's like super dogmatic about ecosystems. Like, I think that like all of those ecosystems have like lots of redeeming qualities. Like, I think what's happening in Zig is like very interesting and like I wish I had more time to like go deep on it and like develop more nuanced opinions about like what it does better. Like, I'm sure it does a bunch of things better than Rust. And there are, I'm sure there are things for Rust to learn from that ecosystem. Similarly, people have a ton of success building stuff in Go. I think that's great, too. I think like, I think all this stuff is great. But I have loved like using Rust and I have complaints about it, but I view those complaints as like things

• 18:33Speaker 00

  that we should improve, especially as like the way we build software changes with LLMs.

• 18:39Ryan Peterman?

  So if, it sounds like you're saying C and C++ today, you would not consider them because their tooling is insufficient, but Zig, Rust, and Go are all, you know, some spectrum of trade-offs or different things?

• 18:56Speaker 00

  Yeah, I mean, I think, I think a lot of people would disagree with me, but like, I don't really understand like why, unless there are like very specific technical reasons, or you're working with existing software. Like, basically, I don't really know why I would start a net new project in C or C++. I certainly would not, like, at a personal level. I guess if they're like the ecosystems you know really well, like, that's, then it makes a lot of sense to do it. But if I was like a new programmer, like looking to learn systems, or if I knew those languages equally well, like, I guess I just don't understand like why I would do that, which I'm sure I'll get like roasted for, but I, but I, I just don't really care. Like, now

• 19:41Speaker 00

  I actually care about a lot of things that Rust gives me that I didn't care about as much before, like memory safety. I don't think I even understood or cared about what memory safety was when I started working on Rust, to be honest with you. And now I like, I think it's like a pretty amazing thing. It's like, you know, memory, I want to build things that are like safe, that don't crash on users, that are really fast, really performant, that don't have to make compromises, basically. And like, I feel like I can do that in Rust. And so for me, it's like, I don't know, it's kind of like the perfect language. I mean, it's not the perfect language, but it is my favorite language to use right now.

• 20:12Ryan Peterman?

  In today's world, it's not unreasonable that you could almost completely transpile or convert your existing codebase into any language of your choice.

• 20:23Speaker 00

  Yeah.

• 20:24Ryan Peterman?

  Like I saw, I think it was on Bun, I forgot, I think it was Zig to Rust.

• 20:29Speaker 00

  Yes.

• 20:29Ryan Peterman?

  Like all in one go.

• 20:31Speaker 00

  Yep.

• 20:32Ryan Peterman?

  Like if you studied some other language and you thought it was better, you could totally rewrite everything from Rust into, I don't know, Zig or Go. Would you ever consider doing that for the dev tools that you've built, for instance?

• 20:47Speaker 00

  Yeah. It's such an interesting time to be building software because things are changing. This is not a novel comment, but things are just changing like so fast. Like I didn't, I didn't really program with agents at all until like Christmas, like December break, basically, of this year, or of last year, rather. And like, that's really not that long ago. Like, I mean, I was like using Cursor and stuff, but like I wasn't like, now, like I haven't edited code in an editor in a very long time. Like I, everything I do goes through Codex and like even if it's like small edits, I'm just prompting and kind of like, it's like, it's completely different. And that's not that long of a time, right? And so, I don't know what's it gonna look like in six months, hard to say.

• 21:30Speaker 00

  But so, you know, the reason that I mentioned that is, like, I think Bun, like doing that rewrite, it's like, I wouldn't do that right now.

• 21:44Speaker 00

  But I'm actually like, I'm glad that they are like pushing, that they are experimenting and pushing the envelope of like what's possible. And, and they're doing it in the face of like a lot of criticism. And it will be interesting to see how it plays out for users. Because I think a few different things I thought about is like, oh, reasons not to do that. Like, one is, Okay, maybe you don't understand the code anymore, right? Like, if the code got completely rewritten, quote unquote, then as a human, you may not understand any of the code anymore. And that can be a big problem. I think there are mitigating factors around that, which are like, they tried to do a very direct transpilation, quote unquote, I mean, it's not exactly a transpilation,

• 22:27Speaker 00

  but they tried to do like very one-to-one. And so that hopefully helps with like, understanding all the abstractions. If they're building so heavily with agents, like, does it even matter? Like, to what degree do they need to understand different parts of the code? I actually don't know the answer to that, which is where my, everything's changing so quickly, comes from. It's like, it's actually a little bit hard for me to say right now how much that actually matters anymore of, if they did transpile the whole codebase, like, how well do they need to understand things at different levels of abstraction? Like, they should, you know, it's, it's actually a very, like, I think, complicated question. So one reason is like, how well do you understand the code? That's a confusing. The other is like, you're kind of trading, like, when you do a rewrite like that,

• 23:13Speaker 00

  you're trading known issues for like, new unknown issues. Like, imagine you merge that and it closes like, 50 open issues on GitHub, in a literal sense. Okay, that's great. But you may have now caused like, 50 new issues that didn't exist before, that you don't know about. Because your test suite is, even a great test suite is only like, an approximation of like, your program's correctness. There's like, Hiram's Law, which is like, anything, I'll get the phrasing wrong, but I mean, the gist is basically, any like, implementation detail in your software, someone eventually becomes to rely on that. And so even things that aren't encoded as behavior

• 23:57Speaker 00

  become part of your API, whether you like it or not, which is a scary thought. But basically, that means that even if you pass your whole test suite, there's probably like, lots of behavior that was like, implicitly encoded in your program, that's like, changed. And the unfortunate thing is, you end up pushing that onto users. Like, they are the ones that have to run into that and then report it and figure out what went wrong. And so that's like, that's actually like, my main area of concern with an automated rewrite. I actually trust the models enough that... I would consider allowing them to do it. But the scarier thing for me is basically, what's the impact for users? Like, how do you and how do you roll that out safely? So, I don't know. It's kind of a, we're not gonna like, we don't have anything planned to like, do that for any of our own projects. But it is like, an interesting question.

• 24:41Speaker 00

  I mean, I think maybe like, the third piece that I do think about, especially right now, where like, everything's changing so quickly. And there is, you know, the, basically, the spectrum of like, how you write software has like, gotten like, way wider. This is a terrible analogy, but there's like, so many different ways to build software now. Like, there's like, the true like, Andre Karpathy definition of vibe coding, where you're like, don't look at the code at all. All the way to like, you know, don't use L L Ms and blah blah. There's like, a lot of different stuff in between. And I actually think that like, right now, I have pretty different approaches to different projects, in terms of what I do.

• 25:24Speaker 00

  Like, we, like, I built, you know, a tool

• 25:31Speaker 00

  last week, that was, it's basically personal software. It's like, a custom Rust linter to help us enforce certain things in our projects that I've like, always wanted to enforce, but didn't fit into existing tools. And I didn't read any of the code. Like, G P T five five did the whole thing. And, and it's like, okay, I can have a really different standard for that than U V. Because that project, we're the only users. It's, it's easy to tell if it's correct or not. You know, not to go too deep into the weeds on like, what the project does, but like, we know if the output is like, incorrect. And, and there's no other users.

• 26:14Speaker 00

  U V is relied on by millions of, you know, millions of engineers and like, all the, like, every single company. And so, it's like, we want to be really careful about like, what we ship there and how. But there's a lot of room right now, I think, for exploring like, how we build software. You do have to be careful, like, what is your, what, what project are you working on? And like, what responsibility do you have to users? And that's not meant to be a, a subtweet on like, what Bun did. It's just like, how I think about.

• 26:43Speaker 00

  How do we want to push the envelope in like different places?

• 26:45Ryan Peterman?

  Right. I mean, to me, that just reminds me of the pre LLM trade off, the like navigating the, I guess, the risk of the code change and how much you verify it because if it's not risky, like it's a internal dev tool that only you and I are using.

• 27:02Speaker 00

  Yeah, yeah, yeah.

• 27:03Ryan Peterman?

  Yeah, maybe I just, I ship it with, I just say stamp it, it's, don't worry about it, it's not going to break anything. But if it's, you know, the, the critical thing that's blocking customers from checking out or something, maybe we, maybe we have two people reviewing it and there's like a lot of checks in that part of the code base. So I, it feels like the similar spectrum is just now, like, there's a new level of trying less, which is not just, like, a human doesn't even interact with it. You just, you know, ship it and maybe you just stamp it and no one ever even looked at it.

• 27:39Speaker 00

  Right.

• 27:40Ryan Peterman?

  Whereas before, at least to have written it, a human would have had to, you know, do it.

• 27:46Speaker 00

  Yeah, yeah, yeah. Yeah. I mean, I think, well, okay, a lot of reactions to that. So one, I, I guess like the other thing I would just say quickly about Bun is like,

• 27:58Speaker 00

  sometimes I go over to their repo and it's like, it's just so different than our repo. And it's very interesting to look at because if you look at like, basically the number one contributor is like the, the robo-bun bot. And sometimes you'll click on PRs by humans and there's like human accounts talking back and forth, but it's clear that all the comments were written by agents. And you're like, wow, this is like very interesting. And so I look at that, but my perspective on that is basically like, okay, I'm glad that people are like, that they are like experimenting with like different ways to like build software. Like, I don't know that I'm ready to do that, but like, but like, but I do think that like a lot of, basically a lot of things are going to change and I think it's very interesting to be experimenting. So I'm glad that they are experimenting with things is my, sorry, is my like conclusion there.

• 28:40Ryan Peterman?

  Do you block, like if people are just submitting full agent stuff on your repos?

• 28:46Speaker 00

  We, we do. Yeah. So we have an AI policy now, um, which isn't intended to be like anti-AI, but it's really like intended to, because, I mean, we develop like very heavily with LLMs, um, like that is true. But it's, it's more intended to be like, how do we?

• 29:08Speaker 00

  How do we retain useful contributions while filtering out net negative interactions from the repo? Because a contributor coming in and posting a comment that their agent wrote, and then we ask them a question, and then they just paste the agent response back. There's very little value in that, right? We could just ask the question to the agent. And so we want to attain the things that are useful from human contributors, which is insight, ways to reproduce, the information we would need to plug it into our agent. Because there's no point in just having a conversation with an LLM on a GitHub issue.

• 29:53Speaker 00

  It doesn't really do anything for us. So our policy is roughly, you need to understand what you're pacing in, or what you are submitting. Which sounds like a low bar, but it's not.

• 30:12Ryan Peterman?

  How do you police that?

• 30:15Speaker 00

  If we can't tell, and it is agent-written, then I guess that's fine.

• 30:23Speaker 00

  But at least right now, there are just lots of tells.

• 30:30Speaker 00

  Being way more thorough than a human would ever be, including way too much detail, way more formatting, using terminology that a human wouldn't use, like inventing random jargon, tons of links. I guess the broad sign would be putting in way more effort in ways that aren't necessary, in a way that a human wouldn't, is almost always agent-authored. But it's been challenging. I mean, I think the hard thing, Zig has a much more strict LLM policy, which is no LLM-written code, or no LLM-assisted code at all in the project, which is very different than us. I mean, all of my PRs now are LLM-written. And they had a blog post about this idea of, they called it something along the lines

• 31:14Speaker 00

  of contributor poker, where it's like, you're trying to place bets on contributors. Historically in open source, when a contributor came around...

• 31:24Speaker 00

  They might be like new to the project, but they could show like signs of promise or like they're really engaged or really interested and like you give them feedback. They take the feedback and they fix up their PR and then the next time they put up a PR, they've learned from that feedback, right? It's the same way as like mentoring a new engineer who joins your team. It's like they learn from feedback, it compounds, they get better and then they can actually mentor someone else in the future, right? And so you're making investments in people in a way. It's the same for open source. It's like you wanna, I thought about this a lot historically. It's like you wanna like invest in good contributors who are gonna grow to help others and be maintainers in their own, right? That's kind of gone if for PRs, especially that are written by agents, because people don't really learn anything.

• 32:07Speaker 00

  Like someone can put up a PR that's written by an agent and you leave comments and then they take your comments and put them into the agent and then update the PR and then merge it and it's like there's no compounding feedback. And so I actually do like understand that perspective a lot of like the contract between like maintainers and contributors is like pretty different now. And you know, relatedly, again, not a novel insight, but the cost of putting up a plausible PR has gone to zero while the cost to like review and vet a plausible PR has remained the same and is very high, like especially for the projects that we work on where, you

• 32:52Speaker 00

  know, I'm not trying to like overstate it, they're just like hard projects, like TY is our type checker, probably hardest project I've worked on technically to like get a change merged because of the like complexity in the architecture, but also like the problem space, like it's just very, very hard. And so if someone puts up a plausible PR to TY, it might take them two minutes and then it could take us, you know, an hour to understand it. And so it's created like very poor

• 33:20Speaker 00

  Dynamics in open source and I don't know how we've solved that. I think it will like continue to get worse until hopefully it gets better in some way. Like we find ways to solve this, but it's certainly something we've experienced in our projects.

• 33:36Ryan Peterman?

  Right. It sounds like review is becoming more and more of a bottleneck. Is for that Bun rewrite, do you know if it was a human, like humans read that code? Or that was just kinda?

• 33:46Speaker 00

  No, I don't think it was human reviewed.

• 33:48Ryan Peterman?

  So it's just Yolo rewrite the whole thing?

• 33:51Speaker 00

  Yeah, I mean, in a way, I think it's like, I think they did have, and I'm excited for like at least that time of at time of recording. Quote unquote, you know, Jared hasn't published the blog post yet, which is like, I mean, he keeps making jokes about how like the blog post taking way longer than the rewrite. But I'm actually really interested in the blog post because I think they did have a pretty like sophisticated approach to like how the rewrite worked. Which will be interesting to read about. Like it wasn't just like they had one prompt that was like rewrite this in Rust. It was like they tried to have, at least my impression from looking at the PR is like there were, there were actually like different phases of the rewrite. And like each file had to go through like a couple of different phases that had like ways to validate whether it was correct or not.

• 34:35Speaker 00

  So the answer is no, though I like, I don't think a human was reading like significant amounts of that code. Like it's just not possible, basically.

• 34:45Ryan Peterman?

  What if we, we both agreed today that Zig was objectively better than Rust? And then someone on your team said, you know, I can do this rewrite into Zig. Would, would you block it if?

• 34:58Speaker 00

  I think I would block it. I think, I don't know whether I'd be right to do so, but I think like we as a team like aren't there yet. And, and maybe we will like get there and that's how we will feel eventually. But like right now, I think we just feel like right now, there's still a lot of value in like deeply understanding like our code and our tool chain and like the ecosystem. And so I would say we don't want to do that, but it might depend on the project. I don't know.

• 35:34Ryan Peterman?

  All the things that you've built, they're like an order of magnitude faster than what was available at the time. How much of a difference did Rust make on that graph?

• 35:45Speaker 00

  I think it, so it depends a bit on the project. I think, I think in Rough, a lot of it was just Rust. And then over time, I think we've improved on that a lot because, because you can even look at the history of the project, like the project, I mean, hopefully no one like, hopefully this is still true, but like basically over time, the project gets faster. And sometimes that regresses, but like, you know, you basically, maybe I'll put it differently. You could write Rough, you know, in like, a couple of different ways, all in Rust, and they could have really different performance characteristics.

• 36:30Speaker 00

  So that's just to say that like, I generally think of Rust is like, the floor, or the, that's sort of like the baseline performance that you get is going to be significantly better. But you still have to, you still get a lot more out of like, thinking deeply about performance and design. Like if you take the same program in Rust and in Python, like exact same implementation to the closest approximation you can get, like the Rust one will be faster. But you can then take that program and you can probably optimize it another like 10x. I don't know about 10x, but my point is, even within being written in Rust, there's like a ton of room for how to make things more performant, how to write really performant software. And like, again, when I started working on Rough, part of my, my goal was to learn Rust.

• 37:14Speaker 00

  And so I wrote, I did write a lot of like, bad code, which is fine. Like, I shipped something out that was really helpful to people, but it's gotten a lot better, I think, over time, and like, we've made it more and more performant. I think in UV... There was more sort of like architectural innovation beyond just being in Rust, especially because UV, like, so as a package manager, you're doing a ton of I/O, like downloading and, you know, downloading files over the network, unzipping things, like writing them to disk, moving them around. The Linter doesn't have to do as much of that. I mean, it has to read all your files, but there's, there's not like a significant, a huge amount of I/O. The package manager is like mostly I/O.

• 37:58Speaker 00

  It's like, and then you're trying to do things like very efficiently. So there it was more, I mean, Rust was important, but I, I do think that in UV, there's more, you know, architectural things that we did or like ways that we thought a lot about performance, like the cap, the design of the cache is like very, very intentional and makes it so that like repeated installs of the same package on your machine are like, are like near instant because of the way that we like lay out the cache and the way that we install from the cache into your projects. It basically means that if you've installed a package before, installing it again is extremely cheap and both in terms of disk space and time.

• 38:43Speaker 00

  And so that was like, that's like a very different design than any of the other like Python package managers had. So again, it kind of depends on the project. I do tend to think that like, whether you're writing code in Rust or in Python, there's like always room to like be thinking about performance. Like you can always make things faster or slower. Like even if you're writing in Python, like you can still make things like much, much faster by like thinking harder about performance and design. So it's some mix, but it depends on the project.

• 39:14Ryan Peterman?

  OpenAI, Anthropic, Cursor and Vercel all use this product to make their lives better. And the problem it solves is when you're building SaaS or an AI product and you want to sell to other companies, there's all these requirements you need to meet. There's SSO, there's SCIM, there's RBAC, there's audit logs. These are all things that take time to integrate, but aren't the main focus of your app. WorkOS is an API layer that lets you meet all of these requirements in just a few lines of code. So let's say you have a new SaaS product and you want to sell to other companies. WorkOS will solve all of these critical feature gaps for you. You can check them out at workos.com to learn more and get started.

• 39:56Ryan Peterman?

  And I appreciate them for supporting my work and sponsoring this podcast. Over the course of the project, do you have a top few things that were implemented in the project that were technically challenging or most interesting to you?

• 40:11Speaker 00

  I really like this optimization that Andrew on our team, who goes by Burnt Sushi, he's the author of RIP GRAP and a bunch of other things. He's a really amazing engineer. He did this really cool optimization around how we represent versions. It's pretty cool. I mean, basically, if you think about resolving and installing a very complex Python project,

• 40:35Speaker 00

  it turns out that we have to parse and create lots of versions, like as in 1.0.1, 1.0.2, like we just like version objects within the program. Like, we end up parsing and creating a lot of those. And it turns out that actually allocating that memory was expensive, given the scale, like the number of times we were doing it. And he came up with a representation where we can represent like 90-something percent of versions with a single U64 integer. So it's just like way more efficient. And the benchmarks around that and the implementation were very cool. It's like one of the coolest PRs I've read, I think.

• 41:15Speaker 00

  In TY, which is our type checker, there's a lot of very interesting performance work that's happening, especially to make it incremental. So like TY is designed to be a type checker and a language server. And the whole system is like highly incremental. So the idea there is like, if you're in a text editor and you open up like one file, you don't necessarily want to like have to type check your entire project, like all your dependencies, every file in the project, just to get analysis for that file. Like, because... You don't need to. So how do you make that work?

• 41:56Speaker 00

  That's like, that's sort of like, that would be like lazy. You want to be lazy. But the other piece to that is like, if you have a file open and you edit it, or you have like two files open, you edit one of them. You only want to recompute like exactly what you need to recompute. Like, you don't want to have to go and retype check like the entire codebase again. Especially because maybe you're working in a big project like PyTorch and it's like, you have two files open, you edit one of them, you don't want it to like, and then you save, you don't want it to take like two seconds to like retype check the project and like give you a new analysis. So the whole system is built around queries, which is pretty interesting. This is more of like a macro design thing. But we built it on top of a framework called Salsa, which is also what Rust Analyzer uses,

• 42:41Speaker 00

  which is like the popular Rust language server. And now we've intentionally or inadvertently become like, like very large contributors to Salsa. But, but the whole, the whole system is built around that, which has been like very interesting architecturally.

• 42:57Ryan Peterman?

  Interesting. So it's, it's lazy, so it doesn't type check the whole codebase, and it's incremental.

• 43:02Speaker 00

  Yeah, the incremental part is the thing that's hard because you kind of need a way to, basically like, it needs to be able to model kind of like a dependency graph of like everything that's happening in the code. And so then, when you change something, we want to just like flow the data back through all the different pieces, like only the pieces that need to be run. Yeah. So that took a lot of work, but it has come together. And I've been doing a lot of optimization lately with Codex, because it tends to be very good at, especially at like micro optimizations. Like if I'm like, and in TY in particular, we also need to think a lot about memory,

• 43:46Speaker 00

  not just speed, because if you work on a very large project, like you don't want it to take like many, many, many gigabytes of, you know, just to like run your language server. Like ideally, you want it to be like relatively efficient. So like I spend a lot of time now, kind of like continuously optimizing like memory usage and performance. And I can just set a goal that's like, try to reduce memory, like Salsa memory by like 1% on this project. And like, you can't do like these, these things that you might like, like try to do.

• 44:18Speaker 00

  And it's very good at just like coming up with like very reasonable things. And so I don't know, I find that very cool because it's kind of like you can almost have like continuously. I won't say self optimizing, that's like a grandiose, but you can kind of be like continuously like just like optimizing your software. So I've been enjoying that a lot, but, uh, but most of those are like, um, yeah, trying to find ways to like represent things that take up less memory. Or trying to come up with like, sometimes it's like trying to come up with broader redesigns to like fix pathological performance.

• 44:55Ryan Peterman?

  That one optimization where with the version numbering and seeing that you can limit it just to like a U64. If you think back to some of these more creative optimizations that were done, that was in the human era. Do you think if you just, I don't know, ran codex and you're like, hey, just don't break things, but lower. Do you have faith that it would come up with that or is that kind of a step above what?

• 45:20Speaker 00

  That's a very interesting question. If you just, at least in my experience, like if you just ask these things to reduce memory or play in performance by some, you know, moderate percentage. It will typically come up with things around the edges as opposed to like larger redesigns or reconsiderations. But you can get to those larger redesigns if you prompt and collaborate with the agent. And so if you sort of ask like, well, like, should we be thinking a bit bigger about like why we have to represent the data this way? Like you can, you can get to like bigger ideas. And so that might be an idea that you could have gotten to with prompting if you were like, um, you know, okay, let's start by like profiling and figure out where we're spending a lot of time.

• 46:04Speaker 00

  And then maybe the, you know, it would eventually come back to you and say like, we're spending a lot of time in like version parsing and like version like ball, like version drop and version allocation and like blah, blah. And we'd be like, okay, well, like, how could we represent these like more compactly? And we'll probably start by finding like micro optimizations in the representation of like, well, this field, like you can find these two fields to like save a few bytes, like blah, blah. But I think if you kept pushing it, like it's possible it would get there. But that's not the thing it's going to come up with by default. Mitchell Hashimoto, um, who was like one of the Hashi Court founders, works on Ghosty. He had a post that was insightful, I thought. Last week or this weekend about a renderer he wrote where it was like a really terrible renderer.

• 46:48Speaker 00

  I'll probably butcher the tweet, but it was like, it was a really terrible renderer. And then he hadn't like intentionally, and then he had an LM like optimize it and it made it like 10 times faster. And he's like, great, right? Actually, no, like my handwritten version was like 100 times faster. And it's like, you know, like you lose, if you're not like using your brain to like, think about from first principles, like how fast should it be? And like, how should the system work? Like, then you just like ship all this, these like accumulated like, I mean, I won't necessarily call it slop, but it's like, you just ship these like things and you're like, yeah, oh my God, I made it 10 times faster. But really it should be like 100 times faster. And so I think, I don't know, it shouldn't be controversial. There's still a lot of room for like using your brain, right? I'm like, uh, but I do, I do think it's like, I struggle with this stuff a lot.

• 47:33Speaker 00

  I mean, like I'm changing how I write software a lot. And, um, you know, I've had people on my team, like, cause I'm using agents a lot. And, and also it was, you know, to some degree, trying to like,

• 47:45Speaker 00

  push our team to like, use agents more. I mean, I mean, some of that for me came from a place of like, we build tools for software engineers. And a lot of our users are now using agents. And so if we like, if, if, if everything good that we do, that's an exaggeration. But if everything good that we do comes from like, understanding our users very well and like how they work, then like, we should probably be like using agents that we understand like what it's like to build software for, with them, so that we can build better tools for our users. That was part of my motivation. And part of it was just kind of seeing how it can change, like how you work. And I don't know, I think I'm shipping a lot more. But, but it has been like, um, it has been like a difficult process.

• 48:28Speaker 00

  Like I've had people on the team tell me, um, uh, which I think is great that they tell me this. But they're like, oh, it used to be the case that like, whenever you put up a PR, I could like review it pretty minimally. Cause I had a lot of confidence in it and like your work. And now it's like, when you put up a PR, I actually have to review it really closely because you're not like writing it anymore. It's like the agent. And I was like, wow, that's like, that's like very interesting to, they're completely right. Which is, and like, and the same thing happens to me. It's like, if I go to sleep and then wake up in the morning and look at one of the PRs I put up. And I'm like, wait, this is terrible. You know what I mean? It's like you can, it's, it's just, uh, it is easy to trick yourself into like,

• 49:10Speaker 00

  Basically believing work that isn't at the same standard as what you would do before. And I don't think, I think we have, we don't really know what to do with that. And I'm kind of learning, getting better and like, I mean, this was, I think I'm doing a lot better job now than I was in like, probably, you know, like February or something where I was like, I was just like, you know, fully agent. And I'm like, now I'm like a little bit more like, um, but so my point is like, that was like a pair of powerful kind of like moment for me when someone on the team said that. And I was like, wow, like you're right. And I, and I see that in other people on the team's work too. Like, it's not just limited to me, but it's like, um, you know, it really does throw a lot of things on their head.

• 49:50Ryan Peterman?

  If you generate pure slop, that's, that's easy, but I think there's this gray area where you, you generate partially AI slop, or it's, you know, it's, it's acceptable, but it's not at the bar that you used to have. Do you, what are the tactics that you've used on the team that have worked for combating this kind of gray area AI slop?

• 50:12Speaker 00

  I'd like to get to a world, we're not here and I don't know if we'll ever get there. I'd like to get to a world where if you put up a PR and it's all green, then the odds of it getting merged are like extremely high. Right? Um, because that would mean that like you have automated verification for like most of what matters. And we have a lot of that in our projects and we've tried to add more over time and I think it's helpful. Like we have, like in TY especially, we have, um, tons of like benchmarks, like that run under Valgrind, like, um, through, through COD speed, like on every PR. So we get like, and that includes memory. So we benchmark like memory usage and, um,

• 50:57Speaker 00

  simulation time and wall time, like on every PR. And then we also have a really big suite of ecosystem, um, tests. So basically every time you put up a PR, we run before and after on like a bunch of projects in the ecosystem. And then we create this report of the diff of all the diagnostics, like the errors that got removed and added and everything. So we have like, over time, we've tried to do like more and more. Uh, these are important, honestly, even before we had agents, like these were like, we basically couldn't build without these things. But the point is like, I want to have like more automated verification and try to get better at that. And that includes things too, like,

• 51:38Speaker 00

  We basically assume now that anyone on the team that puts up a PR has already run that through Codex Review, like, probably several times. And that's basically an assumption. I mean, we could automate that process.

• 51:53Ryan Peterman?

  Codex Review is just an agent reviewing the code and double checking?

• 51:56Speaker 00

  Yeah, it's just running Codex and then just doing slash review.

• 51:58Ryan Peterman?

  I see.

• 51:58Speaker 00

  That's it, yeah. It's not that fancy. I mean, it's not, sorry, it's not that sophisticated, but it's just like, because now, it's like if a contributor puts up a PR, that's the first thing that we do.

• 52:08Speaker 00

  Because it tends to find good things. You know, I think, I think the things that I've, so, so like, basically, I think one bucket is like, how do you

• 52:21Speaker 00

  create more like automated systems that just help get things, make sure things are right? And that also includes things like trying to improve your like agents.md file over time. Like, if there are things, if there's feedback you're giving in a review that the agent's not respecting, try to find a way to help the agent learn that, even learn skills. Like, we have some shared skills on the team, stuff like that. Like, none of this stuff is very sophisticated, by the way. It's like, pretty simple. The other piece is like, how do I make sure that I put in the work to ensure that I'm creating a good PR? And so for me, that's like, I really should understand, it's, again, it sounds like a really not, it really sounds like a low bar,

• 53:06Speaker 00

  but I should understand like each line in the PR. I know, it's crazy.

• 53:14Speaker 00

  But also, I do, I do try to review each PR myself in the GitHub UI. This is something that's always found really helpful. Like, if you actually just like open up your PR and click files and read through it as if you were a reviewer, you tend to find things that you would miss if you were just looking at your local diff. I find that very useful. And then the other is trying to like encode skill, I guess this is a little bit more in the first category, but trying to encode skills, or trying to encode in skills, things I'm consistently getting wrong, that the agent is getting wrong. Like, I had, like, a recent example would be, I

• 53:56Speaker 00

  I found that I was often getting feedback on PRs that was of the form, what, you know, this condition here, there's like if statement, what cases is intended to catch? Because if I commented out, all the tests pass. And so I was like, okay, I should probably have a pass before I put up any PR where I have the agent like go through and check, like, are these conditions still relevant? Or are they left over from a priority factor or something else? So I don't know, I'm not, I'm still learning, but those are some of the things I've been doing. Yeah, it's again, I think it's like, it's a pretty hard time to be like building software. But I felt for a long time, or I had a fear that AI was going to make us

• 54:38Speaker 00

  more productive, but that programming would be like a lot less fun.

• 54:44Speaker 00

  Because I just like love, I just like love programming.

• 54:48Speaker 00

  And I was like, oh, now I'm gonna have to spend all my time like reviewing code and like prompting this like idiot agent to like, that's like keeps getting things wrong and like, but ultimately like is probably more productive.

• 55:03Speaker 00

  I actually feel way better about that right now than I did like a few months ago. And I don't, I don't exactly know why, like, I think, I think it's because, well, I think the agents getting better and the tooling getting better and me getting more comfortable with it is one factor. I think the other is I've grown to appreciate more of the, the kinds of things that like working with agents has unlocked. Like the cost of running an experiment is incredibly low. There are so many things I've wanted to try or like questions I've wanted to answer that I can now answer like almost instantly, like I, like a sort of a dumb example. We, UV and UV, everything is snapshot tested.

• 55:48Speaker 00

  So like basically all of our testing is effectively running UV and verifying the output. That's how we test like basically the entire program. And so that, and we have a lot of tests. That means we have a lot of test output. And the test output is, it actually ends up in the test files. So we have Rust files, like we have a File called like lock. R S that tests all our U V lock. It's all our U V lock tests and it's very, very long in part because it has all the lock output snapshotted in the test. And I was like, hmm, like, what if we stored the snapshots in separate files? Like, would that somehow make our like, compiles faster?

• 56:31Speaker 00

  Because then you don't have, technically, that's like Rust code. And so it's like, would that all disappear? And like, would that make like our builds faster or blah, blah, blah? And I'd always want to do that, but it sounded like, like to do that, to do that experiment as a human would be like extremely painful because you have to convert all of those tests. And I just had an agent do it in the background while I did a bunch of other things. I got a bunch of data on it. And the answer is no. But it's like, you know what I mean? Like, I, I mean, sorry, the answer is a little bit more nuanced. It actually does have a good impact. If, um, if you're just iterating on the snapshot outputs, you no longer have to recompile your program at all because this, this outputs are stored somewhere else. Anyway, that's the thing that makes a difference on you. But my point is, I'm just like running experiments like that, like all day, like, like trying things that were like, used to be hard,

• 57:16Speaker 00

  like, used to cost a lot to, um, to answer. Uh, the work of then going from that to like production, there's still like real work there. But so, you know, I think one piece is the tools and the agents getting better. The other is things that I just wouldn't have been able to do before that I can now do like incredibly easily. And then, um, and then the third is, I think I'm more and more realizing that like a lot of the value I get from building software is not,

• 57:45Speaker 00

  uh, is retained. Because some of it is like thinking hard about like the, not, it doesn't have to be typing out the code, but it's like thinking hard about like the layout of a data structure. A lot of it is merging a P R that closes a user issue. I get a lot of satisfaction from actually like, like fixing and improving something. It's not necessarily just from typing out the code. So I do feel for people a lot who

• 58:08Speaker 00

  Feel like they're losing something by like working with agents because I do feel that myself. But I feel better now than I did a few months ago about my like, like what it's like to work as a software engineer with agents.

• 58:23Ryan Peterman?

  You mentioned in the like, one of the performance optimization examples, you said, if you just unleash Codex, it kind of does this local optimizations and it's very good at that, but it's not great at kind of like some of the, I mean, today it's more human ingenuity of like system level optimizations. And it kind of reminded me of this tweet that you had, you said, it says, I'm slightly concerned by how much garbage I would be churning out if I was trying to use these tools without significant software engineering experience.

• 58:56Speaker 00

  Yeah, I remain concerned about that.

• 58:58Ryan Peterman?

  Yeah. It reminds me, and similar to what the Mitchell Hashimoto tweet that you said, it was like, there's a very big difference between Codex, go and do this versus, you know, you are like wielding it like this tool and you're kind of like prodding in the right direction.

• 59:16Speaker 00

  Yeah.

• 59:18Ryan Peterman?

  Yeah, I was curious your thoughts on that because it seemed like it went pretty viral and I think a lot of people are thinking about that.

• 59:24Speaker 00

  Like being a great software engineer is like

• 59:29Speaker 00

  like more like useful than ever. Like I don't like it's,

• 59:35Speaker 00

  you know, it's still the case that I think that the people on our team who are like the strongest like software have the strongest engineering skills are like the most effective, like even at using agents. And, you know, it's funny because there's a lot of talk about like token maxing. I don't know if you're familiar with this, right? Yeah. The idea of like, should you have a leaderboard? For example, this is getting tweeted about a lot. Companies that have token leaderboards and it's like, how do you like creates terrible incentives to just like use as much tokens as possible. And it's funny because we do, that does get tracked or sorry, there's not a leaderboard, but like token usage is something you can like look up, for example, you know, internally, I mean,

• 1:00:17Speaker 00

  and it's interesting to look at because I don't care like who on the team is using the most tokens like. And there are people on the team who are like, I'm actively trying not to care about that, like being where I am on the token leaderboard, which I think is like, great. Like, I don't care. Like, they don't have, like, they should just do, like, do a great job. And like, that's fine. I don't care if they use a lot of tokens or not. But it is interesting to look at the token leaderboard, because some of the people on the team who are most productive are like, using like, a lot of tokens. Right? It's like, like, there is like, a correlation. I'm not saying it's causal, but I'm saying, like, I do think that a lot of great engineers, like, are able to, like, use agents very effectively, and like, to hopefully to like, multiply their skills. So, I, I do think it would

• 1:01:02Speaker 00

  be like, really hard to be an early career software engineer right now. And I'm not spending that much time with early career engineers right now, just based on, like, at Astral, we're just a very small team, and we tended to hire very senior. But it's, it is sort of hard for me to think about, like, what, what would we, like, how would I learn, basically? Like, what would the iteration loop be? I would be like, learning

• 1:01:36Speaker 00

  from codecs, I guess, as opposed to, like, the other way around, basically? I mean, like, a lot of the times, I'm actually like, instructing codecs, right? And like, trying to correct it and priding a safeguard on it. Or Opus, or whatever you're using. And so, yeah, I think, I just don't know, like, where you would get, it would just be way too easy to fall prey to, like, a lot of the bad things that happen when you use agents.

• 1:02:08Ryan Peterman?

  You built this proof of concept in Rust, and it was really well received. But why did you start a company around it? And how did the, like, the raising go and all of that? What was the motivation there?

• 1:02:20Speaker 00

  Yeah, yeah. So I left Spring. I was actually convinced to leave by a friend who, a very close friend, who left Meta around the same time. And he was like, "We should start a company together." And I was like, "Okay, fine." I mean, it wasn't quite that simple. But I did, you know, I left and then we kind of went into the, you know, the quote-unquote "ID-maze" of like, "What do we want to build?" Like, we went into it not knowing what we wanted to build. And we spent a bunch of time exploring the Venn diagram of ideas of like, there were things he was interested in that I thought were not interesting and vice versa. And there was some stuff in the middle. And we spent time exploring the stuff in the middle.

• 1:03:04Speaker 00

  But then, in all my spare time, I was working on like developer tools. Because that's what I thought was really interesting. And it wasn't quite like a fit for him, basically. But around then, I was working on Ruff, I was working on a couple other projects that you can see in my GitHub that are like, I don't know, probably not as interesting. But they were like, I was like experimenting with like lots of different things at the time. Like, I was pretty interested in WebAssembly. I wrote a, sort of like a CICD toolkit in TypeScript. It's sort of like you wrote pipelines in TypeScript and it transpiled them to Docker. And I was like, "Oh, this could be cool." Like, I don't know, I was like building a lot of stuff.

• 1:03:45Speaker 00

  And at that point in time, I decided I wanted to start developing relationships with investors, but that I wasn't ready to raise money because I didn't know what I was actually going to build. And so my thinking was, "I'm going to try to like get connected to some people who like to invest in this kind of stuff, with an eye towards reaching back out to them in like three, six months. And being like, 'Hey, we had a great conversation, now I'm working on X.'" So that was my thinking. So I got connected to a couple of investors who invest in like software infrastructure and developer tools. And... I had some good conversations and then like the problem is things just can move really quickly. And so I didn't actually expect to like start raising so soon. But I effectively got convinced and at the same time, Ruff was growing a lot.

• 1:04:30Speaker 00

  And so I effectively got convinced that there was enough there to start a company, which is a very interesting process. I was like, I was like, I don't know quite know how this becomes a company. But I was effectively convinced that there was enough there to build a company, which is interesting.

• 1:04:44Ryan Peterman?

  By the investors?

• 1:04:45Speaker 00

  Yeah, yeah, yeah. Which I'm grateful for.

• 1:04:51Speaker 00

  The other thing I was convinced of was, which is funny, is if I hated it, I could stop in like six months. It's like if you decide it's not for you, you could like give the money back.

• 1:05:04Ryan Peterman?

  The investors said that.

• 1:05:05Speaker 00

  Yes, which I actually think was brilliant because like I was probably never going to do that. But it did make me feel like there was less pressure.

• 1:05:14Speaker 00

  So, sorry, very like weird roundabout details, but, but basically like I was like working on Ruff more like full time eventually. And then it was kind of like, I would say collaborative with like the potential early investors where it was like, we would just have like long conversations about like my ideas. And then it basically become clear that it was like, there's enough here. Like you and I like brought out kind of like a product roadmap, which honestly like a lot of it stayed true to like what we ended up building. What we've ended up building so far at least. And, and from there it basically became like, yeah, we want to fund you. Like here are the terms that we would do. And I was like, oh, wow. Okay. So I guess this is happening.

• 1:05:57Speaker 00

  So, so it was, it was just interesting and I had never done anything like this before. I mean, I was just, I was just, you know, an IC software engineer my whole career. And then I suddenly I was like starting a company. And it's funny because I actually think like the decision to like go all in and like start the company.

• 1:06:20Speaker 00

  Wasn't that stressful? I actually think a lot of the stress came later as the company started to succeed. Because at the beginning, I had, I kind of had nothing to lose. But then as the company started to succeed, you know, I was like, oh, wow, like, the company's kind of working. Like, what's going to happen from here? And I had a team of 20 people who were depending on me. And so, I don't know, just for me as a founder, it was like, I'm pretty like risk averse. But I actually found that starting the company was an easier, it was, it was an easier decision than maybe some of the stressful things that came later to navigate. But it happened very quickly and sort of unintentionally.

• 1:07:03Speaker 00

  And in, in a symbiotic way with investors, which was, which I hadn't really anticipated.

• 1:07:09Ryan Peterman?

  I didn't expect the, the investors. I mean, because I thought the founders hungry for the fundraising. They go, please fund me. But the investors were like,

• 1:07:19Speaker 00

  I think it can happen a million different ways.

• 1:07:20Ryan Peterman?

  Yeah.

• 1:07:21Speaker 00

  And like, actually, I mean, we did three fundraises. We did a C, a series A, and a series B. We actually never even announced the series A or the series B. They were, I mean, they were announced in the acquisition blog post, but we sort of complicated. It's like, we, we basically just never got around to it. And

• 1:07:43Ryan Peterman?

  Why didn't you guys publish it? Because isn't that good marketing for talent?

• 1:07:46Speaker 00

  It's good marketing, but basically like we, they're kept being like reasons that we like wanted to wait a little bit. And it's also like, to me, it always felt like a lot of work. And then I was like, it just never, honestly, it just never felt like the most important thing. Because we were like building a lot of stuff and like the company was going well. And I was like, I don't, we don't need the marketing. We did finally plan to announce it, but then we got bought. So it didn't happen. But, but each of those.

• 1:08:18Speaker 00

  Each of those fundraises was like preemptive, basically, like initiated by investors, which was, which is like, I mean, it's like fortunate because like the company was like going well and people wanted to invest. But it was also just like a funny position for me to be in, I guess. I think I, I think I was just always a little bit conservative and I wasn't super aggressive about trying to go out and raise money. And, but, but we were lucky enough to be building things that people really had a lot of confidence in, had a lot of belief in.

• 1:08:52Ryan Peterman?

  That probably gave you a lot of negotiating leverage because like one of the best positions you'd be in is that you, you're willing to walk away. You don't need it. And by definition, you didn't even come there to begin with. They came to you and they said, please take the money.

• 1:09:07Speaker 00

  That's true. Yeah, yeah. I wouldn't say I'm a great negotiator, but yeah, I guess at least I had a strong hand. But we were very lucky with investors. We had, we had amazing investors, like super supportive and really like aligned with how I wanted to build the company. Like never any conflict, never any like pressure to do anything specific, like just support. And maybe the only thing, maybe the only piece of consistent feedback is that I could have been more aggressive. And, but I don't know, it's like, just in terms of like scaling and growing and doing more. But I liked the way that we operated and yeah, we just had such a good, we had such good experiences with our investors too.

• 1:09:52Speaker 00

  So I felt lucky about that because it can easily go the other way.

• 1:09:55Ryan Peterman?

  I imagine if someone invests, there's a promise of future revenue, but how, how's this company make money? Because you guys give away your software for free, right?

• 1:10:05Speaker 00

  Yeah. We launched a commercial product in like August of last year. And it was sort of, it was called PYX. It was kind of like a hosted counterpart to UV. So like a lot of our, a lot of UV users will purchase like a private registry software instead of using the public registries, either for like security, or maybe they need to publish their own private artifacts, things like that. And we basically built our own private registry that had some first-class support for UV. And it let us solve a bunch of problems that we saw, you know, in our issue tracker around users using other solutions. Also let us build something really fast because we like vertically integrated the client and the server.

• 1:10:49Speaker 00

  There were like special things they could do to be like really fast and really easy to use and all that. And we spent a while selling that. And our revenue actually grew like pretty well. I mean, it wasn't like, I think in this era of AI, you're constantly hearing about fastest company to go to 100 million in revenue and things like that. It wasn't, it didn't look like that, but we were definitely able to sell this to like some big enterprises. And I mean, the cool thing was we had very good, because we built the open source and everyone was using our open source. We had a really good funnel, like, like we had a small number of extremely high-quality customers is the way I would put it.

• 1:11:31Speaker 00

  So the general idea for like how we wanted to make money was keep, like build open source and like the tooling. The tooling remains like free, open source, like entirely non-commercial. And then we build software that's kind of like the natural next thing you need when you're using our tooling. So if you're using UV, you, there are probably solved problem, there are likely problems we can solve for you with like a private registry. And then the funnel is like people using UV, they have these problems and then we sell them the registry. And the registry was meant to be like one piece of a platform of like a bunch of different things we sell, like a Python cloud. So that's what we were like building towards.

• 1:12:10Speaker 00

  I mean, part of the cool thing about the acquisition is we'll be able to take parts of that platform and make it basically freely available.

• 1:12:23Speaker 00

  Like we did, just as an example, a big part of that was.

• 1:12:30Speaker 00

  We did a lot of, like, in Python, you know, there's a big part of the community that uses Python with GPUs, like PyTorch, and then there's like a whole ecosystem around PyTorch of like, software that kind of like builds against PyTorch. And we, that's, for a variety of reasons, the ergonomics around that are not great. Like, it can be kind of hard to work with, hard to install, hard to get, hard to install the right version, depends on the GPU that you have, and like, what version of CUDA you have installed, and like, all this stuff. And so, we, we kind of tried to solve that in a, in a certain way, where we built our own distribution, you could think of it kind of like a Linux, in the sense of like a Linux distribution, like, we pre-built a lot of things like that, that all worked together, and made that

• 1:13:15Speaker 00

  available to customers. And so, now, we can actually take that and just make that like freely available to everyone, because we're no longer, we no longer are trying to build like an independent business, we're just trying to build great tools that like grow a broader ecosystem. So, I mean, more, more to come on that, but that's been kind of like one cool thing that I'm, I'm excited about.

• 1:13:34Ryan Peterman?

  As a first time founder, especially with the engineering background, was there anything that kind of, like a surprising learning, or something where you would share that with someone if they were an engineer and they were going down that founder path?

• 1:13:48Speaker 00

  There are so many different ways to do it. Like, I, I'm actually like not a huge fan of people giving startup advice in general, because so much of this industry is like survivorship bias of like, you could give the exact same advice to like, you know, like, basically, you can go to like two talks, and people give you like two of like incredibly successful founders give you like completely contradictory advice, and you're like, I don't really know what to do. And I, I tend to view that as like, you've got to figure out what's true to you. And so, you know, like, there's all these decisions you have to make, like in-person versus remote, right? Like, have a principle, and like, stick to it. Like, either one can be great. Like, we built the whole company early, it went super well, like, it was great. Do I think there are benefits to

• 1:14:33Speaker 00

  being in person? Of course. Would I have been able to hire the team that we put together if we were in person? Absolutely not. And so, like, you know, there are trade-offs to all these things, you just have to like be principled and figure out like what's, what resonates with you. You know, like, and there's like a million of those decisions that you have to think through. I feel fortunate in a lot of the ways I got to run the company. Like, I tried to spend like very minimal time fundraising and with investors. And I tried to optimize for investors I really trusted as opposed to trying to talk to a billion different investors and like pit them all against each other to get the best terms. Like, my, my goal was always like, find a partner that I really trust and get good terms and then move quickly, spend as little time on fundraising as possible.

• 1:15:15Speaker 00

  And so, you know, think about, think about what you care about and focus on that. There's no right answer to a lot of this stuff. I got a lot of advice to get a co-founder when I first started the company. And I basically ignored that. I mean, I like, well, sorry, I did ignore it because I didn't get a co-founder, I guess. But like, I, for me, it was kind of like, I kind of knew they were right, probably, but I didn't really have someone in mind. I didn't want to force it, and so I just didn't. And, you know, over time, I felt like it would have been helpful to have someone to, that's kind of like in, like in the trenches with you in the same way. But, you know, you find other ways to compensate for it.

• 1:16:00Speaker 00

  Like, I don't know, I lean on my wife a lot, probably more than I, you know, more than I should. Probably more open with my employees than I otherwise would be, which is, I think, which can be good or bad, but it's like, I share a lot with, you know, with, with the early team, like especially when I need advice on things. Try to build a network of like other founders, especially here in New York. I have like a pretty good group of, there's like six or seven of us. We try to get dinner twice a year. Doesn't sound like a lot, but everyone's busy. And, and that's been like a really, yeah, that's been like a really helpful like support system. So you just find other ways to compensate. But, yeah, it's, I don't know, there's no, I really don't, I really think there's not

• 1:16:43Speaker 00

  one right way to do it. Yeah, and it extends everywhere. I mean, like also, I don't know, everyone on X is talking about like, you know, like, should everyone in your company be working seven days a week, this kind of thing? And it's like.

• 1:17:01Speaker 00

  I don't know, like, we just built a very different company and we were able to do it. Like, that was not, I mean, I work, I work basically all the time. And that's a choice I made and that I'm like very happy with. But I don't expect people on the team to work all the time. And I want it to be a company where, you know, you can be highly, highly successful, working normal hours, but also where you're rewarded for, you know, your output and your performance. And I try to keep all those things in sync. But like, again, there's no one right, one way to do it. Sorry, there's no one right way to do it. Like, other companies, that is their culture and that's what they want to do. And like, sure, whatever, like, if that's what you sign up for, like, just make sure that you're honest about like what you're doing. So, I don't know, I just think, figure out what you care about.

• 1:17:46Speaker 00

  And in my opinion, don't focus too much on how you think you should be doing things. Think about how you want to be doing things and what actually fits with how you want to work.

• 1:17:55Ryan Peterman?

  What's your top book recommendation, whether technical or maybe something that helped you as a founder?

• 1:18:01Speaker 00

  I don't read a lot of like, technical material. I have definitely watched a few talks that were very influential to me, which is a little bit different.

• 1:18:10Ryan Peterman?

  What are those talks?

• 1:18:13Speaker 00

  The Zig creator, Andrew Kelly, had a really good talk about like data oriented design that like I learned a lot from. I mean, even apart from learning things, it sort of inspired me to like look at software quite differently. And so I think about that talk a lot. Yeah, things like that. It's really good. Yeah, it's about, well, I haven't watched it in a while, so I'm probably messed it up. But it's like, a lot of it's about basically design decisions they made in like the Zig compiler and, and just like thinking about like memory and allocation and all this stuff. And I was like, I watched that when I was pretty early in my career of like systems programming and I was like, wow, like, these people really care about what they're doing. And I was like, that's cool.

• 1:18:55Speaker 00

  Yeah.

• 1:18:56Ryan Peterman?

  Yeah, and the last question is if you could go back to the beginning of your career, like right when you graduated college, what advice would you give yourself, knowing what you know now?

• 1:19:07Speaker 00

  I don't know, it's so hard to give myself advice without feeling like I have all this hindsight bias. I mean, I think, I guess, I guess there are some things where I'd basically like tell myself that it is the right decision and I shouldn't worry so much, but it's hard to know how that would've played out, you know, if I did it a thousand times over. But for example, like, I think when I left school and I went to work at Khan Academy, part of me was definitely like, wow, should I be taking more of like a big tech job? And am I gonna really regret not having that experience? Or like, or like, I mean, the, like, like, the Khan Academy, like, pay was good, but it was like, the big tech pay was certainly like better. And I was seeing my friends get like huge bonuses and like all this stuff.

• 1:19:52Speaker 00

  And I was like, I felt sort of insecure about, should I be doing that? And I think it actually, you know, in the long term, really paid off. Like, I think, I mean, maybe it would've been great, but like, the experience I had was really what I wanted, and I thought I made that decision for the right reasons. So, I think some of these things are like, I don't know. Everything happens for a reason, you know? And it's like, have some faith in like what you're doing for the right, you know, for the, if you're making decisions based on principles, like, I think it can, it can work out.

• 1:20:24Ryan Peterman?

  When we started this conversation, you said, you know, the reason you got into what you got into is because you tried all these different ecosystems and you had a broader perspective. Imagine if you went to Google and you just kinda, I don't know, used Google's closed thing and then you might not have had the same insight.

• 1:20:43Speaker 00

  Yeah, and, you know, I, I, I think about this with Spring too, because Spring, like, I was there for four and a half years and, and then I left right as they decided to do like, you know, like a pivot, and then they ended up like joining Genentech. But we had set out to build like, This kind of crazy drug discovery company, we were focused on aging and it was all with like computer vision. It was like very, very ambitious what we were trying to do. Yeah, yeah. And then afterwards I was like, oh, like the company didn't like, it didn't have like the huge exit or like the huge, you know, we didn't achieve like all of our dreams. Like, did I waste all my time? And ultimately I was like, no, because I learned by being an early employee even and like seeing all

• 1:21:27Speaker 00

  these stages, I learned so much about how to run like my own company and also all the technical learnings that I rolled into my own company. So again, I feel bad, I feel weird giving this advice because it's like everything worked, everything has worked out really well. But I like, I feel like there are moments in time where I could sort of doubted some of my career decisions and then in hindsight, like you can only connect the dots looking backwards. Like it, like everything kind of added up to finding the thing I really love, which is like building tools and, and feeding into that.

• 1:21:55Ryan Peterman?

  Awesome. Well, thank you so much for your time, Charlie. I really appreciate it.

• 1:21:58Speaker 00

  Yeah, thank you. No, it was super fun.

• 1:22:00Ryan Peterman?

  Hey, thank you for watching this podcast. If you liked it and you want to see the show grow, please support with a comment or a like. Also, if you have any recommendations for people you want me to bring on, please drop a comment. Guests like Barbara Lizkoff, Mike Stonebreaker, Mark Booker, these were all people that I brought on because someone left a comment. On another note, aside from the podcast, I'm working on building the ergonomic keyboard that I wish existed. Here's a glance at the prototype. It's a split keyboard, so there's two sides. This is in the case. But yeah, we launched on Kickstarter and we hit our goal within eight hours of launching. I really appreciate it if you were one of the people who grabbed one of the early units. We're now working on the long journey of building the tooling now. And so if

• 1:22:44Ryan Peterman?

  you still want to pick one up, I've left the late pledges open on Kickstarter, so you can grab one there. I'll put a link in the description. Thank you again for watching the podcast and I'll see you in the next episode.