OpenAI Launches Codex: An Autonomous Programming Agent

OpenAI just launched Codex, a brand-new coding agent that can build features and fix bugs autonomously. We’ve been testing it at Every for a few days, and I’m impressed. I invited Alexander Embiricos, a member of the OpenAI product staff responsible for Codex, to demo the agent live on a special edition of AI & I. We talk through: - What Codex is and how it works. Codex’s UI allows developers to see the list of tasks the agent is working on, how many lines were changed for each, and the status of the PR. It’s built for the senior software engineer who wants to delegate and review tasks efficiently. - How OpenAI is thinking about agents. Codex is one piece of a unified super-assistant OpenAI wants to eventually build—an agent that helps users easily get things done by selecting the right tools for them behind the scenes. - Why an “abundance mindset” is best for interacting with agents. Codex is designed to allow users to delegate many tasks at once without getting caught up in the details. This lets you point an abundance of agents at a specific task, like a difficult bug—it’s worth it even if only one of them succeeds. - OpenAI’s vision for the future of programming. In the future developers will probably spend less time writing routine code and more time guiding agents, reviewing their work, and making strategy decisions. Programming will become more social, letting teams easily delegate multiple tasks at once, allowing people to focus on ideas and collaboration instead of routine coding. Timestamps: Introduction: 00:00:52 The product decisions behind Codex’s interface: 00:01:40 How Codex works under the hood: 00:06:20 Why you need an abundance mindset to work well with agents: 00:14:06 Setting Codex to work on a real task in “Ask” mode: 00:16:28 How OpenAI is thinking about designing agents: 00:18:54 The future of programming is social: 00:31:16 Reviewing Codex’s work live: 00:37:21 How the landscape of agents will evolve: 00:39:41

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Dan Shipper?

  Huge news. OpenAI is launching a new coding agent. It's called Codex. Codex is a web-based software engineering agent that's designed to work pretty much autonomously. So you can have it working on many different features and bugs in parallel while you watch. I've been using Codex for a couple of days and it's pretty great. So I invited Alexander Ambirikos, a member of the product staff at OpenAI, who's responsible for Codex, to come on the show and talk to us about it. We're going to demo Codex together and talk through all of the product decisions that led to its release. We walk through Codex screen by screen and he'll talk to us about how OpenAI is thinking about agents and what OpenAI's vision is for the future of programming. Let's jump in.

• 0:52Dan Shipper?

  Alexander, welcome to the show.

• 0:54Speaker 01

  Hey, thanks for having me.

• 0:55Dan Shipper?

  So for people who don't know, you are a member of the product team at OpenAI and you are one of the people responsible for building Codex, which is a new programming agent that is launching today. Well, we're recording, technically we're recording yesterday, but it's launching today, Friday. I'm super excited about it. I got to try it. I have a review coming out along with this podcast and I just wanted to go through it with you and understand like all the things that you put into it and all the ways that you're thinking about it. So thank you for coming on the show.

• 1:23Speaker 01

  Yeah, excited to be here and thank you for being an early tester.

• 1:26Dan Shipper?

  Awesome. So what I want to do is just like give people right off the bat a sense for like actually what it looks like and like we can sort of talk about it together. So, uh, so this is Codex, terminate processes. Great. This is Codex and it's a coding agent, right? So I can type in a task like, please replace the headline on the homepage with Codex is out now. And I can just press code. And one of the really cool things is it doesn't go into like an agent experience. Like it just adds it as a task. So it's like you're, it's basically like built to make me just. Create a bunch of tasks and then not look at them, more or less is what that's how I feel about it.

• 2:08Dan Shipper?

  So tell, tell me about like this screen and how you thought about this and, and why you did it like this.

• 2:15Speaker 01

  Yeah, so I'm so excited that you're screen sharing this and showing people so we get to know that about all the details. But you know, for anyone listening, basically Codex is, it's a cloud-based software engineer that can work on like many tasks in parallel. And you know, there's a ton of AI tools out there. I feel like programming with AI is completely different now than it was like two years ago. And this form factor that we're looking at is kind of the beginning of us thinking about what is it going to look like even like a year from now, today? And the big shift is, you know, today when people are accelerated, um, working with AI.

• 2:49Speaker 01

  A lot of it is like very, like a very tight feedback loop. It's like very collaborative, like a lot of like really good tab completion, right? And like really good chat. Um, and sort of where we see things going is, um, we have these really awesome reasoning models. And we want to give them time to think and we want to let them use tools safely. So they can do more stuff, for example, like running commands to execute tests to be more sure of their changes and then iterate. And so we basically want to give the agent its own computer to do work in. And so as we started thinking about that, uh, you know, you kind of realize, okay, this, this feels a bit like delegation.

• 3:26Speaker 01

  And I'm sure we'll get into this, but we tried a bunch of different form factors for what it would feel like to delegate to an agent, including like maybe the most obvious one, which is just like talking to chat GPT and having it do stuff. But the short version of it is what we realized is that engineers wanted like this really functional tool to be like very efficient in how they delegate. Um, and so the thing you see here, it's like, you know, it's very structured. You've got the list of tasks that you have. You can see how many lines were changed in each. You can see what the status of the PR is, which is ultimately the thing you're going for. Um, and so this ended up just being a thing that we iterated our way to.

• 4:01Speaker 01

  That we thought was the best first version to ship. But actually, like, you know, I would love it if we end up talking a bit about what it might look like in chat GPT as well, because we we definitely plan to bring it there.

• 4:09Dan Shipper?

  I would love to do that, and and I have a bunch of follow ups on that specific point about like, sort of the chat GPT versus what what this is point, but. Um, so to give people like a sense um for how this actually looks, so a couple, I did a couple of things yesterday, I actually like shipped a feature on one of our products that I've never. I've never coded in, um, like, I've never talked that repo. And I shipped a feature to prod uh yesterday with Kieran is the Kieran is the um GM of that product, and he's a really, really talented engineer who's also early, an early tester of of Codex, like he was. Watching me the whole time to make sure I didn't mess anything up, but it was really cool. Um, and basically like to show people what it looks like.

• 4:50Dan Shipper?

  Uh, we have this uh, we have this UI in um, wow, this is not great one.

• 5:02Speaker 01

  You're looking for the diff, right?

• 5:04Dan Shipper?

  Yeah, where is it?

• 5:06Speaker 01

  Um, you should be able to click, um, there you go, yeah, nice. By the way, this UI that Dan's showing us has been like editing, like changing massively like every single day as we race towards launch, so you know, thanks again for testing our lead.

• 5:17Dan Shipper?

  Yeah, this this may not be what you actually see, or depending on when you're listening to this, it definitely is not going to be what you see, but uh, for now. For people who are listening, basically, I clicked into one of the tasks and you can see basically for the feature I wanted to build, we have this, we have a um, a view that I wanted to create a a persisted collapsed state. We wanted to be able to like collapse a particular view, and I just said, I just said, do that, basically. Um, and you can kind of see a uh, log of all the things it thought while it was doing the um, doing the task, and then it gives me like a summary of. Um, what it did, I can see a diff, um, and I can also just push that right to GitHub, and one of the things that um, that I noticed is the the summary is very concise.

• 6:02Dan Shipper?

  And in talking to Kieran about the code that it generates, um, the code seems very um, minimal or like terse. Uh, can you tell me a little bit about how you thought about making this thing code and and talk about its um, talk about what it makes?

• 6:18Speaker 01

  Yeah. It's a great question. So Codex, the agent runs on a custom model that we trained just for this product called Codex One. Codex One is a version of O3 that's optimized for real-world software engineering, but not only optimized for real-world software engineering, but also optimized for this form factor, which is, hey, it's going to go off and do a bunch of work and then you're going to get a diff back. And, you know, once you're using this a lot, actually, you're going to be getting many diffs today back. And, you know, I'm sure this thing we talk about a lot with even human software engineers, like, would you rather review, like, 10, 100-line diffs, or 20, 50-line diffs, or, you know, 1,000-line PR? Right? Definitely, the thing is, you don't want to review the

• 6:58Speaker 01

  1,000-line PR, right? And so, and as we trained this model, we both wanted to make sure it was great at coding. And so the model, like, slightly outperforms O3, actually, on, like, evals, like, Sweetbench. But more importantly, actually, we wanted it to produce, like, code that was really mergeable, and then talk about the work that it did, in a way that was, like, actually really reviewable for the human reviewer. So, we put a lot of effort into style here, you know, making sure it's not doing things like adding extraneous comments, making sure that the style that it's using is actually like the style in your codebase, and not like its own style that it has and thinks is correct. Okay, and so getting to your point about the PR descriptions, you know, early versions we've had of tools like this, or models that did this, would tell you everything about what

• 7:43Speaker 01

  they did. And, you know, just actually to give you a bit of the story, like, in our UI, we would, like, show you kind of the model's thoughts first, and then we would show the diff underneath. And whenever, like, a full-time software engineer was trying to use this a lot, they were always like, "Man, I just want to see the diff. Like, I don't want to read the description until I see the diff. And if the diff is good, then maybe I'll look at the description." And so we kind of ended up realizing, like, "Hey, there's actually, like, kind of a spec for what a good PR description looks like." And that's, like, pretty concise, and only maybe explaining, like, at a high level what it is, the non-obvious parts. Like, you definitely don't want to duplicate content that's in the diff. Then the other thing we realized is that, because it didn't run on your computer, you didn't necessarily see each iterative step. And so you really want to know, how validated is this change? And how did the model try to validate the

• 8:28Speaker 01

  change? And so, you can see here on the left side, the model's also describing the testing that it ran. And here we put a lot of effort into, first of all, just making sure that it's like never hallucinating, but also making sure that it tells you when tests pass and when tests failed. It's like super important in a concise way. And then lastly, perhaps the thing I'm most excited about, like if you mouse over that little like terminal icon next to the testing.

• 8:52Speaker 01

  Yeah, so they sort of like the last thing on the third line there. It's a little icon, it's like a carrot. Yeah. So we'll actually, the model learns how to cite its output. And so it's not just saying like, hey, here's a test I ran. You know, in the case of tests that passed, which it didn't here, you know, it's not saying like, hey, the test passed. Believe me, it's saying, hey, the test passed. Here's like a deterministic citation that we did to actually pull apart from like the log out. So you can review it yourself and be super sure. And in this case, the test it tried to run failed and it's kind of telling you this because maybe you would want to run it yourself or like upgrade the environment it's operating in to so that those tests can be run. And it's giving you the output there. So you can like go and review it and improve it.

• 9:35Dan Shipper?

  That's really interesting. This, this particular repo, so I actually, this one we're looking at, I, I, I kicked off the, the collapse state task in the wrong repo at first. So this is the wrong repo. This is an internal. This is an internal app we built and so it does not have any tests. And I was just kind of curious what it would do if, if I gave it a task that was totally not, not possible to accomplish. And it found a section, something else to collapse. So that's kind of just like an interesting little detail, but yeah, okay, that, that makes a lot of sense. What things that I notice about this that feels different from like a dev in, for example, is I don't see this, it has an environment that it uses that it sets everything up and it runs a test and all that kind of stuff.

• 10:20Dan Shipper?

  But it doesn't have a browser, for example, so it's not going and like logging in to the app to check if it's working or not. Like, tell me about that decision. Why'd you do it that way?

• 10:30Speaker 01

  Yeah, I would say that's more of a just like sequencing decision rather than like a provision decision. Like ultimately, kind of like the thing that we view ourselves as building like super long term is just like one super assistant. And it has a bunch of tools and you don't even have to decide, you know, if you even want to use code mode, or if you want to use like, I don't know, newsletter publisher mode, you just have one assistant, ChatGPT, and you can just ask it for stuff and it'll use the appropriate tools. And it'll also just like answer quickly if that's the right thing, or it'll like do work if that's the right thing. So, long term, that's, that's where we're trying to get to, just like one thing with all the tools. But what we're starting with is kind of like, hey, we want to ship a research preview, we want to like iteratively deploy this out in like small steps as capabilities grow. And we're going to start like the smallest,

• 11:14Speaker 01

  most constrained thing possible. So, for instance, it doesn't take multimodal inputs yet. We'd obviously love to add that. It doesn't have a browser, that would obviously be useful for validating front-end changes. It doesn't even have network access. So, the way that it runs is, so when, when you kick off a task, we set up a container, pull the repo into that container, and then with network access, we run some set of commands that like you wrote. So it's not the agent running those commands, it's like you writing those commands. And so that can pull in like, you know, any dependencies like npm install or, or you know, whatever you need. And then actually, we cut off internet access. And I think we updated the logs recently to show that. So if you were to like click into logs and scroll to the top, you would see like no internet access all the way at the

• 11:56Speaker 01

  top. And then you would see no internet access. Yeah. So like, we ran the environment setup that you put in there, and then we turned off internet and then the agent runs. And that's just for absolute maximum safety. You know, I haven't actually ever seen it happen, but you know, there's like a theoretical risk or a tail risk that there could be like some exfiltration, maybe the agent wants to go to Stack Overflow and ask a question about some code and it pasted it, right? That's a theoretical scenario. And so we're starting really small in scope, and then over time, we'll start like adding more and more capabilities.

• 12:26Dan Shipper?

  Got it. And that actually brings me to my next thing, which is, which is one of my gripes about this. Which is, it's not like, the fact that it's not in ChatGPT means that it feels like a very one-shot type user experience, which to me makes it feel very like, it's definitely for senior engineers where you can kind of like see the entire thing you need to build in your head and you just type it out and then it's done. Whereas for me, for example, like, I might want to like go back and forth and chat with, chat with it to like figure out, okay, like, what is it that I'm actually trying to build?

• 13:01Dan Shipper?

  Or in this case, in this particular session, I asked it to build, I asked it to build a feature and then I realized after I built it that I needed to add something else. So I asked it a follow-up. And in the follow-up case, it didn't work quite as well as the like one-off, one-shot thing. So tell me about that.

• 13:20Speaker 01

  Yeah, it's, it's really interesting. Like delegating to agents requires like a bunch of like mindset updates that actually I think will take time for people to learn how to get the most use out of them. And will like, I think, end up temporarily in a state where we kind of have tools that feel more like you're delegating and tools that feel more like you're collaborating. And then our goal at least is bring those together. So you don't think about it. Again, like I think about like our future super assistant. I don't think about if I'm delegating, I just ask it for stuff and it just does stuff.

• 13:49Speaker 01

  But you know, people are pretty used to working like collaboratively with tools locally on their computer right now. And in fact, we have the codec CLI, which is exactly that, right? Like, so you can do a back and forth CLI, have it execute some commands, it'll ask you for permission because it's, it's on your computer, but you can kind of sit with it and kind of pair program with the CLI.

• 14:06Speaker 01

  So a little bit about the mind shift, mindset shift that we've seen people who get a lot of value out of, out of codecs, like the agent that you delegate to. I'll just share for, because it's, you know, kind of interesting here. One of them is like an abundance mindset. So something that we've seen a lot of people who are using this a ton do is they just fire off many, many, many tasks without thinking actually too hard about whether it's going to work or, you know, whether it's like perfectly described or not. And whenever you need to do something new, you just kind of fire off a new task. And so, for example, a really common use case is like on-call triage. It's like, maybe you don't know if, what the bug is. You, you know, you were talking about how like, oh, like I've noticed it works best if I can kind of see through the full arc of the feature. Like you don't need to know that. You just fire it.

• 14:52Speaker 01

  And then sometimes it's like, hey, here's like the exact fix. And you're just like, wow, like amazing. Not only did I save myself time, but that that bug was fixed faster than I could have ever gotten to it otherwise. Other times, maybe it just like provides a draft and then you have to land it in your own time. And and other times it like just maybe finds where the bug is or helps you just reason about it. And so it's kind of like this idea of like, this is a service to me that accelerates me to developer. And I'm going to use it without necessarily always knowing if it's going to like provide a mergeable PR or not. That's like on call triage as an example, another like very analogous place we see people use it is like.

• 15:30Speaker 01

  In every morning, like people will kind of like think about what they want to do. And then like they're at home, they'll just be on their phone, like fire off some tasks. We have mobile support coming soon, but we already support mobile web and a mobile app on iOS. We'll support soon. So they'll just be like, this is what I want to do, fire off some stuff, then get to their desk. And like, look at what they want to use. So it is, it is quite one shoty in the way people use it, like those two examples I gave you. So I think you're right about that. And so we've started poking at this, like, does it need to be one shoty? And so you saw in the UI, there's, when you send a task, you can choose if it's going to be in ask mode or code mode. And code mode is definitely the thing we've spent the most time on. But we're starting to experiment with making ask mode faster.

• 16:16Speaker 01

  And so like, you know, one way that this could evolve potentially is where when you want to do what I just described, like fire off like a one shot or a few shot thing, you basically ask code mode. But if you want to be a little bit more exploratory, you can try ask mode. And so I don't know if you tried this, but one thing you could do is ask for some recommendations using ask mode. And that should be relatively fast. It'll probably finish during like pretty quickly during this call. And then we could try kicking off some of those tasks if you want.

• 16:44Dan Shipper?

  Okay, so what recommendations should I ask for?

• 16:46Speaker 01

  Yeah, like, I don't know, is there like a class or that you find very complicated that you would love to refactor? In some code somewhere?

• 16:53Dan Shipper?

  Wish that I was familiar enough with this code base to tell you.

• 16:57Speaker 01

  Okay, what about, what about just suggest, let's run two prompts. And again, this is abundance mindset. Let's run one like suggest places to add documentation. That's a safe one. Actually, a lot of people use it for documentation and it's really good at documentation. And then let's do another one, which is like. Find some bugs and suggest some tasks to improve the bugs or to fix the bugs.

• 17:21Dan Shipper?

  Cool. I think that this is really interesting. I want to pause here for a sec, like the sort of abundance mindset that you're talking about where, um, yeah, you can just like direct intelligence at a problem and you don't have to like have it like that well formed. And if it doesn't work, it's fine. And that's just a different way of thinking about how to how to program or how to manage a resource that I think is really interesting. And yeah, like if if there's some sort of like priority bug that's going on, like, why not send off like three different workers to figure out while you try it yourself? Because if they fix it, like, that's great. And I think, obviously, there's always a time and a place for understanding the full system and and maybe stepping through it step by step. But it's like, it's not every

• 18:04Dan Shipper?

  time. And that's, that's just an interesting shift. It's a, it's a different skill, it seems like.

• 18:09Speaker 01

  Yeah, it's like, I think, I figure you probably use chat GPT deep research on occasion. Right? And like, if I think about like how some of my habits have changed, like I'm much, I deep research a lot of things that I wouldn't have spent, like, I don't know how long it would take me to do a deep research, but probably quite a long time, like hours. And now I like read sort of deep researchers on custom queries that I have like way more often. It kind of lets me be more curious in a way. Um, yeah, like, you know, many things a day, basically. And I think there's kind of a similar thing here with agents in general.

• 18:40Dan Shipper?

  Yeah, it's like you find you have more questions to ask when it's really easy to ask a question to get a really good answer, you know? I find that too. I'm just, I've learned way more because of O3 and deep research and just like AI in general. What about like, I feel like there's a tension here that I sense in this product and I've sensed myself in trying to build these things. There's a sort of tension between making something good for a particular situation, like, for example, like, you know, uh, this is great for picking off our bug or feature. But you kind of get this like loss of flexibility, where it's not, for example, it's not nearly as good at just doing follow up questions and chat GPT is just great at

• 19:21Dan Shipper?

  that. And so how do you think about that when you're designing something like this, this sort trade off between specializing and the brittleness or loss of flexibility that you get? And how do you manage that?

• 19:33Speaker 01

  Yeah, I think the long-term goal, you know, it's the, I'm sure you hear this a lot, but you know, it's the G and AGI. It's to build something pretty general and

• 19:43Speaker 01

  a lot of specialization, at least when it comes to like building UI, is a bunch of design decisions that we have to make. But if you're just talking to an entity, it can make a bunch of decisions throughout the right behavior and just kind of be general for you, right? And so

• 19:56Speaker 01

  kind of the, where I see things like this going and maybe just like working with agents in general is you can have a conversation with an agent and that is, that should be an incredibly general thing, like I was mentioning earlier, it shouldn't only be for coding. It should just be an agent and it should be able to work across any modality, whether you're, you know, in your car or at your desk. So that thing stays general and it should be able to like do all the things it needs using tools, et cetera.

• 20:22Speaker 01

  But, you know, when we're professionals doing a specific thing, like right now, we're recording this podcast and this is like some pretty specialized UI. But it's really good. Like it would be quite annoying to have to like use a chat interface to mute myself or just my volume here, right? And so kind of the way I think about it is there can be nearly an infinite number of like bespoke, possibly even AI-generated like UIs that like serve the specific purpose of the user who knows kind of like exactly what they want. And so like for me, like the most interesting thing about this Codex Research preview isn't actually the UI, although don't get me wrong. We've spent a little, we're like rewriting it massively as we speak, but the most interesting thing is actually

• 21:01Speaker 01

  training a model that's designed to work more independently and to make the most of that asynchronousity and its own compute environment. And then building the actual compute environments that we can scale up and that like let people set up the agent to have the tools that it needs. So, once you have those two things, right, the model and the compute environments, we can then start bringing that, that UI to everywhere. So, you know, I would, like I mentioned, I would love for that to be more readily available, just like if you're talking to chat GPT.

• 21:29Speaker 01

  I would love for that to be available also closer to where developers spend their time, right? So if you spend a lot of time in terminal, then like the Codex CLI would be a perfect place for you to be able to like. You know, delegate to that agent. Similarly, if you spend time in your editor or in your CI or in your issue tracker, like all of these places should be places where the agent is just sort of like ubiquitously available for you to delegate to it.

• 21:49Dan Shipper?

  Well, so, so let me just push on that a little bit. So, uh, so in a, in a perfect future, basically I'm talking to chat GPT and chat GPT is the one that's like calling Codex and telling it to, okay, go do what, do this thing and then handling the follow ups to be like, actually, I want you to do it a little bit better. And I'm maybe not even talking directly to it. Chat GPT is almost like the middle manager.

• 22:12Speaker 01

  Yeah, that, that, I think that is one possible future. Um, and I think that is a future that might, you might use if you're like on the go, like on your phone, not at your desk, or, you know, let's say I, I don't spend like a ton of time like on marketing, but I want to like interact with some like marketing tools, like maybe I'd go through chat GPT because I don't have mastery of the, the actual like underlying systems. But then if I'm like a full time person who spends a lot of time, like coding or, you know, whatever thing that it is that I'm doing, I probably have a lot of like mastery that I've learned over time and that also brings me great joy in my life, hopefully, to spend time with these tools. And so then I would like go straight to the tool. And the AI should be like just available in that tool.

• 22:56Dan Shipper?

  Yeah. So I totally get that, especially for the, for like the interface layer and like having specialized interfaces for specialized tasks makes total sense. What about at the model layer? Like, obviously the goal is to have a perfectly general model that no matter what you ask it, it can like just fulfill the request. But also it seems like, um, for example, when you do, you know, you did reinforcement learning on this model to make a better, more sort of like senior software engineer type personality. But even within that, like this model clearly has a different personality than Claude Code, um, or it's like the underlying model for Claude Code. And so like, how do you think about that? There's, oh, there's always, even if you're trying to be as general as possible, there's always these sorts of like trade offs between making it good

• 23:38Dan Shipper?

  for that thing and making it generally, you know, it's, it's, it's flexibility in other areas or its personality.

• 23:45Speaker 01

  Totally. Yeah, I mean, I feel like at OpenAI, at least, we've kind of done this like a few times now where we've shipped ChatGPT, a ton of people use it. And then we get some feedback from a specific audience we care about. So like with GPT 4.1 developers. And we just, you know, spend a bunch of time with them. This is a different team, but you know, they spend a bunch of time like talking with developers, understanding their feedback. And kind of like creating like a different set of evals that we realized we like wanted to improve on, we cared about. And so then we had this decision, right? Like, do we try to, you know, get a bunch of like improvements into the mainline model?

• 24:25Speaker 01

  Which takes time and has a bunch of trade offs, or do we kind of want to speedrun it on the side, ship a separate model? And then like reintegrate some of those changes. And I think our belief as a company is that like there are going to be times where we kind of want to speedrun something. So like often just maybe even for learning purposes. Just to like have the flexibility, like really go deep with the set of customers and like the kind of like generate the evals that they have in their heads.

• 24:49Speaker 01

  And then from there, like ship it, validate, and then take those configs or the, you know, those changes that we're making to the model and remainline them. And when it like, it seems like whenever we can do that, the whole overall system gets better and maybe even more powerful than a custom side thing that we've done.

• 25:04Dan Shipper?

  That's really.

• 25:05Speaker 01

  Yeah, go ahead.

• 25:07Dan Shipper?

  And what like what that makes me think about, because one of the, one of my other like little gripes with this is it reminds me kind of of operator. Where like this cool thing, but it's like separate from ChatGPT and like it's. I don't think that they've, you guys have released an operator update since the first one. And so it just feels like it's like hanging out there and and not necessarily getting improved or not, I'm not using it as much as I am using like sort of base ChatGPT. And so is that kind of the like one version of of the vision is obviously that Codex stays stays as a separate thing. But then you integrate all the learnings back into like the base O3 that I use inside of ChatGPT. And that's how everything coexists together. Or how do you kind of make sure that you're not building like this Balkanized thing that's

• 25:51Dan Shipper?

  kind of forgotten about in, you know, three months or whatever?

• 25:54Speaker 01

  Totally. Yeah, man, I have so many, so many fun thoughts here. One thing I'll say is that, uh, yeah, we have a lot of. So like these capabilities that we've we've talked about like being kind of like different and separate like are coming together and like that's the thing we think a lot about and it's I'm very jazzed about it. So excited to share more as that happens, but let's take let's take Codex.

• 26:21Speaker 01

  The uh in my mind here, there's there's kind of like two interesting things. The first is like what we said, which is like, hey, anything we learn as we like build this Codex model is like really helpful to us just as we think about agents in general and like, you know, like this idea of like, say, like citations, right? Or like describing your work really efficiently for the user to read after you've done a lot of work. Like tuning the steerability so that it like kind of makes the right amount of assumptions, which I'm not saying we nailed perfectly, but like that's a that's a theme like that's a generalizable thing for agents and like something that's going to be useful in general. Similarly, and I don't know if you have this in your briefing, but we are planning to release an update,

• 27:05Speaker 01

  an updated small model for use in the Codex CLI.

• 27:09Speaker 01

  And, you know, there's a lot of, I mean, it's very pretty obvious, right? There's a lot of overlap between these two models. And then actually, you know, a lot of the learnings that we have from like both of those Codex models, like we plan to use even in like mainline models, right? So there's a lot of that generalizing that comes back. I feel like your question also, though, is a bit about user experience and like having to go to operator, right? Or having to go to Codex. And so like, you know, I think

• 27:31Speaker 01

  Codex is kind of interesting because it's maybe one of the first products or features we've built into ChatGPT where we have a professional audience that like spends a lot of time in their tools every day. You know, like a lot of how like ChatGPT overall is quite a general tool. And I think this is like maybe some of the first time that we're getting like fairly specialized. And so my view here is that like, we're not exactly sure what this should look like. I I think like ChatGPT a year from now, where we have more of this will look fairly different from ChatGPT today. But the goals we'll have in mind will kind of be these things that I've been saying, which is like, I think we're trying to build a generalized assistant and we cannot lose sight of that. But we should give ourselves the room to give users the like power UI that they want.

• 28:14Speaker 01

  So for instance, just to give you an example, going back to your opening question about the UI. Yeah, it's a list, right? And it's a list and it has features like archive. Like, why do we have that? Well, it turns out that people internally who use Codex use it a ton every day. And when they're using it and it's just like mixed in with chat, it's a little hard to find your things. And then, you know, there's kind of this workflow you develop where you're basically you're like, hey, I have an idea. Like, for example, like, so I'm on the product team. I mostly do small changes. People make fun of me affectionately at work. But, you know, this string should be different or there's this bug here, right? And if the bug's really hard to fix, I'm like, that's vegetables. I'm not going to eat that. Oh, you know, I'll put it in the.

• 28:57Dan Shipper?

  You're small to nudge here.

• 28:58Speaker 01

  Yeah. But if it's like an easy bug, you know, I'll wham it. I'll explain what the word wham is later, but that's our code name for the project. And so, you know, I'm whamming a bunch of stuff and then I don't want to track it separately. Like, I just want to have this list of things and then like when they're done, my workflow is basically look at each one and make a decision on if it's mergeable or if I want to like pull it open to my computer, test it, maybe change it. Or if I'm just like, this is over, like, I don't have time to do this. And so that workflow really needs that UI of like, what did you open a PR? What is the status of that PR? And the ability to archive so that I'm only looking at the things that I my current tasks, basically. So that's just an example of like the power user UI that.

• 29:37Dan Shipper?

  That makes a lot of sense.

• 29:38Speaker 01

  Yeah, we feel is important. Yeah.

• 29:39Dan Shipper?

  Yeah, I feel like chat is only appropriate when it's incredibly underspecified what you actually want to be doing. Whereas as soon as you know exactly what you should be doing, like chat is not the right, really the right interface.

• 29:51Speaker 01

  I'll add one thing to that, which is actually the other cool thing about chat is that it unlocks a lot of like emergent behaviors that are just mind blowing, right? So one, I like, I'm obviously really excited to bring to bring us back into chat as well. But, you know, one experience that I had when we'd prototyped in chat was.

• 30:10Speaker 01

  I had a, I had asked for a change on, on the front end. And actually, when I was getting started on this project, I had, I had spent, I've spent like five years, including before OpenAI, just like writing native macOS apps.

• 30:23Speaker 01

  And so, you know, I'm not super familiar with the latest, you know, tailwind, whatever stuff. So I asked for this change and I, there are a bunch of like classes were added in line. I'm like, I don't even, what is this class? So I asked, but we were in chat. So I just replied with, what is this? Can you explain it to me? And because we were in chat and the context of like what was changed was just like in line and chat, chat replied quickly, which was the thing you were saying is one of your gripes that this doesn't reply as quickly, right? Chat replied quickly and it asked me, do you want me to generate an image for you? And I was just mind blown and I said, absolutely. And then it generated an image and, you know, I was very happy. And so I think, I think it's important that we, we always like make sure that like the overall assistant has access to these tools so we can experience things like this and then like work those into the more functional APIs as well.

• 31:08Dan Shipper?

  How do you feel about, and with your own personal experience and what you see internally or I don't know, other beta customers or alpha customers that are using this, like how this changes what it is to be a developer, what the developer experience is? And I'll give you an example. In, in my testing over the last couple of days with Kiran, again, he's a GM for Cora. He's, you know, super technical. I'm like technical, but like in a, I can just build the stuff to make it work kind of way. And what was really interesting is we just sat together on a video call. We were both screen sharing. We both had a bunch of agents up and we were just like talking about how we wanted to make the product better, Cora better. And then he would just say, okay, I want you to like, you know, build this little thing. And I, and I'd be off being like, I want to fix this little bug.

• 31:52Dan Shipper?

  And both of us have like these different agents, all working on different features and bugs all at the same time. And we're just chatting while we do this. Like we have the brain space to be able to like talk about what's going on. You know, an agent would come back, then we'd like kind of look at it together and be like, is this good? Is this not good? And then either edit it or merge it. And it was just like a very different, it was like social coding in this like weird way where we're just like chatting together and work is getting done, which I've never really experienced before. And feels like a new model for building things that was not really possible previously. And I'm kind of curious if that resonates with your experience or if there are other things that you're noticing about what it, how this changes what it is to be a developer, what developing looks like.

• 32:37Speaker 01

  Yeah, that sounds pretty fun. I even say even like before we had any, you know, like before Codex, I've already started feeling like coding is like more social. It's like, even if it's just me coding with an LLM.

• 32:52Speaker 01

  And yeah, you know, like that startup I was telling, I mentioned that I was working on, like I built the first prototype for one of the things that became it in the airplane without Wi-Fi. And I would never do that today. Like if there was no Wi-Fi, it's just over. Like I'm gonna watch a movie. I just can't do it. And you know, I do think this is, we're kind of on the precipice of a never like change of similar magnitude. So the vision for like what it should feel like to be a developer, if we get things right, and maybe what it feels like in general to like just be a knowledge worker. The vision in my mind is you should be able to do the work that you want to do. And maybe that's because it's work that's difficult to automate, which is a lot of work.

• 33:35Speaker 01

  Maybe it's work that's like very ambiguous, that requires a lot of like complex decisioning. Or maybe work that you just want to do for fun, or you want to be creative. And like, that's the stuff you do. And I think it's really important that we invest in making sure that when you're doing the work yourself, you're maximally accelerated, right? So like, things like the Codex CLI that you can pair with, I think are critical for us to invest in. But then the lion's share of kind of like known work should just be done by agents that you're delegating to. And so I think this kind of like shifts where we spend our time. And maybe we're spending more time in planning or like thinking about what to do, more time in design.

• 34:16Speaker 01

  And then more time in like validation as well. Now, I do think it's interesting, like, you know, right now, I think at least for the next couple of months, we are going to also be spending time thinking about how to set up the environment for agents. And so it's not only like, I think we're only going to be spending our time like being creative. I think there's also going to be a lot of like, it's a bit like becoming a manager and you're thinking about, I have this team and how do I enable my team to be productive?

• 34:45Dan Shipper?

  I mean, and it creates, it like creates new problems that you still need to solve. So like a really interesting example that I'm noticing is,

• 34:54Dan Shipper?

  You can implement so many features that the features start to crowd in on each other. And the product is just feels like less well thought through because like you have so much throughput. You just go like do this little thing and it just takes 10 minutes. And then you've done that like 50 times and you're like, this product is like way too bloated. And no one's like step back and been like, how does all this stuff fit together?

• 35:14Dan Shipper?

  And that I think is like, we need new product management or like hygiene practices for. We have so much capacity to build stuff or fix bugs. Like, how do we deal with the the overproduction of of features almost, you know, which is a new problem?

• 35:33Speaker 01

  Well, I have some takes on this.

• 35:36Speaker 01

  So I don't think this applies to everyone, but at least I'm happy to hear what you're saying. And that I think like taste is like fun to exercise.

• 35:43Dan Shipper?

  Yeah.

• 35:44Speaker 01

  And so like, and also like great fun for humans, you know? So it's like, I'm excited like, oh, we can produce too many features. And now we have to like choose which ones. I'm like, this is a great future, like wow, product person's dream. But obviously I know that doesn't apply to everyone, but for me that that sounds awesome. The other thing that I like hope that happens is that

• 36:03Speaker 01

  we actually just build many more software products. And I don't mean like many more features, but like many more apps. And this again is this is just like my personal opinion, but I really like these like small, like beautifully crafted apps that have like a small number of features. And you know, and sort of almost the more niche, the use case, like kind of the more fun it is to use them for a little bit. So like, you know, an example in my mind that I've like poked at, but haven't really spent time on is like maybe like a texting app, but for just me and my wife only, you know? Like, that would just be kind of a slightly ridiculous thing to consider doing while you have a full time job like a while ago. But like, I would love to have that app and like, it doesn't have to have many features, but it's just for me. Right? And I think that like, as software development gets like more and more accelerated, like my hope is that because it's like,

• 36:48Speaker 01

  we can produce more of it, we end up actually using much more, but in much more bespoke ways for people.

• 36:53Dan Shipper?

  That makes sense. Yeah. And I think you can see that in the history of other art forms like, you know, photography or illustration where 150 years ago, like you had to do like woodcut blocks. And then we figured out how to like do mass reproduction of photos and, and it's just, there's a whole, yeah, it's, it's a similar kind of thing where now we all have a camera in our pocket or we can do studio G like generations or whatever, you know?

• 37:15Speaker 01

  Well, yeah, now we're recording a podcast together, right? Like for on a like specific topic. Yeah.

• 37:20Dan Shipper?

  Exactly. Um, so I want to go back. Our, our ask query came back a while ago. So I'm gonna make a get about that. So, so we asked it to suggest documentation. And so it gave me some interesting things. It said the Rebo's guidelines, I'm sorry, documenting Ruby classes with Yard and seeing those controllers that apprehensive JS doc blocks, C development guide, and then just found some places where Yard documentation is lacking.

• 37:48Speaker 01

  Yeah. So this is real quick. This is close. This is not what I was hoping you would see. We clearly have some prompt tuning to do between now and tomorrow. Because let's, can we check out the other one real quick? Because so this is good. But what, what I would love to see in there is that it actually stubs out tasks for you. Oh, there we go. This one did it. And I think that's because we said suggest some tasks. Like we put those words in the prompt. And so it's kind of similar, right? It's answering kind of the question, but it's giving you these buttons you can click to like actually do the thing if you want to do it.

• 38:19Dan Shipper?

  Interesting. Okay. So I'm seeing, so we said, find some bugs and suggest some tasks to fix the bugs. And it says issue number one, and it gives me some relevant code. And then it has a play button on the code, which if I click it, just does the changes that it's, that it's suggesting.

• 38:35Speaker 01

  Exactly.

• 38:37Dan Shipper?

  That's really.

• 38:37Speaker 01

  Right. And so this is us beginning to like, sort of play with this, this, this gripe that you have, right? Of like the sort of the interaction loop. And you know, the us to begin to think like, yeah, how can we make it so that it's like, if you want to be more quickly like sort of collaborating with the model to figure out what to do, you can, you can do it. And also, in a way, how can we encourage it? How can we, you know, this is going back to like, a lot of what we're spending our time thinking about is like, this future where agents are writing a lot of code, like, how do we make that work well?

• 39:06Speaker 01

  You probably would rather like, get a large refactor done in a bunch of small PRs that don't have merge conflicts and all independently compile. So for instance, like a common use case is if there's a large file, you're like, suggest several tasks to break this down into a smaller class, and then you can like, do the refactor in like, a few small buttons. So basically, this is, this is something we're thinking about. And like, I think long term, like, our goal is to like, converge a delegation and pairing, real time pairing, into just like, a single experience.

• 39:36Dan Shipper?

  That's interesting. Okay. And, and we only have a couple of minutes left, so I really need to ask, like, how do you see the agent landscape evolving? So there's a lot of agents. It feels like every programmer is working on their own agent. There's like, the, the big labs, like Claude Code's got, you know, Claude's got their agent. There's, there's like, a ton of startups. There's like, the Devins of the world, and there's a couple that are coming out soon. Like, we're testing like, four or five of them internally. And so how do you think about this, like, Codex versus every other agent, the, the, the difference in the positioning or the functionality? And then do more generally, like, how do you see that market evolving? Is there like, is this sort of like, a one agent to rule them all type situation? Or do you think that there's going to be sort of like, this ecosystem of agents with different personalities?

• 40:20Speaker 01

  Yeah, I mean, I just think this is an incredibly exciting time to be doing software development. Because there's just so much innovation happening, and there's like, so many good products out there. So personally, just like, really excited about that. I think the things that we're, we're really focused on and uniquely positioned to do well are like, well, the fact that we basically can train the model, right? And we can go as deep as we want into making a model that's like, as good as possible for the specific use case, right? And so like, the things that I'm most excited about us doing are like, things like, well, you know, obviously having like, really good coding intelligence. But then beyond that, like, really thinking about like, okay, but you know, this is going to be used by an engineer, delegating to many agents in parallel, like, how do we make that mergeable?

• 41:03Speaker 01

  Like, how do we get the style right, the instruction following, the steering, you know, have its site, its work, stuff like that. And so I think like, for us, like, really leaning into the model is like, definitely like, one thing that we're going to continue to do. And then I think the other thing is like, just like thinking about like the scalable compute infrastructure, it just happens to be something that we're like quite good at here because of the way we do training. And then lastly, like for me, and this might be like a little bit more out there, but like ultimately like, so the reason that I work here, you know, bringing the benefits of AGI to all humanity. And if I think of the shape of that, that's not like some like ultra bespoke thing. It's like this general, like AGI super assistant. And so like, that's kind of the world that I want to live in.

• 41:47Speaker 01

  Like, I want to wake up and just like, well, probably not immediately, maybe go for a walk first, don't look at my phone. But then at some point, I want to just like reach out to an assistant, like have a conversation with it and like, just have one thing and it does stuff. And then I can like dive in and do stuff myself. And so I think like, that's, I feel like the chat GPT is like as an app and as an organization is like very like AGI super assistant pilled. And so like, for me, that's kind of where we end up bringing this towards.

• 42:12Dan Shipper?

  Yeah, that makes sense. The integration on chat GPT is the big thing that's going to make this very different from any other, any other tool like this.

• 42:20Speaker 01

  Yeah.

• 42:21Dan Shipper?

  GPT. Yeah. Alexander, it was so great to chat. Thank you so much for coming on. Thanks for building this. Thanks for letting me try it early. I'm psyched to see where it goes.

• 42:31Speaker 01

  Cool. Yeah. Thanks so much for trying it early. Thank you for all the feedback. Hopefully you noticed many, many of the issues are addressed. And yeah, thanks again.