Codex from 0 to 10M Users: Building ChatGPT Work — Akshay Nathan, OpenAI

There are roughly 100x more people who use code than who can write code. As code that “just works” becomes easier to generate, this group may be the biggest prize of all — if you can get the agentic interface right. A key trend we have been tracking over at AINews is the absolute explosion in Codex usage this year https://www.latent.space/p/ainews-codex-usage-up-10x-in-6-months, with MAU now up >10x from Jan 2026. Less than two weeks after their July 9th launch https://openai.com/chatgpt-work/, OpenAI said ChatGPT Work and Codex had reached 10M users combined (as we cover in the pod, Codex now powers ChatGPT Work, so all ChatGPT Work users are now users of the Codex harness, even if they aren’t traditional engineers) — showing the early innings of what happens when you graduate from coding agents to knowledge work agents: We’ve been calling out how coding agents are “breaking containment” to do everything else https://www.latent.space/p/ainews-agents-for-everything-else?utm_source=publication-search this year to power every other part of knowledge work - and it started with the org chart, with a major reorg last month https://the-decoder.com/greg-brockman-consolidates-openais-product-teams-to-build-an-agentic-future/ that amounted to two of Codex’s most prominent leaders, Greg and Tibo, taking responsibility over product and ChatGPT specifically, completing a “Superapp” consolidation cycle first discussed in March https://www.latent.space/p/ainews-every-lab-serious-enough-about?utm_source=publication-search. With these updates Codex is no longer just a coding tool. In June, OpenAI said knowledge workers already accounting for roughly 20% of Codex’s user base and growing more than 3x as quickly as developers. A product dedicated for knowledge workers was being pulled out of the Codex team. However, knowledge work has a different set of problems and environments than coding. For decades, knowledge work has been scattered across different primitives like documents for writing, spreadsheets for analysis, slide decks for communication, and specialized applications for everything else. ChatGPT Work https://openai.com/index/chatgpt-for-your-most-ambitious-work/?utm_source=chatgpt.com now enables users to work across every primitive with agents. Instead of opening an application and manually operating its features, the user can describe an outcome and collaborates with an agent that can assemble the tools, context, and artifact needed to reach it. From building no-code products at Airtable to leading Productivity Engineering at OpenAI, Akshay Nathan has spent much of his career trying to make the power of software accessible to people who do not write code. In this episode, Akshay joins swyx and Vibhu to unpack the launch of ChatGPT Work, why Codex unexpectedly took off among non-developers inside OpenAI, and the company’s broader plan to bring useful agents from software engineers to knowledge workers and eventually everyone. We go deep on the shared agent harness behind Codex and ChatGPT Work, why OpenAI brought the experiences together without making them identical, and how persistent computers, artifacts, Sites, plugins, memory, and sub-agents are changing what people can delegate to AI. Akshay explains why some teams are replacing decks and spreadsheets with interactive websites, how agents can gather context across code, Slack, documents, and local files, and what OpenAI learned from personal-agent products like OpenClaw. Side note: also don’t miss Abhihek’s sandbox track keynote at AIE, which now powers a lot of the sandboxing for ChatGPT Work… and yes was also broken by an unreleased OpenAI model in the recent HuggingFace incident https://www.latent.space/p/ainews-ai-cybersecurity-becomes-top. Akshay also reflects on how AI is transforming product development itself: why more people will become generalists with a specialty, why ideas and taste become the bottlenecks when almost anyone can build, why LLMs still struggle to generate genuinely grounded new ideas, and why teams must distinguish increased motion from actual progress. We discuss: * Why Codex unexpectedly took off among non-developers inside OpenAI * Why employees felt like using Codex gave them a new superpower * The product insight that led OpenAI to build ChatGPT Work * Why Codex and ChatGPT Work share the same underlying agent harness * How their UX, Git visibility, artifacts, and sandboxing defaults differ * Why OpenAI merged its agent experiences instead of building separate products * How AI is blurring the boundaries between engineering, design, strategy, and operations * Why OpenAI wants the default model configuration to work for most users * When power users should use deeper reasoning, Ultra, or multi-agent modes * Artifacts, agentic spreadsheets, and creating high-fidelity work products * Why interactive Sites may replace decks and spreadsheets * The challenge of designing a simple interface for an agent that can build almost anything * Why users should retry tasks that models could not handle three or six months ago * How AI can gather context for performance reviews without replacing human judgment * The OpenAI automation that turns internal Slack and document activity into memes * What reaching ten million ChatGPT Work and Codex users means for the product * How OpenClaw inspired persistent environments, scheduled tasks, and personal agents * Using ChatGPT for financial planning, budgeting, workouts, meals, and household management * The design tradeoffs behind sub-agents and how much of their work users should see * ChatGPT memory, Chronicle, and long-term context * Why AI may make more people generalists with deep specialties * Why ideas and taste become more important when almost anyone can build * Why LLMs still struggle with the instruction “bring me new ideas” * Measuring productivity through quality at-bats instead of commits, tokens, or pull requests * The critical difference between AI-generated motion and meaningful progress Akshay Nathan * LinkedIn: https://www.linkedin.com/in/akshaynathan/ * X: https://x.com/akshaynathan_ Timestamps 00:00:00 Introduction and Bringing the Power of Code to Everyone 00:01:33 Joining OpenAI and Preserving a Startup Culture 00:02:40 What OpenAI Learned from Enterprise AI Adoption 00:05:28 Why OpenAI Built ChatGPT Work 00:07:17 Codex vs. ChatGPT Work and the Shared Agent Harness 00:12:07 Why OpenAI Merged Its Agent Experiences 00:16:24 Models, Reasoning Levels, and Choosing the Right Default 00:20:26 Artifacts, Agentic Spreadsheets, and Model–Product Collaboration 00:24:22 Why Sites Could Replace Decks and Spreadsheets 00:30:08 Designing an Agent That Can Build Almost Anything 00:34:28 From Developer Agents to Knowledge Work—and Everyone 00:36:07 Power-User Advice and AI-Assisted Performance Reviews 00:40:41 OpenAI’s Internal AI Memes and the Ten-Million-User Launch 00:44:39 OpenClaw, Personal Agents, and ChatGPT as an Operating System 00:50:24 Sub-Agents, Ultra Mode, and How Much Control Users Need 00:54:39 ChatGPT Memory, Personalization, and Chronicle 01:00:19 How AI Is Reshaping Product Development and Tech Roles 01:03:15 Ideas, Taste, and Why LLMs Struggle to Generate New Ideas 01:04:42 Measuring Productivity, Quality At-Bats, and Motion vs. Progress Transcript Introduction: Akshay Nathan, ChatGPT Work, and the No-Code Arc Swyx [00:00:00]: We’re here in the studio with Akshay from OpenAI. Welcome. Akshay Nathan [00:00:07]: Thank you. Swyx [00:00:08]: And with our trusty co-host, Vibhu. So you recently launched ChatGPT Work. You lead Core Product Engineering. It’s been a long journey, into all this. I find it very interesting that you started with no code or low code, with Walrus and Airtable. And to some extent, ChatGPT Work is like the super app of super apps of, well, here is the ultimate no code. You just write a prompt. Akshay Nathan [00:00:32]: Yeah. It’s funny how things come, full circle. I think for a long time in my career, I started my career working consumer fintech, but then after that, like, there’s this hypothesis that, the things that we were able to do with code, like, as engineers, like, if we could bring that to many more people in a more, accessible way, then that would be truly magical. We were working on a startup. It’s funny, like, before LLMs, before vision LLMs, on how to do automated testing with AI. It was just kinda jank, back then, but doing what we can, and then worked at Airtable for a while on the same thesis that, like, if we can bring a database or the primitives behind a database to people, that’d be really useful to them. But once LLMs came onto the scene, it became clear that, this was the missing piece, like, the missing technology required to, like, bring the magic of code to everyone without them having to know what’s going on underneath the hood. And so, like, I think this launch and a lot of the stuff that we’ve been up to is, like, the manifestation of that. From Walrus and Airtable to OpenAI Vibhu [00:01:33]: How was stuff when you joined? So you joined OpenAI 2023. Now we’ve got, so much more stuff, so ChatGPT, Codex app, ChatGPT Work. Have things changed? Joining OpenAI and What Hasn’t Changed Akshay Nathan [00:01:44]: I think the more interesting thing is how things haven’t changed. Like, one, I joined I remember when I joined, it was, like, five hundred people. One thing I was worried about was, like, I was looking for something, more early stage and, like, was it gonna feel startup enough? And I joined, and I was like, “This feels even more startup-y than I could ever imagine.” And, like, that really hasn’t changed even till now. I think the, like, level of, like, bottoms-up ambition and, like, the ability of anyone to, like, do anything or have an idea and ship it is really cool. But on the, like, mission side, I think what was really compelling to me is this mission of, bringing frontier intelligence to everyone. Like, building AGI and then bringing it to everyone. And, I think acknowledging back then that, like, that vision is gonna, not be a linear progression. Like, we’re probably gonna, like, try different products and have different things that succeed and don’t. But the vision has stayed the same, and the mission has stayed the same, and we’re starting to see the pieces, fall together, and that’s really cool. Enterprise Lessons: No One-Size-Fits-All AI Swyx [00:02:40]: You worked on Enterprise. What A lot of people never touch ChatGPT Enterprise. What is something that you learned from there that you’re bringing into your work now? Akshay Nathan [00:02:52]: I think how there’s no one-size-fits-all solution in Enterprise. I remember in the early days of ChatGPT Enterprise, like, when we talked to customers and, like, everyone. That was, like, when I think it was a year after ChatGPT was released, and everyone was so excited to bring, AI into their enterprise. And, there were all these teams being stood up. It was, like, the AI deployment team with, like, these enormous budgets. And if you asked anyone, like, what were they excited about? Like, what were they excited about solving? Like, at first, you’d get, like, kinda like the baseline answers of, like, “Yeah, we have all this context and data and all this stuff.” But then if you ask them, like, “What was, like, a discrete use case that, like, they want AI to enable in their workplace?” You get such a different, like, variance, like, explosion of, different types of answers. And it’s interesting, like, you using, like, these models and these products, you have this box, and you can say anything to it, which is the magic. But it’on the flip side, it also means that, like, you don’t know what to do with it. And in Enterprise, I think a big part of that is, like, meeting the users where they are, like, what use case were they trying to solve, and then teaching them how they can use AI to, like, gain leverage there. Swyx [00:03:56]: Do you meaningfully differentiate that from forward-deployed engineering? Akshay Nathan [00:04:01]: I think there is the go-to-market side of it and then there is the product side of it. I think you need someone on the product side. And I think, like, however good we get at FDE motion, like, I think at the end of the day, if we have a user who’s, like, looking at their computer or looking at their phone, like, it’s our job in the product to, like, be enabling them and showing them where to go. So we’re really excited about that. Vibhu [00:04:24]: Do you think there’s been changes, over the past three years of adoption? So there have been, step function changes. You have reasoning models and whatnot. Is there still the same problems of Enterprise has black box, don’t know what to do with it, or have things changed? Adoption, Agents, and the Next 10x Market Akshay Nathan [00:04:39]: We’re seeing now that, like, there’s this huge uptake, right? Everyone is extremely excited about it. It feels like, many people are, millions, hundreds of millions of people are using ChatGPT. They understand, like, how generally to work with AI. But then, like, every time, like, a new capability gets unlocked, so now, like, we’re seeing with agents, like, there is probably a contingent of, like, early adopters still who, truly get it, who are like, “ we you can do anything. You just have to make sure the right context is there, it’s connected to the right tools, and that you are supervising it, but, like, anything is possible.” But then there’s, like, this, like, 10x or 100x bigger market where, like, they don’t yet get that, or they don’t yet see that. And so I think that’s the next stage here. So to answer your question, like, I think the adoption is there and growing fast, but I think the opportunity is, like, far bigger than that. That’s where we wanna play, especially with ChatGPT Work. ChatGPT Work, Codex, and the Super App Merge Swyx [00:05:27]: Yeah. well, let’s, let’s skip ahead to ChatGPT Work. only, like, a month ago or so, announced. what was the decision process that led into it? there was this, overall merging of the super app. Is that what we’re officially calling it? you deprecated the browser as well. Just, summarize your last, like, couple months of working on this thing. Akshay Nathan [00:05:50]: Yeah. It feels like forever now, but it’s only been a few months. I think maybe the one, impetus that, like- Is most salient is when we release Codex, or even internally had Codex, like, it was really surprising to us, I think we recently put out some stats on this, that there was this, like, real inflection of, like, adoption among non-developers at OpenAI. And, I, through this product development process, like, would go to, like, these UXR sessions to talk to people internally. And the thing that stuck out to me is, like, one, like, you go talk to, like, strategic finance or marketing or whatever, and they’re all using Codex for, their use cases. That part’s cool, but the thing that really stuck out to me is how proud people were that they were using Codex. Like, how, like Swyx [00:06:34]: It’s like, “I’m not supposed to be using it, but I am.” Akshay Nathan [00:06:36]: It was that. It was, like, that they were, early to this, like, new thing, but it was also this thing of, like, they felt like they had a superpower, right? And, what we recognized then is that, like, the power of Codex, the power of agents, like, we already had this massive distribution base of people who have, come to know and love ChatGPT. Like, how do we show that to them? Like, how do we bring it to them? Which is, like, a hard product problem, and it’s, like, a tricky thing, right? There’s many ways you can go about it. And so that’s what we called the Merge and the Super App over time, and ultimately launched it in ChatGPT Work, is how do we do that? But it came from that initial realization that, like, the power was not only for developers, like, much earlier than probably even we thought. Like, it could be extended to everyone. Swyx [00:07:17]: How do you see the products differently? So, like, who is it for, right? So Codex started out even CLI, then app. Now there’s a merge of ChatGPT Codex and ChatGPT Work, so is it the opening for the average user, for enterprise, for work? How do you position it? Akshay Nathan [00:07:36]: I think we want to get it to position it for if you’re doing work-related things, for lack of a better word, right? Who ChatGPT Work Is For Akshay Nathan [00:07:42]: I think productivity is what, like, the pillar that I support. Like, that’s the name of the team. And the reason for that, the reason we call it productivity and not, like, enterprise or, like, work or something like that, is because there’s also personal productivity, right? And, like, I think ChatGPT Work is I’ve seen people do things in their personal lives that you wouldn’t classify as, like, work technically, but, like, these agents are, super capable for. Like, one recent example that someone posted about, on our Slack is, like, someone had, like, a missed package, like they didn’t receive it, and then they got, like, the picture of it, from Amazon or whoever the courier was, and they, like, asked ChatGPT Work to, like, find out where that package is. And, like, the agent, is extremely tenacious and, like, took the image and, like, looked at a bunch of, like, listings around their neighborhood and figured out exactly the apartment complex in which the package was, like, gave them some information. And so, like, I think there’s all these things that, like, you, work-related or productivity-related things, I think that’s what we want the product to be. You asked about Codex. I think we think Codex is, a durable brand, but we have a principle that, like, the user we don’t want a user to get stuck in a tab or an experience where they don’t get the power of the product. And so, like, everything that you can do, in the Codex portion of the product on desktop, you can do in ChatGPT Work and vice versa. But we made some opinionated product decisions on, like, how much of the Git state, if you’re in a Git repo, do we wanna expose to the end user? Or how much do we wanna make the experience of seeing the agents thinking, like, diff forward so that you get exposed to the diffs out of the box. And then, like, on the safety side, like, how do we wanna think about, like, sandboxing and making sure that we have the right defaults in one state versus the other? So, there’s, like, some opinions that go behind that, but we do want We don’t want the user to need to choose which experience they’re in. Swyx [00:09:26]: That is a good goal for AGI, right? Like, people don’t want, like, to hide to choose what version of AGI they want. They just want the AGI to decide for them. can I get an answer or, like It’s not super clear to me. Is the Codex harness and the ChatGPT Work harness the same? Is it just UI affordances, or are there prompt level or even deeper differences? Shared Harness, Different UX: Codex vs. Work Akshay Nathan [00:09:49]: So the harness is the same. The harness is shared. on In both of the products, we made improvements to the harness to make it good for knowledge work, especially as it relates to plug-ins or computer use or artifacts. You get that power regardless of which experience you’re in. On the UX side, there’s opinionated takes that we have when you’re in Codex mode, what the UX should be how the UX should behave, and some stuff around the sandbox like I mentioned, but the underlying harness and capabilities should be the same. Swyx [00:10:16]: I’m just kinda curious. Maybe we can, -- Is there a query that we can run that would look different in the two modes? Akshay Nathan [00:10:23]: Yeah. I tried to create, like ask it to create, like, a retirement calculator spreadsheet or something, in both modes. And then in Codex mode, you might have to be in a repo for this, but you’ll see, like, the diffs of, like, the sheet that it’s creating and stuff like that, and the file edits. But in Work you won’t be able to see that. Swyx [00:10:42]: I think that’s, that’s super clear. And then also the other thing I wanted to dive into was your, the productivity team. what else is there? first of all, what are the top-level teams other than productivity? Isn’t productivity everything? Productivity Teams and Core Chat Akshay Nathan [00:10:55]: So Swyx [00:10:55]: Science? Akshay Nathan [00:10:55]: We have a team focused on ChatGPT. Like, the core chat experience, for consumer, which is like, not, I think all productivity. Like, there’People are using ChatGPT every day for search to, figure out how to write messages to loved ones, to think about, how to, like, learn a new topic, et cetera. And so there’s so much more inside to create images. And there’s so much more in chat that, the hundreds of millions of users are using that warrants, like, a very dedicated effort. And there’s teams focused on enterprise and infrastructure and API and stuff like that, so. Swyx [00:11:33]: I will bring it up. Retirement Calculator Demo and Git-First UX Swyx [00:11:34]: Yeah. So I have them both running. This is ChatGPT Work. There’s a Codex version here. I picked “Five Little Ducks” song, so this will take a while. Akshay Nathan [00:11:43]: Huh. Swyx [00:11:43]: I think we’ll just keep it in the background and, as they finish, we’ll look into some of the differences. Akshay Nathan [00:11:48]: Yeah. But immediately, I think if you flip back to the Codex version you’ll see that, Swyx [00:11:53]: That it assumes Akshay Nathan [00:11:54]: Like the Swyx [00:11:54]: It assumes Git. Yeah. Yeah. Akshay Nathan [00:11:56]: The, like, dynamic island assumes that you’re in a Git repo. And you might miss some stuff because some of it is, like, in the actual chain of thought with those changes and how we display that, but yeah. Swyx [00:12:07]: Is there an unintuitive like, is there a thing that you wanted to ship and then you got feedback, and you were like, “No, let’s not do it?” Like, what’s the thinking behind that? Why Merge the Experiences Akshay Nathan [00:12:14]: In, ChatGPT Work? Akshay Nathan [00:12:17]: I think one direction we could have gone with this is, like, keeping the experiences, like, completely separate. So it’s like, why Swyx [00:12:22]: Different apps. Akshay Nathan [00:12:23]: Exactly, like different apps or even in the same app, like different, completely different experiences. Like, why merge it all? Like, what is. Codex, people love. Like, why bring these products together? And I think the intuition here is that, like, all of our jobs are, like, changing dramatically with AI. Like, for, like, every few months, like, I feel like I wake up, and I’m, like, doing a completely different thing than I was doing a few months ago. And my hypothesis here is that, or I should say our hypothesis is that, like, part of what we’re, we’re building, this technology is giving people leverage. Like, the things, maybe it’s the more mundane parts of your job or parts that, like, if you were able to automate, you’d be able to share more ideas faster or whatever, like, you’re able to do now. And because of that, like, that might blur the lines between someone who’s, like, only writing code or creating strategy docs or, planning events or, helping with marketing or doing podcasts or whatever, right? And so, like, these things are gonna get blurred over time. And so, like, trying to draw a hard boundary based on, like, the who you are is gonna be, is gonna be tough. And, like, we should enable users to choose, but we shouldn’t box them in. And so a lot of the work that went in here, like, keeping the primitives the same, like for example, plugins are, like, unified across, this product and ChatGPT and the cloud, was because of that. It’s this thesis that, like, eventually things are gonna come together and we don’t wanna be Like, we wanna be prescriptive about when to be in either experience, but we don’t want to box anyone in. Swyx [00:13:45]: I wonder if there’s users who are very tuned to the old ChatGPT harness that is effectively now replaced by the Codex harness. I can’t imagine what that was, but maybe they’re more the more conversational side. Can you compare and contrast the two harnesses? ‘Cause only you’ve seen it. Akshay Nathan [00:14:02]: Yeah. I think ChatGPT, the existing harness, like, still exists today. Like, it exists in this app, Harness Engineering: ChatGPT vs. Codex Swyx [00:14:08]: The classic, right? Akshay Nathan [00:14:09]: The Vibhu [00:14:09]: You just start a new chat, and you don’t go under Work, right? Akshay Nathan [00:14:13]: Yeah. If you start Vibhu [00:14:13]: So Akshay Nathan [00:14:14]: A new chat and go to chat, then you’re, you’re talking to ChatGPT with the instant model. Vibhu [00:14:16]: Oh, we can technically do another. But on instant. Swyx [00:14:21]: Yeah. So this one’s not gonna code or it’s gonna be in line. It’s on a in line in a sandbox. Akshay Nathan [00:14:26]: It’ll Vibhu [00:14:27]: Oh, that’s cool Akshay Nathan [00:14:27]: We try to push you to go to Work if you’re creating a spreadsheet. Yeah, but this is Swyx [00:14:30]: And this is a router decision? Sorry. Is it a router decision? Akshay Nathan [00:14:34]: This is the decision that, the model is making, and then, like it sees that you’re able to. or you’re trying to do something that would be better served in Work mode. But I think your question was like, what are the advantages of, like, the chat, like ChatGPT chat harness? Swyx [00:14:48]: It’s more broadly, like, I wanna, do an oral history of harness engineering. Right? the ChatGPT harness lasted us from, let’s call it the ‘01 era, until now, and now it’s being replaced by the Codex harness effectively. And they’re, they’re overlapping somewhat, but I’m curious what changed if there is. Akshay Nathan [00:15:10]: My perspective on this is, like, there’s, there’s, there’s there’s like a constant process of, like, divergence, convergence, divergence, convergence. And in chat, like, many of the use cases I was talking about before, like, search or learning, I think we’re, we’re really optimizing for latency and optimizing for personality and, like, different things that, over time, like the product The reason people love ChatGPT is because we’ve been optimizing for those things and working on them for so long. Codex, what we learned was that, like, if you give the agent access to this infinitely flexible environment as a computer, it can do really powerful things. And so when we think about, like, okay, well, for knowledge work, like, what is which mode should we choose? It was like it felt more natural to us to bring that to this, like, computer environment and, maybe abstract some of the details of this computer away from users who might not be used to that, but, like, give them that same power. But ultimately, I think that we want the power in all places, right? We wanna meet people where they are. So I’m sure there’ll be work down the road in order to get things to be, equivalently capable in all scenarios. But it’s just a question of, like, what we’ve been focusing on the product on historically and what we’re focusing on now. Models, Defaults, and the Reasoning Slider Vibhu [00:16:24]: I think alongside that, outside of just harness and when to use Codex, ChatGPT, or Work, there’s also the new models you’ve released, right? any guidance there? So people love to min-max what to use, like only use Terra on high reasoning versus, for this, you wanna use Sol here, ignore all these Akshay Nathan [00:16:44]: There’s 32 options. Vibhu [00:16:46]: But, that being said, for people that are expanding, so, productivity trying stuff for work that don’t have the breakdown of what all this is what’s, what’s the advice, right? Akshay Nathan [00:16:59]: Well, I think before the advice, like the first thing is, like, none of this would be possible without these models. Like, the, I think you asked earlier, like, what was, like, the inspiration for work and, like, early on, like I mentioned, like, what we were seeing with Codex, but that was also because the models were getting infinitely more capable. That’s happening again. I think it’s like another step function jump now. And to answer the question on advice, like we want this default to be the best possible. Like, we wanna be opinionated about the default, and so we’ve we’ve chosen a default that we think is gonna be the best for everyone. And, we have for power users options under the hood. We could One could argue that there might be too many right now, and we’re, working on simplifying it. But you can extend, the reasoning level, and you can change between the different model classes if you need to, but the default should be the best for most use cases. So my advice to most people would be to stick to that. And then, if you reach a situation in which you think that you could, you wanna try, a different configuration, if you’re not seeing either the efficiency on the cost side or the quality on the intelligence side, then you can change the defaults and see if you can get something better. But we think that the default should be good enough. Swyx [00:18:09]: I have, I’m just gonna run something by you since you have way more experience than me. I’ve recently been doing Sol Lite but with goal, with the idea that the goal augments the reasoning effort, but with more terminations and turns. Swyx [00:18:24]: Is that a good way to think about it as opposed to Sol Ultra or Sol, Extra High? Akshay Nathan [00:18:29]: Yeah. It’s hard to say because Swyx [00:18:31]: Yeah. It’s like an interaction effect. Akshay Nathan [00:18:33]: exactly. It’s like there’s a preference on, for you as an individual, like how do you like to collaborate with the models? Like how many of those like terminations, as you call them, do you want where, you can steer or make sure that it’s doing the right thing? Akshay Nathan [00:18:46]: I think generally people should try whatever works for them. I think that like using Ultra or the like multi-agent setups are best for like when you have like tasks that are either incredibly complicated, like open explorations or very paralyzable. I think even for tasks using goal, I think is best for tasks that you’ll be able to make consistent progress in a way that’s verifiable over time. But I think for most tasks, they don’t fall into either of those buckets. And so like at least when they’re starting, and so that’s why I think the best first step is like trying it with the default configuration and then seeing like where you wanna go from there. Swyx [00:19:29]: Right. You guys worked on a slider, which is super helpful for reducing the amount of panic. Vibhu [00:19:36]: It’s nice on mobile at least. There’s a nice slider there. Swyx [00:19:38]: It’s nicer. Vibhu [00:19:39]: I haven’t tried it. Swyx [00:19:40]: So you have the advanced view there, but if you click advanced view. Yeah. Vibhu [00:19:44]: Ooh, it’s just a nice slider. Yeah. Swyx [00:19:46]: Very pretty, very colorful. Akshay Nathan [00:19:48]: Yeah. The idea was here was like reduce it to like one dimension even though there’s multiple dimensions, right? Try to project it onto a single dimension for the user. Like, something from that represents like, speed and efficiency on one side and then like quality and thoroughness on the other side. Artifacts, Spreadsheets, and the Work Launch Swyx [00:20:04]: I am just puzzled that it uses Sol so much, like the lower Vibhu [00:20:07]: No Swyx [00:20:07]: Grounds I would’ve used Vibhu [00:20:08]: I think the slider, if I’m not mistaken, is Swyx [00:20:09]: Terra. Vibhu [00:20:10]: Oh, it is. Swyx [00:20:11]: Yeah. See? So they preset Terra to only be the light one. But like I think a lot of people would more people should use Terra. One, because Sol keeps running out of capacity. Vibhu [00:20:22]: I’m the reason. Here’s ten minutes of our Swyx [00:20:24]: There you go Vibhu [00:20:25]: Retirement calculator. Swyx [00:20:26]: Oh, that’s the Excel thing working for you. Vibhu [00:20:28]: This is, Swyx [00:20:28]: Oh my God. Look at that Vibhu [00:20:28]: This is work, and then Codex is still cooking, so we’ll get back into it. I think it’ll be interesting to see the thought process, the reasoning, and also, this is eight minutes on work. Codex is still cooking. Swyx [00:20:41]: Yeah. And by the way, so I’ve, do Gabriel Chua? He’s part of the OpenAI Singapore team. He showed me this, and I was like pretty shocked that this looks like Excel. It edits Excel files. You never paid an Excel license, right? Like, but somehow this is like workable and it’s agentic Excel. Akshay Nathan [00:21:01]: Yeah. one of the big like pushes that we made for this launch was like artifacts, right? Akshay Nathan [00:21:05]: Like both on the model side, like I think if you compare this with GPT-5.5 and GPT-5.4 before that, you’ll see that there’s been pretty dramatic improvements in the quality of these artifacts and then also on the product side. Vibhu [00:21:16]: The UX side is also crazy, like hosted sites and whatnot. No longer needing to host your own little webpage, like it Swyx [00:21:23]: Oh, I have a story about that. I can do, a separate thing. I’ll need to take the visuals here, but we-we’ll, we’ll cut to that later. Was there co-training, because you were moving making this big move and you launched GPT-5.6 on the same day as ChatGPT Work? Was there influence between the model training teams and the harness teams, or did they did the launch dates just happen to line up the same day? Akshay Nathan [00:21:46]: I think the we collaborate heavily with the research teams, and I think that’s like one of the most magical parts of the job, like the most fun parts of the job. But yeah, just using artifacts as an example. Like, a lot of what you’re seeing, like underneath the hood, there’s a lot of work that went into making sure that like, we had the right infra to be able to train the models to get better at this. And then on the product side, like had the right experience for users to be able to collaborate with the model on an artifact like this. In fact, like this whole viewer, like the intuition here is that like, it’s not necessarily that you wouldn’t need an Excel license. This is stage one, right? Like, this is probably not what you meant when you’re like making a retirement calculator. Vibhu [00:22:24]: Yeah, you can iterate very easily. Yeah. Akshay Nathan [00:22:24]: You wanna iterate and like when you’re seeing it, and if this thing is high fidelity to like what you would see in or what your coworkers would see if you were to send this to Sean, like that I think makes it so easier and makes you trust the product in terms of iteration. Vibhu [00:22:39]: When you say coworkers would see, do you see a multiplayer, multi-team collaboration with artifacts? Any things you guys think about that? Multiplayer Artifacts and Collaboration Swyx [00:22:46]: You can already share it, right? Akshay Nathan [00:22:48]: Yeah. It’s inter It’s something that, we’re actively thinking about. one thing that, we’ve noticed internally without talking too much about the roadmap is that like there’s many times when someone will ping me about something, and I will ask ChatGPT Work the question, and then I’ll ping them back the answer. Akshay Nathan [00:23:04]: And then I’ll be thinking like Vibhu [00:23:04]: Like the simplest would be, the three of us are just all on one hosted. Akshay Nathan [00:23:07]: Exactly. And I’ll think about like was I required in this loop or and then maybe it was, rephrase like what they were asking or pulled from certain context or whatever. But like, when I gave them back the answer, that process was also lossy, right? Like I gave them just like my interpretation of what ChatGPT Work cooked up. But like underneath the hood, there’s so much context like in the rollout and stuff that could be interesting. Vibhu [00:23:28]: Yeah, it’s Swyx [00:23:28]: So like the answer was preemptively respond to every inbound request? Akshay Nathan [00:23:33]: No, it was just like literally like this is what I do sometimes as my job. Swyx [00:23:36]: I know you copy-paste and then you’re just a message forwarding service Akshay Nathan [00:23:39]: Yeah. Yeah, exactly Swyx [00:23:39]: From AI to AI. Vibhu [00:23:40]: But I think it’s interesting, right? It helps people understand the capability of what you can ask and delegate that oftentimes people don’t realize until they try or someone shows you, and then you’re like, “Oh, okay. Okay, I see.” Swyx [00:23:52]: I think it’s als there’s also like a, light security issue, where like you’re the permissions layer. Like yes, I could query everything that you query, and I could get an automated response, but maybe I’m not supposed to see it. And that there’s no way I would know because I’m not supposed to know what I don’t know. Akshay Nathan [00:24:07]: Especially as like, with ChatGPT Work, we’re, we’re asking you to connect your plug-ins and, it’s pulling from your local files and stuff like that. Like the amount of context that the agent has access to is like- Deeply personal and like that’s something I think we need to preserve, so that’ll be definitely a challenge. Swyx [00:24:22]: There’s Excel, there’s PowerPoint, there’s Docs, the, grand trio of work. What other formats of work do you think about? like you worked on Airtable. Is there a future where there’s like OpenAI Airtable? Like what does that look like if you ever ended up doing it? Akshay Nathan [00:24:41]: It’s a really good question. I think, Formats of Work: Sites as Knowledge Artifacts Akshay Nathan [00:24:43]: one that you didn’t bring up was Sites, and I think that was Swyx [00:24:46]: Sites Akshay Nathan [00:24:46]: A core part of this launch. There’s one side of Sites that I think people commonly talk about, especially on Twitter and stuff or X, of like, this like prototyping tool. And like we saw that happen with this launch even. The model slider that you guys were referencing earlier, like that was developed almost fully in a Site. Like, the collaboration between design and engineering and product on that was like on a site where we play with, the affordance and figure out how it feels and all of that. But the other aspect that I think is a little bit less talked about is like Sites as like an artifact for knowledge work. I was talking to someone the other day who’s on like our corporate finance team, and like we were mentioning how like now when they have these reports that they’re, they’re working on as a team month to month, historically those things were in slide decks and in spreadsheets, and now they’re just in Sites. And like Sites is the mechanism that they collaborate across the team. And the reason is ‘cause it’s like, it’s like somewhat higher bandwidth. Like, at these tools like PowerPoint and Excel are like infinitely flexible, but at some point you reach the boundary of like either as a human you may not know how to use some feature or something, or the product itself doesn’t support it. But with a site you can do anything. You ask for anything and you can get that. once people see that magic, I think it’s been really valuable. Swyx [00:26:02]: Yeah, let me show you my case study. this involves all the hot topics including ChatGPT Work, but also GPT-5.6 token billionaires and token maxing and Sites and auto research. I’m a fan of this game called Strata. It’s, it’s like a little board game that you Sites, Auto Research, and Research Dashboards Swyx [00:26:17]: That you play with, physical blocks, that come on top of it like that. So over the weekend I took like thirty photos and just threw into ChatGPT. one point seven billion tokens later, out comes this site with a fully playable thing Akshay Nathan [00:26:32]: Wow Swyx [00:26:32]: With 3D, block placement and everything. Because it requires physical blocks and I needed friends to train on it so they can get better, so I can play against them. But also, I could also, do things like train an AI on it and that’s, that Akshay Nathan [00:26:45]: That’s your auto research Swyx [00:26:46]: That gets into auto research. So, you want to train your own AIs, and then make sure they self-play against, each other. I need to set both AIs. So this is AI versus AI, and they’re, they’re gonna self-play. the AIs start out bad and then you want to define a loss function and get good. I wasn’t gonna supervise all this. I was at, I was down in San Mateo, attending a conference. What I ended up doing was, auto researching and on this and creating benchmarks and that there was just way too many parameters for me to read. So I started asking it for a site, and it’s created this lab, panel. Where is there a, is there a shortcut for a site that is created? Akshay Nathan [00:27:28]: You should be able to go in the sidebar to Sites, top of the sidebar. The left sidebar. Swyx [00:27:33]: This one? Oh, left? Akshay Nathan [00:27:35]: Yeah. Just scroll all the way to the top. Swyx [00:27:36]: Oh. Oh, it says Sites. Oh, there you go. Yeah. Akshay Nathan [00:27:39]: Ooh. Swyx [00:27:40]: So it create, it creates the sites. I don’t, I don’t think this is, it is exactly what I wanted, but let me show you what it popped up, right? Like I think as a research artifact, it is very important to communicate, exactly, what is being done. Outputs this thing which I eventually started publishing. So I moved it off of Sites because I wanted more, database and infrastructure than Sites afforded me. But this is like a research output that you can start to mess with and like try to think about like what hyperparameters are you tuning for training AIs. And like I was trying to make like scaling laws and everything and doing all sorts of like game optimization stuff. And the fact that you can just throw this up as a research artifact, like I no longer need to read ChatGPT output. I read Site output. But then there’s also a huge sprawl. Like look at how long this thing is. There’s so many numbers. It is pretty overwhelming, so then I have to start pruning it from there. But, it’s an interesting transition from Markdown effectively that you’re putting out to, you’re putting out a whole functional site. Akshay Nathan [00:28:41]: I think Markdown just isn’t that optimal for people to read, right? Might as well just write HTML website and I don’t know. I think you can do a lot with customizing this, right? You have your skills that explain what you want. Like I noticed they’re quite verbose. I don’t need a lot of this information. Swyx [00:28:57]: It’s very verbose. Akshay Nathan [00:28:58]: So and then the nice thing of having a site side by side is, you just iterate on what you want and what you don’t, right? Swyx [00:29:05]: Yeah. I don’t know if, any that triggers any stories for you of how it’s run internally. Am I doing this right? Akshay Nathan [00:29:11]: Yeah. I think that this is like a workflow that we’re seeing like all different types of teams use, where like the canonical artifact that was previously a deck or something is now becoming a site. And like with a site you, because it’s just HTML, you can like. It’s infinitely flexible. And so, if you want to give more prominence to a certain thing that like in a slide deck would, feel like it was buried, like you can do that. You can have it be like the hero image, right? And so I think that like, people are starting to see that. There’s more work to be done to make these things like much more easier, easy to collaborate on. You mentioned that they’re very, they’re long and verbose, could be broken up. I’m sure that there’s still something to do there. Swyx [00:29:53]: They’re super long. Yeah. Akshay Nathan [00:29:54]: Yeah. But I think we’re starting to see that like there is this aspect of this is a really interesting, format, for people to use, that’s like much more flexible than what they ever had before. Swyx [00:30:07]: I think your job also comes becomes meta. You’re not designing the products. You’re designing a product to make products, and I’m curious how you manage that. Designing a Product That Makes Products Akshay Nathan [00:30:18]: I think one thing that we’ve been Like when we look at the UX, like that we’ve been thinking a lot about is how can we balance like simplicity with capability? Like if we’re designing a product, like you said, that like is made to make up build other things, right? You can build so many different things. But we can’t put that all in front of you because you’ll get overwhelmed. Vibhu [00:30:41]: Yes. Akshay Nathan [00:30:41]: And so we had similar problem or similar challenges even Chat-with ChatGPT, but especially now, like when there’s so much that can be done, I think the balance that we’re constantly trying to strike is like, how can we give the user enough of a UI surface where, they can be expressive, they can tell the agent what they need, they can verify that it’s using the right tools, it’s pulling from the right sources, et cetera, but then it gets out of the way. And then how can we build the right system such that we can show them instead of telling them what can be done? Because so much of this is gonna be like, how do they discover the next use case and the next one after that if they really want to be super powered by the AI. Games, Private Evals, and Show-Don’Tell Vibhu [00:31:19]: Yeah. It’s interesting. I feel like everyone also just has a different way to do it, right? I made a similar version of this same game. I didn’t take any pictures of board or rule game. I threw in at goal eighteen minutes, fifty-three seconds later, a lot of tokens later, I’ve got a similar version. not with all the auto research and whatnot, but Akshay Nathan [00:31:39]: You gotta do all the latest trends. Vibhu [00:31:40]: And yeah, I did it with, did it with Codex, not Work, but it’s interesting, right? Akshay Nathan [00:31:45]: Yeah. And this is GPT Image generating the pro avatars. Very good for game design. Like Vibhu [00:31:51]: And Akshay Nathan [00:31:52]: A lot of game designers were like really into GPT Image for assets. Vibhu [00:31:54]: I will say like the broader takeaway probably is the reason that we do this is more so just to test the tools, right? Like, this was also a test for GPT-5.6 came out. I had done the game on GPT-5.5, right? The ability for me to no longer need it to. I had to feed it the rules. It’s, it’s a pretty niche game. It couldn’t find how to do this on its own. Akshay Nathan [00:32:15]: Oh, yeah. Vibhu [00:32:15]: GPT-5.6 Akshay Nathan [00:32:16]: It is out-of-distribution, which is why I was also very keen on testing the GPT-5.6 capability. Vibhu [00:32:21]: But, this is just as work comes out, as new things come out, these are just our side ways to test things, right? Akshay Nathan [00:32:27]: Yeah. It’s some private eval. That is not this private. Vibhu [00:32:31]: But also valuable because now you can send this to your friends and I learned about this game through seeing this. Akshay Nathan [00:32:36]: It’s a hard game. He’s very good. Vibhu [00:32:39]: It’s good to when no one is competing with you. But yes, it’s a classic RL problem of like self-play, bootstrapping your game AI. yeah, you see how easily work becomes personal and personal becomes work because the thing I do for personal, it directly informs people I work with because I showed it to them. They were like, “Oh, you can do that with GPT?” Which like I imagine is the growth strategy. Akshay Nathan [00:33:02]: Yeah. The show not tell is a big piece that, I think we’ve we’re not still not fully cracked of like, showing people all the things that they can do with the product versus like trying to teach that to them through like, articles or onboarding or whatever. Akshay Nathan [00:33:18]: So meeting them in the moment. Vibhu [00:33:19]: It’s a career risk for me, because I used to be in developer relations, right? Where your job is to show, and then you’re like, “What do you mean? You don’t, you don’t need.” your job is to tell. And then. But the product people are like, “Well, we don’t need you if our product is intuitive enough.” So Akshay Nathan [00:33:37]: Yeah. that’s the magic of the models. So you can tailor the telling or the showing to like specifically what the user needs, like what they care about, what they’ve done in the past, exactly where they are on the adoption journey. So I think that’s like gonna be a super big opportunity. Vibhu [00:33:50]: Seems easier and easier now to tailor custom showing, right? People have different use cases. As much as you said you don’t wanna segment different people into different buckets, right? It’s also not that hard to for people that are in different categories. But the question, is you said your team is more broadly on. What was the term you used? Productivity? From Developers to Knowledge Work to Everyone Akshay Nathan [00:34:12]: Productivity. Vibhu [00:34:12]: Productivity. So how Akshay Nathan [00:34:12]: Which is now work. Vibhu [00:34:14]: Is it work? Is there another distribution that we’re not hitting? Is there a group of people that will have something different than ChatGPT, Codex or Work? Is there more that the mass isn’t targeting? Akshay Nathan [00:34:28]: I see it as like a sequencing, like. The vision is like bring useful agents to everyone. We started with like developers. Like developers historically are like early adopters that are willing to put up with more friction, set things up, et cetera. Like that’s where, Codex started. I think the next opportunity is like what we call general knowledge work, all the other functions around developers. I think when you go from developers to this segment, like there’s inherent challenges with like, this show not tell thing that we’re talking about, making the product more understandable, bringing in new capabilities that matter more for this cohort than matter for developers, things like artifacts, things like computer use, et cetera. And then I think like the same learnings, like similarly how we took the learnings from developers and brought it to, general knowledge work, the next stage will be like taking the learnings from general knowledge work and bringing it to everyone no matter what they’re doing in their lives. And we’re already seeing that a little bit. Like this game example that you have is, something that’s like on the border of like fun and personal life to, your professional life. I use ChatGPT Work full-time at home for everything, like for whatever I’m doing. I used it the other day to come up with a meal plan and like, save that on the like computer environment that it has and something that I can continue going back to. Like is everyone doing that yet? Probably not because the thing says work on it, but eventually, we wanna get people there. Vibhu [00:35:51]: ChatGPT life. Akshay Nathan [00:35:52]: Yeah, exactly. ChatGPT cooking. But I think there’s a lot of, there’s a lot of opportunity there, but I see it as like, we’re, we’re built we built a foundation in software engineering, and we’re gonna take the same learnings that we take from software engineering to knowledge work to everyone. Vibhu [00:36:07]: Do you have any power user advice? I feel like, there’s a group of people that will live it, use it for everything, stay on it twenty four-seven. And then there’s a bit of a gap between that crew and people that, okay, I use it for work. I use it occasionally. Sometimes I type questions. any advice, any learnings, anything you recommend or just, takeaways that you’ve found that help bridge that gap? Power User Advice: Push the Frontier of Imagination Akshay Nathan [00:36:30]: I think a couple things that I’ve seen is like, one, that it really helps to broaden your imagination of what’s possible, and this has been a learning even for me. Like, the technology has progressed so fast that, something that, like, even three months ago, like, no way the models can do this. Like, now it’s like, wow, it’s like it can. Like, Swyx [00:36:52]: Give an example Akshay Nathan [00:36:52]: We’re going through right now our, like, review cycle internally, and, people always talked about this as, like, a thing that the models are good at and like, there’s a cliché of like: Okay, like, no one wants to be writing reviews and, like, we just use AI to do it. But in all seriousness Swyx [00:37:09]: And it can evaluate it as well. Akshay Nathan [00:37:10]: Yeah, exactly. In all seriousness, before it was, like, just, like, slop and, like, I think it was helpful, but, not super productive. Now I’ve found that, like, the model can do a much better job than me, especially in this environment of, like, pulling context on, like, what people are up to, how they’ve like the things that they’ve done to make a difference, highlighting like, wins that they’ve had that, like, I might may not even have seen. It has access to, like, everything, right? Like the code, like, things that they’ve caught, reviews, Slack, everything. And so it’s, like, incredibly powerful in that domain and, like, just like six months ago, the last time we did this cycle, like, I didn’t even I tried using it, but it was not at all helpful. And this time it’s been, like, incredibly helpful and, like, so I think continuing to push the frontier of imagination of what’s possible, even if you tried something before, I think is maybe the my biggest piece of advice. The other, thing is, like, the more you put in, especially in this environment where, like, the model has access to everything on your computer or in ChatGPT Work, like you can create, artifacts over time and save them in your library and, like, the model will continue having access to those. Like, the more information you give it about whatever domain you’re in, whether it’s your life or your work, the more valuable it becomes, and it’ll become valuable in, like, ways that might surprise you. Like, it might pull from context in a way that, may be proactive and that you might not even have thought about. But it needs to have access to those, to that those tools or that context first. Reviews, Agentic Search, and Context Gathering Swyx [00:38:27]: One thing I just wanna talk about the review stuff because I’m still that’s a very sensitive thing and you’re, you’re a founder, you’ve managed people, you’ve hired people. As manager myself, I’m very reticent to put out any LLM-generated things especially when it comes to people, ‘cause it feels like you don’t care. Swyx [00:38:46]: Presumably at OpenAI, people are more open to being eval rated by GPT. But are there any unofficial rules around this? Like, what’s the etiquette? Akshay Nathan [00:38:57]: Oh, I think the etiquette is that, like, I would never write something via, like, well, solely via AI and, like, present it as, like, a review for someone. What I was talking about is more, like, gathering context. That’s the place where it’s incredibly helpful. Swyx [00:39:08]: So it’s just search. Akshay Nathan [00:39:09]: Yeah, exactly. Swyx [00:39:09]: It’s agentic search. Yeah. Akshay Nathan [00:39:10]: It’s like agentic search, but, that you can tailor and steer much more capably than you could before, ‘cause, like, the thing is it’s all there’s a flywheel happening, right? Because of Codex, people are able to do, and because of ChatGPT, people are able to do so much more now than ever before. And if you’re able to do so much more, it’s easy to miss things as well. And so, like, I think we need to use these same tools to keep up with all the impact that people are having and understand, where we can be helpful. Swyx [00:39:39]: I think the thing, like, I run a small company, so easy to search, but at the scale of OpenAI with the amount of messages that you guys put in Slack, do you think that it misses things? Remembering What Humans Miss Akshay Nathan [00:39:50]: Probably, but I think that I also miss things. Swyx [00:39:52]: Like, it doesn’t matter, right? Vibhu [00:39:53]: I think sometimes it’s Swyx [00:39:53]: Like it’s, as it needs to be human-level Akshay Nathan [00:39:54]: It’s all relative, right? Yeah. Vibhu [00:39:56]: Sometimes it’s nice when it finds things you wouldn’t, right? Like right now, my Codex system prompts, they’re set up in such a way that every project I have has a secret- separate, notes MD, and it just writes learnings to there. And then the global one can pull from all these. So sometimes it’ll be like: Oh, there’s this project you did like four months ago. Here’s a note that we had, and it randomly pulls it back into context that I would never do, I haven’t thought about. Vibhu [00:40:20]: And I’m like, okay, this is quite superhuman, right? Like, stuff that would. And, it’ll save like hours on chunking of stuff or find something that’s already been done. I’m like, as much as it might miss stuff, I would too, but it’s very useful when it finds stuff. And I have like a very, non-super engineered solution to this. It’s just marked down files that get pulled whenever they want. Akshay Nathan [00:40:41]: Yeah. I have a funny anecdote about this. Like, recently gearing up to this launch, the team has been, really cooking on it for a couple months, and over that time, like there’s so much conversation and chatter going on in Slack and Docs and elsewhere. And, one of the members of the team set up this, scheduled tasks, like automation to like look at everything that’s going on and, like, come up with the best memes and then post it in one of our shared channels. And like, there are two cool things about this. Like, the first is, like, I think the models are, over time, like starting to become like funny. Swyx [00:41:13]: Funny. Nice. Akshay Nathan [00:41:13]: Whereas like, a year ago, like that was not at all the case. The second is, it was what you were saying, like they find things that in surprising ways that you may not have thought of and like create connections that you may not have thought of. And that really helps with like the meme generation because then you can see something that, genuinely surprises you and, is funny in that way. So yeah, that’s like not like the most productive, use of this the technology, but it does it does uncover this, like this capability that’s emerging, which is just like to find information that you otherwise would not know of. Launch Momentum and the 10 Million User Milestone Swyx [00:41:43]: Talking about the launch, I think, I have pretty much said this is the most successful launch in a long time. I think even more successful personally than 5.0, and they’re announcing ten million users. Does it feel different? You’ve been through a lot of launches. Akshay Nathan [00:41:58]: I think it feels like a culmination. Well, I think two things. One, it feels like a culmination, like I was mentioning earlier, like this like vision mission that we’ve been on for a long time. Like I said, we saw the magic of Codex internally, and then we’re like extremely excited to bring this to many more people and to see it working, to like see us reach, the distribution goal, numbers that you mentioned, like I think that’s like huge and super exciting. The flip side of that is like, there’s so much more to do too. Like, that’s also really exciting. Like, ChatGPT as a whole, like the this product that, everyone almost equates to AI and like loves, has hundreds of millions of users. And so like ten million is really cool, but like we need to get this to everyone. Like, we need everyone to feel this magic. And so that’s the next step from here. But yeah, I think extremely pumped about how it’s going so far and the opportunities. Swyx [00:42:46]: Awesome. I did want to also Because I’ve, I’ve, I’ve been tracking the number closely, it transitioned at some point from just Codex users to Codex plus ChatGPT Work, because they’re same harness. The whole point is that you don’t, you can’t, count them separately. Do you have roughly a billion, ChatGPT users? Why did it just jump to one billion right away? Like, isn’t that the default on ChatGPT or no? Codex, ChatGPT Work, and the Developer Brand Akshay Nathan [00:43:11]: We don’t default you into ChatGPT Work if you’re on ChatGPT Swyx [00:43:14]: If you’re free. Yeah Akshay Nathan [00:43:15]: It’s also only available to paid users right now. And I think there’s like a process of, educating users of what is the value of this product, having them try it, learning from their feedback, and making it better over time. But the goal is to, get as many of the people who love ChatGPT today to like feel the power of ChatGPT Work. But I think it’ll be a journey. Swyx [00:43:36]: Yeah. And Codex will still be alive as a brand for the foreseeable future. And we’ll just toggle between them as needed for UI stuff. Akshay Nathan [00:43:44]: Yeah, I think it’s even stronger point than that. Like, I think we fully intend to like, treat developer. Like, developers have been, a core market for us for so long, and like there’s, there’s so much more that we can do to make Codex great specifically for, software development, and we’ll continue to do that. This doesn’t take away from that at all. If anything, it should increase the utility of something like Codex, because now you can move seamlessly between writing a diff to creating an artifact or, doing a search over your factor. Swyx [00:44:11]: I do wonder how much this terminology leaks to the non-technical user. Like, do they have to learn to say artifact if I want artifact? Or. Akshay Nathan [00:44:20]: It’s funny, like we call it artifacts internally ‘cause that’s what the teams call it. Swyx [00:44:23]: It’s nice. Yeah. Akshay Nathan [00:44:23]: But like externally, like no one says that, no one calls it an artifact. But I think that people like often, like describe things, whatever they’re used to, right? So if, ChatGPT Work is good at creating slides, they’ll say ChatGPT Work is good at creating slides, and that’s what we want. OpenClaw, Personal OS, and Persistent Computers Swyx [00:44:38]: One big Another, it’s July of twenty-six. One big thing that also happens in, for OpenAI was OpenClaw, and that’s I think a lot of people’s first time really maxing a agent for personal stuff, but also crossing over to work in essence same way. As far as I understand, OpenClaw is still independent, but did you go through your own OpenClaw moments? Were there any lessons you took from OpenClaw to Codex or back? Whatever. Akshay Nathan [00:45:06]: I think there’s a lot of inspiration. I did go through my own OpenClaw moment. I, Swyx [00:45:10]: Yeah, tell the story Akshay Nathan [00:45:10]: Me and my wife like set up an OpenClaw to like try to manage everything in our house. Not that there’s like a ton, but it was like quite useful. We gave it a calendar. It started, creating events for us and stuff. At some point, the laptop that we were running on, it died and never got a chance to pick it back up. But there was a lot of inspiration there, like, in ChatGPT Work, in web and mobile, like you get access to this like persistent computer environment where, you can store files, and those files stay around between sessions. And the idea is to be able to enable use cases like this. one of the members of our team uses ChatGPT Work for what they used OpenClaw from before, and then feel like it has like completely transitioned, which is like, workout planning and like meal tracking. which again, it’s like a work-related thing, right? It’s like not work necessarily, but it’s like in personal productivity space. But it has all the same primitives. So it has scheduled tasks. It has the ability to store files on a file system. It has the ability to like reference those things over time. And so you start to see the same types of use cases emerge, which has been really cool. Swyx [00:46:14]: Is there a point that ChatGPT Work completely replaces OpenClaw? they’re independent, so. Akshay Nathan [00:46:20]: Yeah, I’m, I’m not close to it, so I can’t speak to the OpenClaw roadmap, but I don’t think so. I think that there’s gonna be, there’s always a need for like this like incredible, like open source technology that team has built. And I think that we can draw inspiration, in the product and, ChatGPT, I think many more people have like heard about and used ChatGPT than have used OpenClaw. And if we can take the magic from OpenClaw and bring it to them, I think that’ll be a success. I think that like one thing on the ChatGPT Work side that we feel strongly about is that like the core experience is that you come to this product and you have a conversation, start a session, whatever you wanna call it, with this agent. And the magic of the product is that you can do anything in that moment. And we would like to create a product where you don’t have to click a button or to go to a different place, whatever, and you can get whatever functionality exists in, your finances app or where or any other product like in this one place. And so that’s the goal. It’s like it we want an extensible system with plugins where you can connect to the tools that you need in order to be able to accomplish like a financial task, where you can, if you’re doing like science work, like we have an ability to like extend the system in such that you can like write the tech and it performs well. There’ll always be like products that we support that are best in class at those things, but we want as much of the magic as possible in that core experience. Swyx [00:47:45]: Yeah. Do you think that you can do everything you used to do with Wealthfront in ChatGPT Finance? Finance, Data Access, and Centralized Context Akshay Nathan [00:47:50]: I tried it. like ChatGPT doesn’t yet custody, cash and assets for me. So that part, no, not yet. But I, there was like a whole component of like retirement planning and, like financial planning and budgeting and stuff that, we were looking into when I was there. And like with the finances plugin, like that’s all possible with ChatGPT today. So, I feel like at least that component’s replaced for me. Swyx [00:48:17]: I haven’t really plugged it in yet. I’m somewhat scared to look at the answer. Like that’s honestly like the same reason for health and finances. Like I’m like, no. Akshay Nathan [00:48:27]: It’s really good. It’s really cool how we were talking about like the agentic search aspect a little bit earlier, but like, it’s really cool how like, in conventional UX, like if the more power you wanna give to a user, the more like knobs and bells and whistles you need to add. Like, for like these finance and budgeting apps, like there’s always like a bunch of the different filters and like search bars and stuff like that. But like now, like with the right Vibhu [00:48:48]: Connect-connectivity to the right data, you can have whatever you want. You can ask any question you want and into that box and get the answer, and I think that’s super powerful. Akshay Nathan [00:48:57]: I think it’s also nice to just have it centralized in one space, right? You have different health apps. I have one for a smart scale, a watch, all these different things. It’s just nice to centrally co-locate it. Vibhu [00:49:08]: Which is, part of the whole thing of OpenClaw, right? Like that you would have, personal OS, which presumably ChatGPT wants to become. I do think that just relying on, like, just-in-time pulling of data for, let’s say, through via MCP, CLI, API, whatever you do, still not enough. Like I come from a bit of a data engineering background, like you still want like a data warehouse or some caching or semantic layer. do you feel that or do you already have that? Akshay Nathan [00:49:40]: I can’t speak to like all the details on how everything works, but I think it depends on the access pattern, right? Like if you want an answer immediately, then yes, it’s very difficult to do that if you need to pull from all of these sources. But a lot of the like use cases that we wanna enable in ChatGPT Work aren’t necessarily something that you need immediately. It’s more like a task that you want the agent to go and do, and that’s gonna take a certain amount of time. And, with things like programmatic tool calling and stuff now, like some of that time and sub-agents and stuff, like some of that is also parallelizable. And so it’s possible I think it’s very possible that there’s a, the ceiling on what can be done, with MCPs and like calling out to these third-party services has been raised substantially. So we’re really excited about that. Sub-Agents, Ultra, and Product Design Tradeoffs Vibhu [00:50:23]: You mentioned sub-agents. I gotta double-click on that. Ultra is a new mode. You have special affordances in ChatGPT itself to show off the agents. Can’t really do much with them, to be honest. Like just watch. what have been, what have been your experiences, any design issues that you would call out to other builders building with sub-agents? Akshay Nathan [00:50:45]: I think it’s goes back to the balance that I was raising earlier about like, showing builders the power of the tool, but also creating enough of an abstraction to not overwhelm them. I think with sub-agents, the thing that we wanted to show is that you can take a task that, has many parallel tracks or, is complicated in a way that, sub-agents can handle, and this product is for you. Like, the model can accomplish those goals or try to accomplish those goals. And so like that’s the point of like showing them in the product and that’s where we-we’ve gone with the design. There’s another, iteration of this where like you can see exactly what they’re doing and things like that, which I think is like, could converge on like overwhelming, with information. And so this is like the deliberate trade-off that we made for now. Vibhu [00:51:33]: You do display quite a lot of transcripts. Akshay Nathan [00:51:35]: Right. Right. Vibhu [00:51:36]: Or do you Akshay Nathan [00:51:36]: I think it’s hidden by default though, right? Vibhu [00:51:37]: Do you want to display more than that? Akshay Nathan [00:51:38]: No, it’s hidden by default. Yeah. Vibhu [00:51:39]: Some people could want more. So I’m one of those people that will throw a lot of stuff at goal, and pretty much every goal I’ll tell it to use sub-agents. Seems redundant, right? But every time I’m like, “Okay, use sub-agents where possible.” And I have a lot of people, a lot of friends that recommend and do the same. Whereas I’ll sometimes talk to people that are like, “Okay, this is where I want you to use sub-agents for this sub-task,” and I’m sure they would appreciate seeing into how they’re being used. For me, it’s primarily like two things, right? One is net time efficiency, so span out across sub-agents. Two is probably cost, right? Vibhu [00:52:15]: Don’t use big, expensive model. Offload to a lot of smaller, cheaper models. And some people want that level of control. So if you have repetition in what you’re doing, right? Say I want something built where I want it to consistently do this every day, I might wanna go in and fine-tune sub-agents here, sub-agents there. So you can see both, but I think if I’m not mistaken, it’s hidden by default. There’s a dropdown that goes a lot where I’m like, okay I’m just gonna keep, using. Akshay Nathan [00:52:41]: Oh, you can change the model that they use. Vibhu [00:52:42]: I know I tell them to be steered. I’ll say my I know Anthropic offers this in Cloud Code. You can tell Fable to use Sonnet or Opus to use Sonnet as sub-agent, so pretty trivial thing. You tell it to span out sub-agents with Sonnet, it’s cheaper, faster. I would assume if it’s not there, it could be built there. But I think there’s a side of Akshay Nathan [00:53:02]: It’s too many toggles. Vibhu [00:53:04]: It’s not a toggle. It’s just, you tell it in chat. Akshay Nathan [00:53:07]: You’re prompting it. Yeah. Vibhu [00:53:07]: The way I do it is prompt it, right? And I think this is something that gets abstracted unless it’s something you built for repetition, right? So if I’m building something, say that’s, podcast prep, right? Research into people, do a very deep extensive research, that I might wanna configure to cheaper, faster model just for web search, right? I can see a world in which you want both. I think the default is pretty good right now, where it’s hidden, but you can drop down and get some more info into what’s done. Vibhu [00:53:34]: I know people talked a lot about it on GPT-5.6’s launch. this thing loves to use a lot of sub-agents and causes the ChatGPT app to just crash because it’s so processor-heavy. But, Akshay Nathan [00:53:47]: For what it’s worth, that’s not my experience. Yeah, I haven’t had a crash from sub-agents. Vibhu [00:53:52]: I haven’t either. I have We both have big laptops. But I know people brought it up. There was a topic of discussion that we didn’t see the same, but it is another vibe eval, right? People are like, “Okay, the amount of sub-agents Sol is wanting is crazy.” And I’m like, “I think this is okay. I think it’s good.” But just stuff people bring up. Akshay Nathan [00:54:12]: I think when we launched the product too, we weren’t as opinion about like who is Ultra for and like when should they be using it. And since then we’ve made some changes to like, require you to turn it on and find it in the advanced setting ‘cause that’s who it is for. It’s for like power users who understand what’s gonna happen because it also, depending on your use case, can use more of your limits as well. Vibhu [00:54:33]: Yes. Akshay Nathan [00:54:33]: So that’s where I think a lot of the feedback was coming from. Vibhu [00:54:36]: It’s okay. Reset the limits. Always reset the limits. Akshay Nathan [00:54:39]: Well, it’s, today we’re resetting because of this. I wanna change topics to one last piece of the harness, memory. A lot of people are commenting on memory recently. ChatGPT’s new memory system used to suck, it’s not very good. And then this guy also the same thing, and Samir, who you presumably work with Memory, Chronicle, and Personalized Context Akshay Nathan [00:54:55]: Talking about memory. What can you say there? I think that, Samir and the team have made a ton of and then the research teams have made a ton of, updates and improvements over time. I think when I talk to friends, family members about what they love about ChatGPT, like the fact that it knows them, that they feel like their ChatGPT is their ChatGPT, I think comes up probably number one. In ChatGPT Work, in the Cloud, like by default, all conversations like inherit from your ChatGPT memory, so you’ll know they’ll know context about you, and they’ll also be able to write back to this memory. Vibhu [00:55:27]: With it, like a small text write. Like you tell me when you’re writing, right? Is it Akshay Nathan [00:55:31]: No, it’s part of the same like memory V3 system that we launched. Vibhu [00:55:36]: Yeah, Memory V3, yeah. Akshay Nathan [00:55:37]: So I think that’s been really powerful because, going from ChatGPT to ChatGPT Work feels like an extension of what I’ve already been doing with the product for sometimes many years. So that’s been awesome, and it’s awesome to see that like people are recognizing the improvements here. Vibhu [00:55:51]: Is there So it’s a retrieval problem, right? Like, are you retrieving the right things? Are you over-focusing on the wrong things? Is there like a more false positive or false negative, if that makes sense? Like, what’s the bigger problem? Akshay Nathan [00:56:05]: So I don’t work on memory directly so it’s hard to say what the bigger problem is with like certainty. But I think you’re right. I think that like, the there’s two sides of it. It’s like, making sure it knows things about you, but then also having the EQ to like bring those things up at the right moments proactively or surprising you in ways that are positive, not negative. Akshay Nathan [00:56:21]: So I think it’s a very challenging problem, but something that I think we feel very there’s a huge opportunity to get right, which is like why we’ve made like big investments in it. Vibhu [00:56:29]: How do you see the side of, okay, when you’re building ChatGPT for work different than the regular chat app, different than Codex, managing memory across different projects, collaboration and whatnot, how do you see the side of what’s separate from the harness, right? So if I have four threads on one project any learnings on how to build memory systems there? For background as well, to steer it a bit, is when you do chat style applications, I’d say you have a lot of one-offs, right? Vibhu [00:56:58]: When you switch to work it might be something you’re doing for a month, something you do a lot, right? Now, as I add more sessions, there’s a lot more than just single-threaded, right? Vibhu [00:57:08]: And there might be memory there. Akshay Nathan [00:57:10]: I think first I challenge that like the depth of the memory or the like value of it is like fundamentally different across chat and work. Like it is true that like, there are a lot of like shorter sessions on chat, but I think, the ChatGPT, the product has had like a ton of longevity, in, as long as this technology has been around and people use it for work-related, like productivity-related things already today. And so I think we found that there’s a lot of value. I found this my personal usage, like all these one-offs add up over time into something like quite durable and like quite a good representation of who I am. I know like from time to time, something will go viral on X about like, ChatGPT telling you everything it knows about you, and people are always surprised like how deep that is. Vibhu [00:57:55]: The fun roast me? Akshay Nathan [00:57:57]: Exactly. So like, I think like the That’s all to say that like I think there’s a lot of depth there in the existing, ChatGPT product, and so that’s why I think we think it’s valuable to bring into the work product. But the other reason I brought that up is because I think like hopefully we can use some of the same fundamental primitives and systems to extend memory here as well, and I know this is something that the team that focuses on this is like working through right now. Vibhu [00:58:20]: I wanted to bring up one element of memory, which I honestly don’t really use much, and I’m curious if you do: Chronicle, which was, is up on screen right now. It’s a super memory or like what is it? Akshay Nathan [00:58:33]: I think the idea is that like it can learn from, how you’re using your computer and like it’s another input source, into memory. And, I think it’s, experimental right now and something that like isn’t default off. But I’d recommend that you try it. I think that it’s like quite interesting how It goes back to a conversation we were having earlier on like, you were asking like, “Does it Can ChatGPT miss things?” Like does it, on Slack, when it’s searching, does it miss things? ‘Cause there’s such a volume of stuff, right? And like it’I, you can ask the same question about like everything that you’re doing on your computer. Like, is it gonna know everything that you’re doing? Is it gonna capture the intent and stuff like that? Probably not, but like it probably will find things that you might not know about. And then if it can surface those to you in relevant times, in proactive ways, like when you’re doing tasks, and I found at least that it can be quite helpful. So it’s worth trying. Vibhu [00:59:24]: So mostly for insights and longer term. Akshay Nathan [00:59:27]: Yeah, exactly. Like insights and it builds context that makes, that can make you more productive on certain tasks. But it’s, it’s hard to describe without feeling it. Vibhu [00:59:37]: I will say you can feel it pretty well. Like the idea of what they’re saying here, right? Just check through my memories or check through my logs and add skills. Pretty underrated, right? Akshay Nathan [00:59:48]: But that’s automations. You can repeat that using a cron job. Checking through your memories and creating skills. But I think the creation of the memories from Chronicle itself is like what’s different. It’s like you have much deeper memories because you have Chronicle on. Vibhu [01:00:01]: It’s there. I don’t use it much, but maybe I just, I need more examples. I imagine you guys use a lot of it internally, so I’m always fishing for use cases. Akshay Nathan [01:00:10]: I would just try turning it on and then like Vibhu [01:00:13]: It just auto works? Like it Akshay Nathan [01:00:14]: Yeah, and seeing like where it might start helping you. I think you’d be surprised. Vibhu [01:00:18]: Yeah. Amazing. I think that was, about it in terms of like the overall, coverage of ChatGPT Work. I think there’s been a lot of like good progress and discussion on building and all these things. There’s a lot of like ex-founders in the community, in OpenAI as well. Do you think that things have changed a lot? like your overall reflection of building, pre-AI and post-AI. Akshay Nathan [01:00:44]: I think things have changed a ton. I think it’s like super exciting to see how quickly you can go to, from idea to something real today. whereas like even before, like I think, five, 10 years ago, like it’s fast if you were scrappy and, like, willing to build the minimal viable thing. But, like, now the extent of what you can build is, like, much broader. And I think that also, like, what we’ve seen internally building is, like, that gives you an opportunity to validate much more quickly, to talk to users, to talk to internal doctors, et cetera, and, like, make sure you’re on the right track. And, like, that loop I think has been has become more closed than ever before, and that’s, like, a win for product development. I think it’s a win for consumers and users too because ideally that means they’re getting much more better much better products out the gate. Building Before and After AI Vibhu [01:01:32]: Does it mean your teams are smaller? Akshay Nathan [01:01:33]: I think there’s much more to do now. So I think people can accomplish more individually or in a small team than they were that would require more people than before. But there’s, at the same time, there’s also more to do, so I think the teams are much more ambitious. Vibhu [01:01:50]: Have you seen any changes in scopes of roles and building teams and how we used to have teams, say, a few years ago versus what ideal teams look like now? Akshay Nathan [01:01:58]: I think we’ve seen a blurring in the lines between, like, the typical product development functions, like between, like, EM/PM, engineer, designer, et cetera. Like Vibhu [01:02:08]: Yeah, I wanna bring up this quote. There will be, only four jobs left in tech. There’s AI slop cannon, the people who just, like, they’ll burn a bunch of tokens. And then there is SRE, the people who. people who are more responsible. There’s grown-ups who sell things, and then there’s hot people. Akshay Nathan [01:02:27]: This is an interesting take. I think my suspicion is that there’s everything everyone will be, like, shaped in a way, in that, like, AI will enable everyone to become a generalist. Like, things that, like, I never would be able to, like, come up with a design before and, like, even now, like, I don’t have maybe, like, the visual taste required, but I can iterate on something with the help of AI. But then people will have a specialty, and that’s, like, the straight line in the T or the upward line in the T. And so, like, you can have a specialty that you’re interested in. With the help of AI, you can go deeper and become better at over time, but then you’ll also be a generalist. And so with that foundation, the way you can accomplish is, like, almost limitless. Team Shape, Shaped Builders, and Taste Vibhu [01:03:07]: What are you bottlenecked by in terms of specialties? Like, do you need more designers? Do you need more slop cannons? Do you need more hot people? Akshay Nathan [01:03:15]: I think the bottleneck some becomes, like, ideas and taste. I think because anyone can build now, I think, it really is the era of, like, bottoms-up ambition. And because there’s so much to be built, like, you’re always gonna be bottlenecked by, the amount of ideas and amount of things that you’re doing at any given time. Vibhu [01:03:37]: Do you think models help solve that? Akshay Nathan [01:03:39]: Models? Vibhu [01:03:40]: Yeah. I have the example of, like, I have a front-end design skill that’s like, they give me four drastically different examples of what this looks like. Sure, it burns a lot of tokens, but. And then I’ll mostly just condense down, “Okay, I like this part. I like this part. Let’s draw these together.” And it’s like, yeah, I had a vision, but, like, I don’t know. Akshay Nathan [01:04:01]: I would say that the one automation that I would love to work and it doesn’t work is bring me new ideas, right? somehow LLMs are just not it. One interesting part about ideas is, like, they’re not, like, in a vacuum. It’s, like, not. They usually come from somewhere and, like, in product development, like, they’re coming from talking to users or reacting to, friction that you’re seeing or feedback, building on some foundation that you already had planned out before, whatever. And so I think that’s where, like, I think there will always be value in these, like, generalists that we talked about, like, closing that loop and then having coming up with those ideas that are grounded in that feedback or talking to users, whatever it is. Defining and Measuring Productivity Vibhu [01:04:41]: Cool. You were gonna. You lead the productivity team. How do you define productivity? Akshay Nathan [01:04:46]: I think our mission is to make it possible for people to do things that they weren’t able to do before. And right now we’re thinking about it from the perspective of knowledge work. And so when I look at knowledge work, I think about people are no longer siloed by their roles. They’re no longer siloed by maybe the, background or training that they have. Like, no matter what function you’re in, you can suddenly build things. You can suddenly get access to data that you otherwise might not be able to interpret, et cetera. And then I think that extends to your personal life, where we want to give you leverage at the end of the day. Like, we want the models and the product to be able to give you leverage so that you can, create time for yourself to do the things that you love. Vibhu [01:05:25]: Does that also translate to a way to measure productivity? Like, what is new? Akshay Nathan [01:05:29]: The end is Vibhu [01:05:30]: How do you measure leverage? Akshay Nathan [01:05:31]: I think we haven’t figured this out yet. Part of the reason is it’s so diverse. Everyone has different goals, and really the true measurement is, like, their ability to achieve that goal. Did we help you or did we not? Akshay Nathan [01:05:44]: And it’s very difficult without knowing what that goal is up front and also tailoring it for every individual. Vibhu [01:05:48]: And the thumbs up and thumbs down from ChatGPT doesn’t give you anything, right? Akshay Nathan [01:05:52]: You don’t know if they’re thumbs downing the content of the answer, the vibe of it Vibhu [01:05:56]: Oh, yeah Akshay Nathan [01:05:56]: Whether or not it helped them with their goal. I think that’s difficult. But it’s something that I think we will need to figure out and the industry at large will need to figure out because, that’s how we measure success, if this is what we’re, we’re Vibhu [01:06:06]: Do you think it’s changed, productivity and how you measure it? you said there’s a lot more work that can be done, a lot more scope. has it changed? Akshay Nathan [01:06:15]: I think it was always true that what you really wanted to measure is, like, was your team, was the individual, were you personally able to hit the goal, or are you closer to hitting that, whatever your goal is, right? But I think previously we used proxies for this. So, like, code commits or Vibhu [01:06:31]: Lines of code Akshay Nathan [01:06:31]: Lines of code or whatever. Vibhu [01:06:33]: Story points. Akshay Nathan [01:06:34]: Yeah, exactly. Story points. And, like Vibhu [01:06:36]: They’re coming back, by the way. Akshay Nathan [01:06:38]: maybe. But that is for a part of the change. And, like, I think with AI now, those proxies starting to fall apart. Like, you, the number of tokens you use or the number of pull requests you make are, like, no longer, like, maybe as hypercorrelated with that, is your team able to hit the goal or are they on track to hit their goals? So I think we’ll need to come up with new, measurements. Vibhu [01:07:02]: For the managers listening, give them one thing to try. At-Bats, Motion vs. Progress, and Closing Akshay Nathan [01:07:06]: I think for me, what’s important is like at-bats. Are we as a team building the muscle to have not just quantity of at-bats, but quality? Like, are we able to go all the way from, like, generating an idea, building it out, getting the feedback, reacting to that feedback, validating or invalidating the hypothesis, going on to the next idea? Are we able to do that really efficiently? And like, that goes to like, the actual like code that’s being written or the designs that are being made or the specs that are being written, whatever, but also the culture of the team. Like, do we have the humility and, are able to like go through that process many times and stay motivated and excited throughout that? so that’s the thing that like I think is important now, especially when we’re on the frontier of this technology and like there’s so much to build, there’s so much to do. That’s probably the most important thing that we look at. Vibhu [01:07:54]: Any traps people fall into around measuring productivity with your teamwork on. I feel like there’s a lot of, okay, we added a lot of LMs. We have dashboards for this and that, but not much has changed, right? Akshay Nathan [01:08:06]: That is the trap, yes. Vibhu [01:08:09]: And the broader source of the question is for the managers and teams building, how should they approach this? Akshay Nathan [01:08:18]: I think maybe the trap is like conflating motion and progress. I think motion is much easier now than ever before because of the tooling that we have. But progress requires you to be like very prescriptive and deliberate about like what you’re trying to achieve, and it goes back to our question of measurement, right? Like you wrote we were talking about like, can we, OpenAI, like figure out how to measure productivity for our users? That’s, that’s a very hard problem because of the diversity. But like as a team, like you should have a really prescriptive and deliberate view on like what progress looks like for you and for your team. And if you don’t have that, then it’s very easy to conflate these two things. Vibhu [01:08:57]: I think at-bats is a really great thing. I’m, I’m really glad. I like the discussion between motion and progress. I think that’s a quote that we’re gonna feature on the write-up. You’ve been very generous with your time. Thank you so much and congrats on ten million. Akshay Nathan [01:09:08]: Yeah, thank you for having me. Vibhu [01:09:09]: The next one at a hundred in two months. Two weeks. Thank you. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe https://www.latent.space/subscribe?utm_medium=podcast&utm_campaign=CTA_2

• 0:03Speaker 00

  Okay, we're here in the studio with Akshay from OpenAI. Welcome.

• 0:07Speaker 01

  Thank you.

• 0:08Speaker 00

  And with our trusty co-host Veebu, we... So you recently launched ChatGPT Work. You lead core product engineering. You know, it's been a long journey into all this. I find it very interesting that you started with no code or low code with Walrus and Airtable. And to some extent, ChatGPT Work is kind of like the super app of super apps of, well, here is the ultimate no-code. You just write a prompt.

• 0:32Speaker 01

  Yeah, yeah, it's funny how things come like full circle. I mean, I think for a long time, my career, I mean, I started my career working in consumer fintech, but then after that, like, there's this hypothesis that, you know, the things that we were able to do with code, like, as engineers, like, if we could bring that to many more people in a more accessible way, then that would be truly magical. We were working on a startup. It's actually funny, like, before LLMs, before vision LLMs, on how to do automated testing with AI. And it was just kind of jank back then, but, you know, doing what we can. And then, and then worked at Airtable for a while on, you know, the same thesis that, like, if we can bring a database or the primitives behind a database to people, that'd be really useful

• 1:14Speaker 01

  to them. But once, I think, LLMs came onto the scene, it became clear that, like, this was like the missing piece, like, the missing technology required to, like, bring the magic of code to everyone without them having to know what's going on underneath the hood. And so, like, I think this launch and, you know, a lot of the stuff that we've been up to is, like, the manifestation of that.

• 1:33Speaker 02

  How was stuff when you joined? So you joined OpenAI 2023. Now we've got, you know, so much more stuff. So ChatGPT Codex App, ChatGPT for Work. Have things changed?

• 1:44Speaker 01

  I actually think the more interesting thing is how things haven't changed. Like, I guess, like, one, I joined, I remember when I joined, it was like 500 people. One thing I was worried about was, like, I was looking for something, you know, more early stage, and like, was it gonna feel startup enough? And I joined, I was like, this feels even more startup-y than I could ever imagine. And like, that, that really hasn't changed even till now. I mean, I think the, like, level of, like, bottoms-up ambition and, like, the ability of anyone to, like, you know, do anything or have an idea and ship it is really cool. But on the, like, sort of mission side, I think what was really compelling to me is this mission of, you know, bringing frontier intelligence to everyone, like, building AGI and then bringing it to everyone. And I think acknowledging back then that, like, that vision is gonna, you know, not be a linear progression, like, we're probably gonna,

• 2:28Speaker 01

  like, try different products and how different things that succeed and don't. But the vision has stayed the same and the mission has stayed the same. And we're starting to see the pieces fall together, and that's really cool.

• 2:40Speaker 00

  You worked on enterprise. What, a lot of people never touched ChatGPT for enterprise code. What is something that you learned from there that you're bringing into your work now?

• 2:52Speaker 01

  I think how there's no, like, one-size-fits-all solution in enterprise. I remember in the early days of ChatGPT Enterprise, like, we would talk to customers and, like, everyone, that was like when, I think it was a year after ChatGPT was released, and everyone was so excited to bring, you know, AI into their enterprise. And like, there's all these teams stood up that were being stood up, as like, you know, the AI deployment team, like, these enormous budgets. And if you ask anyone, like, what were they excited about? Like, what were they excited about solving? Like, at first, you'd get, like, you know, kind of like the baseline answers of, like, we have all this context and data and all this stuff. But then if you ask them, like, you know, what was like a discrete use case that, like, they want AI to enable in their, in their workplace? You get such a different, like, variance, like, explosion of different types of answers. And it's

• 3:36Speaker 01

  interesting, like, you know, you using all these models and these products, you have this box and you can say anything to it, which is the magic. But it's on the flip side, it also means that, like, you don't know what to do with it. And in enterprise, I think a big part of that is, like, actually meeting the users where they are, like, what use case were they trying to solve? And then actually teaching them how they can use AI to, like, gain leverage there.

• 3:56Speaker 00

  Do you meaningfully differentiate that from forward-deployed engineering?

• 4:00Speaker 01

  Or? I think there's, like, the, like, go-to-market side of it, and then there's, like, the product side of it, I think.

• 4:05Speaker 00

  Good new.

• 4:06Speaker 01

  You need to see more in the product side, yeah. And I think, like, however good we get at FDE Motion, like, I think, at the end of the day, if we have a user who's, like, looking at their computer or looking at their phone, like, it's our job in the product to, like, be enabling them and showing them where to go. So really excited about that.

• 4:24Speaker 02

  Do you think there's been changes, you know, over the past three years of adoption? So there've been, you know, step function changes, you have reasoning models and whatnot. There's still the same problems of enterprise has black box, so know what to do with it? Or have things changed?

• 4:39Speaker 01

  I mean, we're seeing now that, like, there's this huge update, right? Everyone's extremely excited about it. It feels like, you know, many people, like, millions, hundreds of millions of people are using ChatGPT. They understand, like, how generally to work with AI. But then, like, every time, like, a new capability gets unlocked. So now, like, we're seeing with agents, like, there is probably a contingent of, like, early adopters still who, you know, truly get it, where, like, you know, you can do anything, you just have to make sure the right context is there, it's connected to the right tools, and then that you're supervising it, but, like, anything is, is possible. But then there's, like, this, like, 10x or 100x bigger market, or, like, they don't yet get that or they don't yet see that. And so I think that's the next stage here. So I guess to answer your question, like, I think the adoption is there and growing fast, but I think the

• 5:23Speaker 01

  opportunity is, like, far, far bigger than that. That's where we want to play, especially with ChatGPT Work.

• 5:27Speaker 00

  Yeah. Well, let's, let's skip ahead to ChatGPT Work, only, like, a month ago or so, announced. What was the sort of decision process that led into it? You know, there was this overall merging of the super app. Is that what we're officially calling it? You deprecated the browser as well. Just, I guess, summarize your last, like, couple of months of working on this thing.

• 5:50Speaker 01

  Yeah, it feels like forever now, but I guess it's only been a few months. I think maybe the one, one, like, impetus that, like, is most salient is when we released Codex or even internally had Codex. Like, it was really surprising to us. I think we recently put out some stats on this, that there's this, like, real inflection of, like, adoption among non-developers at OpenAI. And I, you know, through this product development process, like, would go to, like, these UXR sessions, talk to people internally. And the thing that stuck out to me is, like, one, like, you know, you go talk to, like, strategic finance or marketing or whatever, and they're all using Codex for, you know, their use cases. That, that part's cool. But the thing that really stuck out to me is how proud people were that they were using Codex, like, how, like...

• 6:34Speaker 00

  It's like, I'm not supposed to be using it, but I am.

• 6:36Speaker 01

  It was that. It was like that they were, you know, early to this, like, new thing. But it was also this thing of, like, they felt like they had a superpower, right? And what we recognized then is that, like, the, the power of Codex, the power of agents, like, we already had this massive distribution base of people who have, you know, come to know and love ChatGPT. Like, how do we show that to them? Like, how do we bring it to them? Which is like a hard product problem. It's like a tricky thing, right? There's many ways you can go about it. And so that's all we called the merge and the super app over time and, and ultimately launched in ChatGPT Work is how do we do that? But it came from that initial realization that, like, the, the power was not only for developers, like, much, much earlier than probably even we thought, like, it can be extended to, to everyone.

• 7:16Speaker 02

  How do you see the products differently? It's like, who is it for, right? So Codex started out even CLI, then app, now there's a merge of ChatGPT Codex and ChatGPT Work. So is it the opening for the average user, for enterprise, for work? How do you position it?

• 7:36Speaker 01

  I think we want to get to position it for if you're doing worky-related things, for lack of a better word, right? I think productivity is like, is, it's actually what, like, the pillar that I support, like, that's the name of the team. And the reason for that, the reason we call it productivity and not, like, you know, enterprise or, or, like, work or something like that is because there's also personal productivity, right? And like, I think ChatGPT Work is, I've seen people do things in their personal lives that you wouldn't classify as, like, work technically, but like, these agents are, you know, super capable for. Like, one, one recent example that someone posted about on our Slack is, like, someone has, like, a missed package, like, they didn't receive it, and then they got, like, the picture of it, you know, from Amazon or wherever the courier was. And they, like, asked ChatGPT Work to, like, find out where that package is. And,

• 8:19Speaker 01

  like, the agent, you know, is extremely tenacious and, like, and, like, took the image, like, looked at a bunch of, like, listings around their neighborhood, and figured out exactly the apartment complex in which the package was, like, gave them some information. And so, like, I think there's all these things that, like, you, you know, worky or productivity-related things. I think that's what we want the product to be. You asked about Codex. I think, we think Codex is, you know, a durable brand, but we have a principle that, like, the user, you know, we don't want a user to get stuck in a tab or an experience where they don't get the power of the product. And so, like, basically, everything that you can do, you know, in the Codex portion of the product on, on desktop, you can do in ChatGPT Work and vice versa. But we made some opinionated product decisions on, like, you know, how much of the get state, if you're in a get repo, do we want to expose to the end user?

• 9:04Speaker 01

  Or how much do we want to make the, the experience of seeing the agents thinking, like, diff forward so that you're getting exposed to the diff side of the buy? And then, like, on the safety side, like, how do we want to think about, like, sandboxing and making sure that we have the right defaults in one, one state versus the other? So there's, like, there's some opinions that go behind that, but we do, we do want, we don't want the user to need to choose which experience they're in.

• 9:26Speaker 00

  That is a good goal for AGI, right? Like, people don't want, like, to hide, to choose what version of AGI they want. They just want the AGI to decide for them. Can I get an answer or, like, it's not super clear to me. Is the Codex harness and the ChatGPT Work harness the same? Is it just UI affordances? Or are there actually prompt level or even, even deeper differences?

• 9:49Speaker 01

  So the harness is the same. The harness is shared. On, in both of the products, we made improvements to the harness to make it good for knowledge work, especially as it relates to plugins or computer use or artifacts. You get that power regardless of what your experience you're in. On the UX side, there's opinionated takes that we have when you're in Codex mode, what the UX should be, how the UX should behave, and some stuff around the sandbox that I mentioned. But the underlying harness and capabilities should be the same.

• 10:16Speaker 00

  I feel just kind of curious, maybe we can, is there a query that we can run that would look different in the, the two modes?

• 10:23Speaker 01

  Yeah, I tried to create, like, ask it to create, like, a retirement calculator or spreadsheet or something in, in both, in both modes. And then in Codex mode, you might have to be in a, in a repo for this, but you'll see, like, the diffs of, like, the, the sheet that it's creating and stuff like that, and the file edits. But in RAC, you won't be able to see that.

• 10:41Speaker 00

  I think that's, that's super clear. And then also, the other thing I wanted to dive into was your, the productivity team. What else is there? First of all, you know, what, what are the top-level teams other than productivity? Isn't productivity everything?

• 10:54Speaker 01

  So, you know, we have a team focused on, on ChatGPT, like, the core chat experience for consumer, which is, like, you know, not, I think... All productivity, like there's, people are using chat GPT every day for search to, you know, figure out how to write messages to loved ones, to think about how to like learn a new topic, et cetera. And so there's so much more inside to create images. There's so much more in chat that, you know, the hundreds of millions of users are using that, you know, obviously that warrants like a very dedicated effort. And there's teams focused on enterprise and infrastructure and API and stuff like that as well.

• 11:33Speaker 02

  I will bring it up. Yeah. So I have them both running.

• 11:35Speaker 01

  Yeah.

• 11:35Speaker 02

  This is work. There's a Codex version here. I picked five to six souls, so this will take a while. I think, I think we'll just keep it in the background and, you know, as, as they finish, we'll look into some of the differences.

• 11:48Speaker 01

  Yeah, but immediately, I think if you flip back to the Codex version, you'll see that.

• 11:53Speaker 00

  That assumes, it assumes Git.

• 11:55Speaker 01

  Exactly. They're like dynamic island assumes that you're in a Git repo.

• 12:00Speaker 01

  And you might miss some stuff because some of it is like in the actual chain of thought with changes and how we just play it up.

• 12:06Speaker 00

  But is, is there an intuitive, like, is there a thing that you wanted to ship and then you got feedback and you were like, no, let's not do it? Like, what's the thinking behind that?

• 12:14Speaker 01

  In chat GPT work?

• 12:16Speaker 02

  Yeah.

• 12:17Speaker 01

  I think one direction we could have gone with this is like keeping the experiences like completely separate. So it's like, why?

• 12:22Speaker 00

  Different apps.

• 12:23Speaker 01

  Exactly. Like different apps or even in the same app, like different, completely different experiences. Like why merge it all? Like what is, you know, Codex, obviously people love, like why, why bring these products together? And I think the intuition here is that like, all of our jobs are like changing dramatically with AI. Like for, I mean, every few months, like I feel like I wake up and I'm like doing a completely different thing than I was doing a few months ago. And my, my hypothesis here is that, or I say our hypothesis is that like, part of what we're, we're building in this, this technology is giving people leverage, like, you know, the things, maybe it's the more mundane parts of your job or, or parts that like, if you were able to automate, you'd be able to share more ideas faster or whatever, like you're able to do now.

• 13:01Speaker 01

  And because of that, like, that might actually blur the lines between someone who's like only writing code or creating strategy docs or, you know, planning events or helping with marketing or doing podcasts or whatever, right? And so like, these things are going to get blurred over time. And so like trying to draw a hard boundary based on like the who you are is, is going to be, is going to be tough. And like, we should enable users to choose, but we shouldn't box them in. And so a lot of the work that went in here, like, you know, keeping the primitives the same, like for example, plugins or like unified across this product and chat GPT in the cloud was because of that. It's this thesis that like eventually things are going to come together. And, and we don't want to be, like, we want to be prescriptive about when to be in either experience, but we don't want to box anyone in.

• 13:45Speaker 00

  I wonder if there's users who are very tuned to the old chat GPT harness that is effectively now replaced by the, the Codex harness. I can't imagine what that was, but maybe they're more, the more conversational side. Can you compare and contrast the, the two harnesses? Cause only you've seen it.

• 14:02Speaker 01

  Yeah. I mean, I think chat GPT, the, the existing harness like still exists. It like exists in this app.

• 14:08Speaker 00

  The classic, right?

• 14:09Speaker 02

  You just start a new chat and you don't go under at work, right?

• 14:12Speaker 01

  Yeah. If you start a new chat and go to chat, then you're, you're talking to chat GPT with the system model.

• 14:17Speaker 02

  Do another. I guess, you know, on instance.

• 14:21Speaker 00

  Yeah. So this one's not going to code or it's going to be inline. It's not in line with sandbox.

• 14:26Speaker 01

  Yeah. Actually, we try to push you to, to go to work if you're creating a spreadsheet.

• 14:30Speaker 00

  This is a router decision.

• 14:31Speaker 01

  Sorry?

• 14:32Speaker 00

  It's a router decision.

• 14:34Speaker 01

  This is the decision that, you know, the model is making, then like, you know, it sees that you're able to, or you're trying to do something that would be better served in work mode. But I think your question was like, what are the advantages of like the, the chat, like chat GPT chat harness?

• 14:48Speaker 00

  It's more broadly like, I wanted basically do an oral history of harness engineering. Right? You know, the chat GPT harness lasted us from, let's call it the O one era until now. And now it's being replaced by the Codex harness effectively, and they're, they're overlapping somewhat. But I'm curious what changed if, you know, if there is.

• 15:10Speaker 01

  My perspective on this is like, there's, there's all, there's sort of like a, a constant process of like divergence, convergence, divergence, convergence. And in chat, like many of the use cases I was talking about before, like, you know, search or learning. I think we're really optimizing for latency and optimizing for personality and like different things that, over time, like the product, the reason people love chat GPT is because we've been optimizing for those things, working on them for so long. Codex, what we learned was that like, if you give the agent access to this infinitely flexible environment as a computer, can do really, really powerful things. And so when we think about like, okay, well, for, for knowledge work, like what is, which mode should we choose?

• 15:55Speaker 01

  It was like, it felt more natural to us to bring that to this like computer environment. And, you know, maybe abstract some of the details, this computer away from users who might not be used to that, but like give them that same power. But ultimately I think that we want the power in all places, right? We want to mute people where they are. So I'm sure there'll be work down the road in order to, to get things to be equivalently capable in, in all scenarios. But it's just a question of like, what we've been focusing on the product on historically and what we're focusing on now.

• 16:24Speaker 02

  I think alongside that, outside of just harness and when to use Codex chat GPT or work, there's also the new models you've released, right? Any guidance there? So people love to min max what to use, like only use Terra on high reasoning versus for this, you know, you want to use soul here, ignore all these.

• 16:44Speaker 00

  32 options.

• 16:45Speaker 02

  Yeah, yeah, yeah. But that being said, you know, for people that are expanding, so productivity, trying stuff for work that don't have the breakdown of what all this is. What's, what's the advice, right?

• 16:59Speaker 01

  Well, I mean, I think before the advice, like the first thing is like, none of this would be possible without these models. Like I think you asked earlier, like, you know, what was like the inspiration for work? And like, you know, early on, like I, I mentioned like what we were seeing with Codex, but that was also because the, the models were getting infinitely more capable. That's happening again. I think it's like another step function jump now. And to answer the question on advice, like we want this default to be the best possible. Like we want to be opinionated about the default. And so we've, we've chosen a default that we think is going to be the best for everyone. And, you know, we have for power users, options under the hood. We could, one could argue that there might be too many right now and we're, you know, working on simplifying it. But you can extend, you know, the reasoning level and you can change between the different model classes if you need to.

• 17:43Speaker 01

  But the default should be the best for, for most, most use cases. So my advice to most people would be to stick to that. And then, you know, if you reach a situation in which you think that you could, you want to try a different configuration. If you're not seeing either the, the efficiency on the, on the cost side or, or the, the quality on the intelligence side, then you can change the default and see if you can get something better. But, but we think that the default should be good enough.

• 18:09Speaker 00

  I have, I, I'm just going to run something by you since you, you have way more experience than me. I've recently been doing soul light, but with goal. With the idea that the goal basically augments the reasoning effort, but with more terminations and turns. Is that a good way to think about it? As opposed to soul ultra or soul, you know, extra high.

• 18:29Speaker 01

  Yeah. It's hard to say because.

• 18:31Speaker 00

  It's like an interaction effect.

• 18:33Speaker 01

  Exactly. It's like, there's a preference on, you know, for you as an individual, like how do you like to collaborate with the models? Like how many of those like terminations, as you call them, do you want where, you know, you can steer or make sure that it's doing the right thing? I think generally people should try whatever works for them.

• 18:53Speaker 01

  I think that like using ultra or the like multi-agent setups are best for like, when you have like tasks that are either incredibly complicated, like open, open expirations or very parallelizable. I think even for tasks, using goal, I think is best for, for tasks that you know that you'll be able to make consistent progress in a way that's verifiable over time. But I think for most tasks, they actually don't fall into either of those buckets. And so like, at least when they're starting. And so that's why I think the best first step is like trying it with the, the default configuration and then seeing like where you want to go from there.

• 19:29Speaker 00

  Right. You guys worked on a slider, which actually is super helpful for reducing a amount of panic.

• 19:35Speaker 01

  Yeah.

• 19:36Speaker 02

  It's nice on mobile at least. There's a nice slider. I haven't tried it.

• 19:40Speaker 00

  So you, you, you have, you have the advanced view there, but if you click advanced, yeah.

• 19:44Speaker 01

  Yeah.

• 19:45Speaker 02

  Just a simple slider, yeah.

• 19:46Speaker 00

  Very, very pretty, very colorful.

• 19:48Speaker 01

  Yeah, that idea was here was like reduce it to like one dimension, even though there's multiple dimensions, right? Try to project it onto a single dimension for the user. Like, you know, something from that represents like, you know, speed and efficiency on one side. And then like, sort of like quality and thoroughness on the other side.

• 20:04Speaker 00

  I am just puzzled that it uses Sol so much, like the lower.

• 20:07Speaker 02

  No, no, I think the slider, if I'm not mistaken, is.

• 20:09Speaker 00

  Terra.

• 20:10Speaker 02

  Oh, it is.

• 20:11Speaker 00

  You see? So they preset Terra to only be the light one. But like, I think a lot of people actually would, more people should use Terra. One, because Sol keeps running out of capacity.

• 20:22Speaker 02

  I'm the reason, you know, here's 10 minutes of our.

• 20:24Speaker 00

  There you go.

• 20:25Speaker 02

  Retirement calculator.

• 20:26Speaker 00

  Oh, that's the Excel thing. Oh, my God, look at that.

• 20:29Speaker 02

  This is work and then Codex is still cooking, so we'll get back into it. I think it'll be interesting to actually see the thought process, the reasoning. And also, you know, I guess this is eight minutes on work. Codex is still cooking.

• 20:41Speaker 00

  Yeah, and by the way, I, so I've, do you know Gabriel Chois? He's part of the OpenAI support team. He showed me this and I was like, pretty shocked that this looks like Excel. It edits Excel files. You never paid an Excel license, right? Like, but somehow this is like kind of workable and it's agentic Excel.

• 21:01Speaker 01

  Yeah, I mean, one of the big like pushes that we made for this launch was like artifacts. Both on the, on the model side, like I think if you compare this with 5.5 and 5.4 before that, you'll see that there's been pretty dramatic improvements in the quality of these artifacts. And then also on the product side.

• 21:16Speaker 02

  The UX side is also crazy, like hosted sites and whatnot, no, no longer needing to host your own little web page, like.

• 21:23Speaker 00

  Oh, I have a story about that. I can, I can do a separate thing. I'll need to take the, the visuals here, but we'll, we'll cut to that later. Was there co-training, I guess, because you were moving, making this big move? And you launched 5.6 on the same day as ChatGPT Work? Was there influence between the model training teams and the harness teams? Or did they, the launch days just happened to line up with the same day?

• 21:46Speaker 01

  I think the, you know, we collaborate heavily with the research teams. And I mean, I think that's like one of the most magical parts of the job, like the most fun parts of the job. But yeah, I mean, just, just using artifacts as an example, like, you know, a lot of what you're seeing, like underneath the hood, there's a lot of work that went into making sure that like, you know, we had the right infra to be able to train the models to get better at this. And then on the product side, like had the right experience for users to be able to collaborate with the model on an artifact like this. In fact, like this whole viewer, like the intuition here is that like, you know, it's not necessarily that you wouldn't need an Excel license. This is stage one, right? Like, this is probably not what you meant when you're like making a retirement calculator. You want to iterate and like, when you, when you're seeing it and if this thing is high fidelity to like

• 22:28Speaker 01

  what you'd actually see in, or what your coworkers would see if you were to send this a shot. Like, that, that I think makes it so easier and makes you trust the, the product in terms of iteration.

• 22:39Speaker 02

  When you say coworkers would see, do you see a multiplayer, multi-team collaboration with artifacts? Any, any things you guys think about?

• 22:46Speaker 01

  You can already share it, right?

• 22:47Speaker 02

  Yeah.

• 22:48Speaker 01

  Yeah, it's something that, you know, we're actively thinking about. One thing that, you know, we've noticed internally without talking too much about the roadmap is that like I, there's many times when someone will ping me about something and I will ask ChatGPT Work the question and then I'll ping them back the answer.

• 23:04Speaker 02

  Like the simplest would be, you know, the three of us are just all on one hosted.

• 23:07Speaker 01

  Exactly. And I'll think about like, you know, like, was I required in this loop? Or, and then maybe it was, you know, rephrase like what they were asking or pulled from certain context or whatever, but like, you know, when I gave them back the answer, that process was also lossy, right? Like I gave them just like my interpretation of what ChatGPT Work cooked up. But like underneath the hood, there's so much context like in the rollout and stuff that, that could be interesting.

• 23:28Speaker 00

  So like the answer was preemptively respond to every inbound request?

• 23:33Speaker 01

  No, it's just like literally like this is what I do sometimes as my job.

• 23:36Speaker 00

  I know, you copy paste and then you, you know, you're just a message forwarding service from AI to AI.

• 23:40Speaker 02

  But I think it's interesting, right? It helps people understand the capability of what you can ask and delegate. That oftentimes people don't realize until they try or someone shows you and then you're like, oh, okay, okay.

• 23:51Speaker 00

  Yeah. I think it's also, there's also like a light security issue where like basically you're the permissions layer. Like, yes, I could query everything that you query and I could get an automated response, but maybe I'm not supposed to see it.

• 24:02Speaker 01

  Yeah.

• 24:03Speaker 00

  And then, yeah, there's no way I would know because I'm not supposed to know what I don't know.

• 24:07Speaker 01

  Especially as like, you know, with ChatGPT Work or asking you to connect your plugins and, you know, it's pulling from your local files and stuff like that. Like the, the amount of context that the agent has access to is like deeply personal. And like, that's something that we need to preserve. So that'll be definitely a challenge.

• 24:22Speaker 00

  There's Excel, there's PowerPoint, there's Docs, you know, the grand trio of work. What other formats of work do you, do you think about? You know, like obviously you worked on Airtable. Is there a future where there's like OpenAI Airtable? Like, you know, like what, what, what does that look like if, if you ever ended up doing it?

• 24:41Speaker 01

  It's a really good question. I think, um, I mean, one that you didn't bring up was sites. And I think that was a core part of this launch. There's one, one side of sites that I think people commonly talk about, especially on Twitter and stuff, or X of like, you know, this sort of like prototyping tool. And actually like we saw that happen with this launch, even the, the model slider that you guys were referencing earlier, like that was developed almost fully in a site. Like, you know, the, the collaboration between design and engineering and product on that was like on a site where we play with, you know, the, the affordance and then figure out how it feels and, and all of that. But the other aspect that, that I think is a little bit less talked about is like sites as like an artifact for, for knowledge work. I was actually talking to someone the other day who was on like our corporate finance team.

• 25:26Speaker 01

  And like, they're mentioning how like now, when they have these reports that they're, they're working on as a team, month to month, historically, those things were in slide decks and in spreadsheets. And now they're just in sites. Like sites is the mechanism that they collaborate across the team. And the reason is because it's like somewhat higher bandwidth, like, you know, at, these tools like PowerPoint and Excel are like infinitely flexible, but at some point you reach the boundary of like, either as a human, you may not know how to use some feature or something, or the product itself doesn't support it. But with a site, you can kind of do anything, you can ask for anything and you can get that. And once people see that magic, I think it's been really valuable.

• 26:02Speaker 00

  Yeah, let me show you my, my case study. This involves all the hot topics, including ChatGPT Work, but also 5.6, token billionaires and token maxing and sites and auto research. I'm a fan of this game called Strata. It's basically, it's like a little board game that you, that you play with physical blocks. They come on top of it like that. So over the weekend, I, I took like 30 photos and just threw it into ChatGPT. 1.7 billion tokens later, out comes this site with a fully playable thing with, with, with 3D block placement and everything. Because it requires physical blocks and I needed friends to train on it so they can get better, so I can play against them. But also, I could also do things like train an AI on it. And that, that's.

• 26:45Speaker 02

  That's your auto research.

• 26:46Speaker 00

  That gets into auto research. So you want to train your own AIs and then make sure they self-play against each other. I need to set both to AIs. So this is AI versus AI and they're, they're going to self-play. Obviously that the AI start out bad and then you want to define a loss function and get good. I wasn't going to supervise all this. I was at, I was at Doudou San Mateo, attending a conference. What I ended up doing was auto researching and, and on this and creating benchmarks. And there was just way too many parameters for me to, to read. So I started asking it for a site and it's created this, uh, this, this, this lab panel. Where, is there, is there, is there a shortcut for a site that is created?

• 27:29Speaker 01

  You should be able to go in the sidebar to sites, top of the sidebar, the left sidebar.

• 27:33Speaker 00

  This one? Oh, left?

• 27:34Speaker 01

  Yeah, just scroll all the way to the top.

• 27:36Speaker 00

  Oh, oh, it says sites. Oh, there you go. Yeah. So it creates, it creates the sites. I don't, I don't think this is, it is exactly what I, what I wanted. But let me, let me show you what it popped up, right? Like, I think as a, as a research artifact, it is very important to communicate exactly what is being done. I'll put this, uh, this thing which I eventually started publishing. So I moved it off of sites because I wanted more database and infrastructure than sites afforded me. But this is, this is like a research output that you can start to mess with and like try to think about like what hyper parameters are you tuning for training your AIs? And like, I was trying to make like scaling laws and everything and doing all sorts of like game optimization stuff.

• 28:18Speaker 00

  And the fact that you can just kind of fill this up as a research artifact, like I no longer need to read ChatGPT output, I read site output. But then there's also a huge sprawl, like look at how long this thing is. There's so many numbers, it is pretty overwhelming. So then I have to start putting it from there. But it's an interesting transition from Markdown. Yeah. Actually, that you're putting out to, you're putting out a whole functional site.

• 28:41Speaker 02

  I think Markdown just isn't that optimal for people to read, right? Might as well just write HTML website and, I don't know, I think you can do a lot with customizing this, right? You have your skills that explain what you want. Like, I noticed they're quite verbose. I don't need a lot of this information.

• 28:57Speaker 00

  It's very verbose.

• 28:58Speaker 02

  So, and then the nice thing of having a site side by side is, you know, you just iterate on what you want and what you don't, right?

• 29:05Speaker 00

  Yeah, I don't know if any, that, that triggers any stories for you of how it's run internally. Am I doing this right?

• 29:10Speaker 01

  Yeah, I mean, I think that this is like a workflow that we're seeing, like all different types of teams use. Where like the canonical artifact that was previously a deck or something is now becoming a site. And like with a site, you, because it's just HTML, you can like, it's infinitely flexible. And so, you know, if you, if you want to give more prominence to a certain thing that like in a slide deck would, you know, feel like it was buried, like you can do that. You can have it be like the hero image, right? And so I think that like, um, people are starting to see that. There's obviously more work to be done to make these things like much more easier, easy to collaborate on. You mentioned that they're very, they're long and verbose, could be broken up. I'm sure there's.

• 29:52Speaker 00

  They're super long.

• 29:54Speaker 01

  Yeah. Yeah, but I, I think we're starting to see that like there, there is this aspect of, this is a really interesting, uh, format for, for people to use. That's like much more flexible than, than what they were had before.

• 30:07Speaker 00

  I think your job also comes, becomes kind of meta. You're not designing the products, you're designing a product to make products. And I'm curious how you manage that.

• 30:18Speaker 01

  I, I think one thing that we've been. Like when we look at the UX like that we've been thinking a lot about is how can we balance like simplicity with capability? Like if we if we're designing a product like you said, that like is is made to make up, build other things, right? You can build so many different things, but we can't put that all in front of you because you'll get overwhelmed.

• 30:41Speaker 02

  Yes.

• 30:41Speaker 01

  And so we had similar problem or similar challenges even chat with chat GPT. But especially now, like when there's so much that can be done. I think the balance that we're constantly trying to strike is like, how can we give the user enough of a UI surface where, you know, they can be expressive? They can they can tell the the agent what they need. They can verify that it's using the right tools, it's pulling from the right sources, et cetera. But then it gets out of the way. And then how can we build the right system such that we can show them instead of telling them what can be done? Because so much of this is going to be like, how do they discover the next use case? And the next one after that, if they really want to, you know, to be superpowered by AI.

• 31:18Speaker 02

  Yeah, it's interesting. I feel like everyone also just has a different way to do it, right? I made a similar version of this, same game. I didn't take any pictures of board or rule game. I threw an at goal eighteen minutes, fifty three seconds later. A lot of tokens later, I've got a similar version. Obviously not with all the all the research and whatnot, but you know.

• 31:39Speaker 01

  You had to do all the latest trends.

• 31:40Speaker 02

  And yeah, I did it with uh did it with Codex, not work, but it's interesting, right?

• 31:45Speaker 01

  Yeah. And this is obviously GPT image generating the avatars. Very good for game design, like a lot of game designers were like really into GPT image.

• 31:54Speaker 02

  I I will say like the the broader takeaway probably is the reason that we do this is more so just to test the tools, right? Like um this was also a test for five point six came out. I had done the game on five point five, right? The ability for me to no longer need it to, I had defeated the rules. It's it's a pretty niche game. It couldn't find how to do this on its own.

• 32:15Speaker 01

  Oh, yeah.

• 32:16Speaker 02

  Five point six.

• 32:16Speaker 01

  It is auto distribution, that's why I also very keen on testing the five point six capability.

• 32:21Speaker 02

  But you know, this is just as as work comes out, as new things come out, these are just our sideways to test things, right?

• 32:27Speaker 01

  Yeah, it's some kind of private evil, I guess, that is not less private. But also valuable because now you can send this to your friends. And I mean, I learned about this game through seeing this.

• 32:36Speaker 02

  It's a hard game, he's very good.

• 32:39Speaker 01

  It's good to when no one is competing with you. But yes, it's a classic RL problem of like self play, bootstrapping your game AI. Yeah, you see how easily work becomes personal and personal becomes work because the thing I do for personal, it actually directly informed people I work with because I showed it to them. They were like, oh, you can do that with GPT, which like I imagine is the growth strategy. Yeah, the the show not tell is a big piece that, you know, I think we we're not still not fully cracked of like, you know, showing people all the the things that they can do with the product versus like trying to teach that to them like, you know, articles or onboarding or whatever.

• 33:18Speaker 02

  Yeah.

• 33:18Speaker 01

  So meeting them in the moment. It's a career risk for me because I used to be in developer relations, right? Where your job is to show and then you're like, what do you mean? You you don't you don't need, actually, your job is to tell. And then and then but the product people are like, well, we don't need you if our product is intuitive enough, so. Yeah, I mean, that's the magic of the the models. So you can tailor the telling or the showing to like specifically what the user needs, like what what they care about, what they've done in the past, exactly where they are on the adoption journey. So I think that's like going to be a super big opportunity.

• 33:50Speaker 02

  Seems easier and easier now to tailor custom showing, right? People have different use cases, as much as you said, you don't want to segment different people into different buckets, right? It's also not that hard to for people that are in different categories. But the question, I guess, is you said your team is more broadly on, what was the term you used, productivity?

• 34:11Speaker 01

  Yeah.

• 34:12Speaker 02

  Productivity.

• 34:12Speaker 01

  Which is now work, basically.

• 34:14Speaker 02

  Is it work? Is there another distribution that we're not hitting? Is there a group of people that will have something different than chat GPT Codex or work? Is there is there more that the mass isn't targeting?

• 34:27Speaker 01

  I see it as like a sequencing, like, you know, the the vision is like bring useful agents to everyone. We started with like developers, like developers historically are like early adopters that are willing to put up with more friction, set things up, et cetera. Like that's where, you know, Codex started. I think the next opportunity is like sort of what we call general knowledge work, you know, all the other functions around developers. I think when when you go from developers to this segment, like there's inherent challenges, obviously, with like, you know, this the show not tell thing that we're talking about, making the product more understandable, bringing in new capabilities that matter more for for this cohort, matter for developers, things like artifacts, things like computer use, et cetera.

• 35:11Speaker 01

  And then I think like the the same learnings, like similarly, how we took the learnings from developers and brought it to, you know, general knowledge work. The next stage will be like taking the new learnings from general knowledge work and bringing it to everyone, no matter what they're doing in their lives. And we're already seeing that a little bit, like this this game example that you have is, you know, something that's like on the border of like fun and personal life to to, you know, your your professional life. I use chat GPT work full time at home for everything, like for for whatever I'm doing. I used it the other day to come up with a meal plan and like, you know, save that on um on the like computer environment that it has and something that I can continue going back to. Like, is everyone doing that yet? Probably not because the thing says work on it, but eventually, you know, we want to get people there.

• 35:51Speaker 02

  Chat GPT life.

• 35:52Speaker 01

  Yeah, exactly, chat GPT cooking.

• 35:55Speaker 01

  But I think there's a lot of uh there's a lot of opportunity there, but I see it as like, you know, we're built, we built the foundation in in in in software engineering and we're going to take this same learning, so we take some software engineering to knowledge work, knowledge work to to everyone.

• 36:07Speaker 02

  Do you have any power user advice? I feel like um there's a group of people that will live it, use it for everything, stay on it twenty four seven.

• 36:14Speaker 01

  Yeah.

• 36:14Speaker 02

  Um, and then there's a bit of a gap between that crew and people that, you know, okay, I use it for work, I use it occasionally, sometimes I pipe questions. Any advice, any learnings, anything you recommend? Or just, you know, takeaways that you found that help bridge that gap?

• 36:30Speaker 01

  I think a couple of things that I've seen is like, one, that it really helps to broaden your imagination of what's possible. And this has been a learning even for me, like, you know, the technology has progressed so fast that, you know, it's something that like even three months ago, like no way, no way that the models can do this, like now it's like, wow, it's like you actually can like.

• 36:52Speaker 02

  You have an example?

• 36:52Speaker 01

  We're going through right now, there are like review cycle internally. And people always talked about this as like kind of a a thing that the the models are good at and like, you know, there's a cliche of like, okay, like I had no one who wants to be writing reviews and like, we just use AI to do it, but I mean.

• 37:08Speaker 02

  And it's evaluated as well.

• 37:10Speaker 01

  Yeah, exactly. In all seriousness, before it was like just like slot, basically, and like I I think it was helpful, but you know, not super productive. Now I've found that like the model can do a much, much better job than me, especially in this environment of like pulling context on like what people are up to, how they've like the things that they've done to make a difference, highlighting like, you know, wins that they've had that like I might may not even have seen, you know, has access to like everything, right? Like the code, like, you know, things that they've caught, reviews, Slack, everything. And so it's like incredibly powerful in that domain. And like, just like six months ago, the last time we did this cycle, like I didn't even, I tried using it, but it was not at all helpful. And this time it's been like incredibly helpful.

• 37:49Speaker 01

  And like, so I think continuing to push the the frontier of imagination, what's possible, even if you tried something before, I think is maybe that my biggest piece of advice. The other, I guess, thing is like the more the more you put in, especially in this environment where like, you know, the model has access to to everything on your on your computer or in chat GPT work, like you can create, you know, artifacts over time and save them in your library and like the model will continue having access to those. Like the more information you give it about whatever domain you're in, whether it's your life or your work, the more valuable it becomes. And it'll become valuable in like ways that might surprise you, like it might pull from context in a way that, you know, may be proactive and that you might not even have thought about. But it needs to have access to those um to that those tools or that context first.

• 38:27Speaker 02

  One thing I just want to talk about the review stuff because I'm still, that's a very sensitive thing. And you're you're a founder, you've managed people, you've hired people. As manager myself, I'm very reticent to put out any LLM generated things, especially when it comes to people because it feels like you don't care. Presumably at OpenAI, people are obviously more open to being evaluated basically rated by GPT. But are there any unofficial rules around this? Like, what's the etiquette?

• 38:57Speaker 01

  Oh, I mean, I think the etiquette is like, I would never write something via like solely via AI and like present it as like a review for someone. What I was talking about is more like gathering context. That's the place where it's.

• 39:08Speaker 02

  So it's just search, it's agentic search.

• 39:10Speaker 01

  It's like agentic search, but you know, that you can tailor and steer much more capably than you could before. And because like the the thing is, it's all, there's sort of a flywheel happening, right? Because of Codex, people are able to do and because of chat GPT, people are able to do so much more now than ever before. And if you're able to do so much more, it's easy to miss things as well. And so like, I think we need to use these same tools to keep up with all the impact that people are having and and and understand, you know, where where it can be helpful.

• 39:39Speaker 02

  I think the the the thing like, obviously, I I run a small company, so easy to search. But at the scale of OpenAI, with the amount of messages that you guys put in Slack, do you think that it misses things?

• 39:50Speaker 01

  Probably, but I think that I also miss things.

• 39:52Speaker 02

  Like it doesn't matter, right? Like it's it's as it needs to be human.

• 39:54Speaker 01

  It's all relative, right?

• 39:56Speaker 02

  Yeah. Sometimes it's nice when it finds things you wouldn't, right? Like right now, my Codex system prompts, they're set up in such a way that every project I have has a secret separate uh notes MD and it just writes learnings to there. And then the the global one can pull from all these. So sometimes it'll be like, oh, there's this project you did like four months ago. Here's a note that we had and it randomly pulls it back in the context that I would never do, I haven't thought about. And I'm like, okay, this is quite superhuman, right? Like stuff that would, and you know, it'll save like hours on chunking of stuff or find something that's already been done. I'm like, as much as it might miss stuff, I would too, but it's very useful when it finds stuff. And I have like a very, you know, non super engineered solution to this. It's just markdown files that get pulled whenever they want.

• 40:41Speaker 01

  Yeah, I actually have a funny anecdote about this, like recently. Gearing up to this launch, you know, the team has been, you know, really cooking on it for, for, for a couple of months. And over that time, like, there's so much conversation and chatter going on in Slack and Docs and elsewhere. And one of the members of the team set up this, um, scheduled tasks, like automation, to like, look at everything that's going on and like, come up with the best memes and then post it in one of our shared channels. And like, there are two cool things about this. Like, the first is like, I think the models are, you know, over time, like, actually starting to become like funny. Whereas like, you know, a year ago, like, that was not at all the case. The second is that it was what you were saying, like, they find things that, in surprising ways, that you may not, not have thought of, and like, create connections that you may not have thought of.

• 41:23Speaker 01

  And that really helps with like, the meme generation, because then you can see something that, you know, genuinely surprises you and, and, um, is funny in that way. So, yeah, I mean, obviously, that's like, not like, the most productive, uh, use of this, of this technology, but it does, it doesn't cover this, like, this capability that's emerging, which is just like, to find information that you otherwise would not know of.

• 41:42Speaker 00

  Talking about the, the launch, I, I think, uh, I have pretty much said this is the most successful launch in a long time. I think even more successful personally than 5.0, and they're announcing 10 million users. Does it feel different? You've been through a lot of launches.

• 41:58Speaker 01

  I think it feels like a culmination, well, I, I think two things. One, it feels like a culmination, like I was mentioning earlier, like, this, like, vision, mission that we've been on for a long time. Like I said, we saw the magic of Codex internally, and then we're like, extremely excited to bring this to many more people and to see it working, to like, see us reach, you know, the distribution goal, I mean, um, numbers that you mentioned, like, I think that's like huge and, and super exciting. The flip side of that is like, there's so much more to do too, like, that's also really exciting, like, you know, ChatGPT as a whole, like, the, the, this product that, you know, everyone almost equates to AI and like loves, you know, has hundreds of millions of users. And so like, 10 million is really cool, but like, we, we need to get this to everyone, like, we need everyone to feel this magic.

• 42:40Speaker 01

  Um, and so that's the next step from here, but yeah, I think extremely pumped about how, how it's going so far and then the opportunities.

• 42:46Speaker 00

  Uh, awesome. Uh, I did want to also, because I've, I've, I've been tracking that, that number closely. It transitioned at some point from just Codex users to Codex plus ChatGPT work, obviously because the same harness, the whole point is that you don't, you can't, uh, content separately. You have roughly a billion, uh, ChatGPT users. Why is it just jumped to 1 billion right away? Like, isn't that the default on ChatGPT or no?

• 43:11Speaker 01

  We don't default you into ChatGPT work if you're on ChatGPT.

• 43:14Speaker 00

  If you're free, yeah.

• 43:15Speaker 01

  It's also only available to, to paid users right now, and, and I think there's like a process of, you know, educating users of what is the value of this product, having them try, learning from their feedback and making it better over time. But, I mean, the goal is to, you know, get as many of, of people who, who love ChatGPT today to like, feel the, feel the power of ChatGPT work, but I think it'll be a, a journey.

• 43:36Speaker 00

  Yeah. Codex will still be alive as a brand for the foreseeable future.

• 43:39Speaker 01

  Yeah.

• 43:40Speaker 00

  Um, and we'll just toggle between them as, as, as needed for UI stuff.

• 43:43Speaker 01

  Yeah, I think it's even, even stronger point than that, like, I think we fully intend to like, you know, treat developer, like, developers have been, you know, a core market for us for so long. And like, there's, there's so much more that we can do to make Codex great specifically for, um, software development, and we'll continue to do that. This doesn't take away from that at all. If anything, it should increase the utility of something like Codex, because now you can move seamlessly between writing a diff to creating an artifact or, you know, doing a search over your character.

• 44:11Speaker 00

  I, I do wonder how much this terminology leaks to the non-technical user? Like, do they have to learn to say artifact if I want artifact or, you know?

• 44:20Speaker 01

  It's funny, like, we call artifacts internally because that's what the teams call it, but like, externally, like, no one says that, no one calls it an artifact. But I think that people like, often, like, describe things whatever they're used to, right? So if, you know, ChatGPT work is good at creating slides, they'll say ChatGPT work is good at creating slides, and that's actually what we want.

• 44:38Speaker 00

  One big, another, I mean, it's July of 2026, one big thing that also happens in, uh, for OpenAI was OpenClaw. And that's, I think, a lot of people's first time really maxing an agent for personal stuff, but also crossing over to work in, in a same, same way. As far as, as far as I understand, OpenClaw is still independent, but did you go through your own OpenClaw moments? Were there any lessons you took from OpenClaw to Codex or back, whatever?

• 45:06Speaker 01

  I think there's a lot of inspiration. I, I did go through my own OpenClaw moment. I, um,

• 45:10Speaker 00

  Yeah, tell the story.

• 45:10Speaker 01

  Me and my, um, my, my wife, like, set up an OpenClaw to, like, try to manage everything in our house. Not that there's like a ton, but it was like, actually quite useful. We gave it a calendar and started, you know, creating events for us and stuff. At some point, the, the laptop that we're running on had died and then we got a chance to, to pick it back up. But there's a lot of inspiration there, like, um, you know, in ChatGPT work, in web and mobile, like, you, you get access to like persistent computer environment, where, you know, you can store files and those files stay around between sessions. And the idea is to be able to enable use cases like this. Um, one of the members of our team actually uses ChatGPT work for what they used OpenClaw from before.

• 45:50Speaker 01

  And I feel like it has like completely transitioned, which is like, uh, workout planning and like meal tracking, um, which again, it's like a worky thing, right? It's like not work necessarily, but it's like in personal productivity space, but it has all the same primitives, like it has scheduled tasks. It has the ability to store files on a file system, has the ability to like reference those things over time. Um, and so you start to see the same types of use cases emerge, which has been really cool.

• 46:14Speaker 00

  Is there a point that ChatGPT work completely replaces OpenClaw? Obviously, they're independent, so.

• 46:20Speaker 01

  Yeah, I mean, I'm, I'm not close to it, so I can't speak to the, the OpenClaw roadmap, but I, I don't think so. I think that there's gonna be, you know, there's always a need for like this, like incredible, like open source technology that, that, that team has built. And I think that we can draw inspiration, um, in the product and, you know, ChatGPT. I think many more people have like heard about and use ChatGPT than have you, have used OpenClaw. And if we can take the magic from OpenClaw and bring it to them, I think that'll be a success. I think that, like, one thing on the ChatGPT work side that we feel strongly about is that, like, the core experience is that you come to this product and you have a conversation, start a session, whatever you want to call it, with this agent. And the magic of the product is that you can do anything in that moment.

• 47:01Speaker 01

  And we would like to create a product where you don't have to click a button or to go to a different place, whatever. And you can get whatever functionality exists in, you know, your, your finances app or, or any other product, like, in this, in this one place. And so that's the goal. It's like, we, we want an extensible system with plugins where you can connect to the tools that you need in order to be able to accomplish like a financial task, where you can, you know, if, if you're doing like science work, like, we have an ability to like, extend the system and such that you can like write the tech and, and perform as well. There will always be like products that we support that are best in class at those things. But we want as much of the magic as possible in that core experience.

• 47:45Speaker 00

  Yeah. Do you think that you can do everything you used to do with Wellfront in ChatGPT Finance?

• 47:50Speaker 01

  I actually tried it. I mean, like, ChatGPT doesn't yet custody, uh, cash and, and assets for me, so, so that part, no, not yet. But I, I mean, there was like a whole component of like retirement planning and, um, sort of like financial planning and budgeting and stuff that, that, um, we were looking into when I was there and like, with the finances plugin, like, that's all possible in ChatGPT today. So, um, I feel like at least that component is replaced for me.

• 48:17Speaker 00

  I haven't really plugged it in yet. Um, I somewhat scared to look at the answer. Like, that's honestly, like, the same reason for health and finances, like, oh my god, I don't know.

• 48:27Speaker 01

  It's really good. I mean, it's, it's really cool how, I mean, we were talking about like the agentic search aspect a little bit earlier, but like, it's really cool how like, you know, in, in conventional UX, like, if the more power you want to give to a user, the more like knobs and bells and whistles you need to add, like, you know, for like these finance and budgeting apps, like, there's always like, a bunch of the different filters and like search bars and stuff like that. But like now, like, with the right connectivity to the right data, you can have whatever you want. You can ask any question you want, um, and into that box and, and get the answer, and I think that's super powerful.

• 48:57Speaker 00

  I think it's also nice to just have it centralized in one space, right? You have different health apps, I have one for a smart scale, a watch, all these different things. It's just nice to centrally co-locate it. Which is, uh, you know, part of the whole thing of OpenClaw, right? Like, that, that you would have, um, personal OS, uh, which presumably ChatGPT wants to become. I do think that just relying on, like, just-in-time pulling of data for, let's say, via MCP, CLI, API, whatever, whatever you do, still not enough. Like, I, like, I come from a bit of a data engineering background, like, you still want like a data warehouse or some kind of caching or semantic layer. Um, do you feel that or do you already have that?

• 49:40Speaker 01

  I can't speak to like all the details on how everything works, but I think it depends on the access pattern, right? Like, if you want an answer immediately, then yes, it's very difficult to do that if you need to pull from all of these sources. But a lot of the, like, use cases that we want to enable in ChatGPT work aren't necessarily something that you need immediately. It's more like a task that you want the agent to go and do, and that's gonna take a certain amount of time. And, you know, with things like programmatic tool calling and stuff now, like, some of that time and subagents and stuff, like, some of that is also parallelizable. And so it's possible, I, I think it's very possible that there's a, the ceiling on what can be done, you know, with MCPs and like, calling out to these certified services has been raised substantially, so we're really excited about that.

• 50:23Speaker 00

  You mentioned some agents, I, I got a double click on that. Ultra is a new mode. You have special affordances in ChatGPT itself to show off the, the agents. Can't really do much with them, to be honest, like, just, just watch. What have been your experiences, any design issues that you would call out to other builders building with subagents?

• 50:45Speaker 01

  I think it's sort of goes back to the balance that I was raising earlier about like, you know, showing builders the power of the tool, but also creating enough of an abstraction to not overwhelm them. I think with subagents, the thing that we wanted to show is that you can take a task that, you know, has many parallel tracks or is complicated in a way that, you know, some agents can handle. And this product is for you, like the model can accomplish those goals or try to accomplish those goals. And so like, that's the point of like showing them the product and that's where we've gone with the design. There's another, you know, iteration of this where like you can see exactly what they're doing and things like that, which I think is like,

• 51:27Speaker 01

  you know, could converge on like overwhelming with information. And so this is like the deliberate trade off that we've made for now.

• 51:33Speaker 00

  I mean, you do display quite a lot of transcripts.

• 51:35Speaker 01

  Right, right.

• 51:36Speaker 00

  I think it's hidden by default.

• 51:37Speaker 01

  You wanted to say more than that? No, no.

• 51:38Speaker 02

  Some people could want more. So I'm one of those people that will basically throw a lot of stuff at goal. And pretty much every goal, I'll tell it to use subagents. Seems redundant, right? But every time I'm like, okay, use subagents where possible. And I have a lot of people, a lot of friends that recommend and do the same. Whereas I'll sometimes talk to people that are like, okay, this is where I want you to use subagents for this subtask. And I'm sure they would appreciate seeing into how they're being used. For me, it's primarily like two things, right? One is net time efficiency. So span out across subagents. Two is probably cost, right? Don't use big expensive model, offload to a lot of smaller, cheaper models.

• 52:20Speaker 02

  And some people want that level of control. So if you have repetition in what you're doing, right? Say I want something built where I want it to consistently do this every day. I might want to go in and fine tune subagents here, subagents there. So you can see both. But I think if I'm not mistaken, it's hidden by default. There's a dropdown that goes a lot where I'm like, okay, I'm just gonna keep using.

• 52:40Speaker 00

  You can change the model that they use?

• 52:42Speaker 02

  I know I tell them to be steered. I'll say my, I know Anthropic offers this in cloud code. You can tell Fable to use Sonnet or Opus to use Sonnet as subagent. So pretty trivial thing, you know? You tell it to span out subagents with Sonnet, you know, it's cheaper, faster. I would assume if it's not there, it could be built there. But I think there's a side of.

• 53:02Speaker 00

  Too many toggles.

• 53:03Speaker 02

  It's not a toggle, actually. It's just a, you tell it. The way I do it is prompt it, right? And I think this is something that gets abstracted unless it's something you built for repetition, right? So if I'm building something, say that's a podcast prep, right? Research into people, do a very, very deep, extensive research. That I might want to configure to cheaper, faster model just for web search, right? I can see a world in which you want both. I think the default is actually pretty good right now where it's hidden, but you can drop down and get some more info into what's done. I know people talked a lot about it on 5.6's launch. This thing loves to use a lot of subagents and causes the ChatGPT app to just crash because it's so processor heavy. But.

• 53:47Speaker 00

  So what is, so that's not my experience. Yeah. I mean, you know, I haven't had a crash from sub.

• 53:52Speaker 02

  I haven't either. We both have big laptops. I know, I know people brought it up. There was a topic of discussion that we didn't see the same, but it is another vibe eval, right? People are like, okay, the amount of subagents solo spawning is crazy. And I'm like, I think this is okay. I think it's good. But just stuff people bring up.

• 54:12Speaker 01

  I think when we launched the product too, we weren't as opinionated about like, who is Ultra for? And like, when, when should they be using it? And since then, we've made some changes to like, you know, require you to turn it on and find it in the advanced setting because that's who it is for. It's for like power users who understand what's going to happen. Because it also, you know, depending on your use case, can, can use more of your, your limits as well.

• 54:33Speaker 00

  Yes.

• 54:33Speaker 01

  So that's where I think a lot of the feedback was coming from.

• 54:36Speaker 02

  It's okay, reset the limits. Always reset the limits.

• 54:39Speaker 00

  You know, today we're resetting because of this. I want to change topics to one last piece of the harness, memory. A lot of people are commenting on memory recently. ChatGPT's new memory system uses it. It's not very good. And then this guy also basically the same thing. And Samir, who you presumably work with, talking about memory. What can you say there?

• 54:57Speaker 01

  I think that, you know, Samir and the team have made a ton of, and then the research teams have made a ton of updates and improvements over time. I think when I talk to friends, family members about what they love about ChatGPT, like the fact that it knows them, they feel like their ChatGPT is their ChatGPT, I think comes up probably number one. And ChatGPT work in the, in the cloud, like by default, all conversations like inherit from your ChatGPT memory. So you'll know, they'll know context about you and they'll also be able to write back to this memory.

• 55:26Speaker 00

  With like, like a, like a small text write, like you tell me when you're writing, right?

• 55:31Speaker 01

  No, it's part of the same like memory V3 system that we launched.

• 55:36Speaker 00

  Yeah, dreaming V3, yeah.

• 55:37Speaker 01

  So I think that's been really powerful because, you know, going from ChatGPT to ChatGPT work feels like an extension of what I've already been doing with the product for sometimes many years. So that's been awesome. And it's awesome to see that like people are recognizing the improvements here.

• 55:51Speaker 00

  Is there, so it's, it's basically a retrieval problem, right? Like, are you retrieving the right things? Are you over focusing on the wrong things? Is there like more false positive or false negative, you know, if that makes sense? Like, what's the bigger problem?

• 56:05Speaker 01

  So I don't work on memory directly. So it's hard to say what the bigger problem is with like certainty. But I, I think you're right. I think that like, you know, the, there's two sides of it. It's like, you know, making sure it knows things about you, but then also having the EQ to like bring those things up at the right moments, proactively or surprising you in ways that are positive, not negative. So I think it's a very challenging problem, but something that I think we feel very, is a huge opportunity to get right, which is like, why we made like big investments in it.

• 56:29Speaker 02

  How do you see the side of, okay, when you're building ChatGPT for work, different than the regular chat app, different than Codex, managing memory across different projects, collaboration and whatnot? How do you see the, the side of what's separate from the harness, right? So if I have four threads on one project, any learnings on how to build memory systems there, you know? For background as well, I guess to steer it a bit is when you do chat style applications, I'd say you have a lot of one offs, right? When you switch to work, it might be something you're doing for a month, something you do a lot, right? Now, as I add more sessions, there's a lot more than just single threaded, right? And there, there might be memory there.

• 57:10Speaker 01

  I mean, I think first I challenge that like the depth of the memory or the like value of it is like fundamentally different across chat and work. Like it is true that like, you know, there are a lot of like shorter sessions on chat. But I think, you know, the ChatGPT, the product has had like a ton of longevity. You know, as long as this, this technology has been around and, and people use it for worky, like productivity related things already today. And so I think we found that there's a lot of value. I mean, I found this with my personal usage, like all these one offs add up over time into something like quite durable and like, like quite a good representation of who I am. I know like from time to time, something will go viral on, on X about like, you know, ChatGPT telling you everything it knows about you. And people are always surprised like how, how deep that is.

• 57:55Speaker 02

  The fun roast me, you know?

• 57:56Speaker 01

  Exactly. So like, I think like the, that's all to say that like, I think there's a lot of depth there in the existing, you know, ChatGPT product. And so that's why I think we think it's valuable to bring into the work product. But the other reason I brought that up is because I think like, hopefully we can use some of the same fundamental primitives and systems to extend memory here as well. And I know this is something that the, the team that focuses on this is like working, working through right now.

• 58:19Speaker 00

  I wanted to bring up one element of memory, which I honestly don't really use much. And I'm curious if you do Chronicle, which was, is up on screen right now. It's kind of a super memory or like, what, what is it?

• 58:33Speaker 01

  I think the idea is that like, it can learn from, you know, how you're using your computer. And like, it's another input source into memory. And I think it's, you know, experimental right now and something that like isn't default off. But I'd recommend that you try it. I think that it's like quite interesting how, it goes back to a conversation we were having earlier on like, you know, you were asking like, does it, can ChatGPT miss things? Like, does it, you know, on Slack, when it's searching, does it miss things? Cause there's such a volume of stuff, right? And like, it's, you can ask the same question about like everything that you're doing on your computer. Like, is it gonna know everything that you're doing? Is it gonna capture the intent and stuff like that? Probably not, but like, it probably will find things that you might not know about.

• 59:15Speaker 01

  And then if it can surface those to you in relevant times in proactive ways, like when you're doing tasks. And I, I've found at least that it can be quite helpful. So it's worth trying.

• 59:24Speaker 00

  So mostly for insights and longer term?

• 59:27Speaker 01

  Yeah, exactly. Like insights and it builds context that makes, that can make you more productive on certain tasks. But it's, it's hard to describe without feeling it.

• 59:38Speaker 02

  I will say you can feel it pretty well. Like the idea of what they're saying here, right? Just check through my memories or check through my logs and add skills.

• 59:46Speaker 01

  Yeah.

• 59:47Speaker 02

  Pretty underrated, right?

• 59:48Speaker 00

  That's automations. You can repeat that using a cron job.

• 59:52Speaker 01

  Checking through your memories and creating skills? But I think the creation of the memories from Chronicle itself is like what's different. It's like you have much deeper memories because you have Chronicle on.

• 1:00:01Speaker 00

  It's there. I don't use it much, but maybe I just, I need more examples. I imagine you guys use a lot of it internally. So I'm always fishing for use cases.

• 1:00:10Speaker 01

  Yeah, I would just try turning it on and then like.

• 1:00:13Speaker 00

  It just auto works, like.

• 1:00:14Speaker 01

  Yeah. And seeing like where, where it might start helping you. I think you'd be surprised.

• 1:00:18Speaker 00

  Yeah, amazing. I think that was the, about it in terms of like the, the overall coverage of ChatGPT work. I think there's been a lot of like good progress and discussion on building and all these things. There's a lot of like ex-founders in the, in the community and in OpenAI as well. Do you think that things have changed a lot? I guess like your overall reflection of building pre-AI and post-AI?

• 1:00:44Speaker 01

  I mean, I think things have changed a ton. I think it's, it's like super exciting to see how quickly you can go to from idea to something real today.

• 1:00:56Speaker 01

  Whereas like even before, like I think, you know, five, 10 years ago, like it was fast if you were scrappy and, you know, like rolling to build the minimal viable thing. But like now the extent of what you can build is like much, much broader. And I think that also, like what we've seen internally building is like, that gives you an opportunity to validate much more quickly, to talk to users, to talk to, to internal doctors, et cetera, and like make sure you're on the right track. And like that loop, I think has been, has become more closed than ever before. And that's like a win for product development. I think it's a win for, for consumers and users too. Cause ideally that means they're getting much more better, much better products out the gate.

• 1:01:32Speaker 00

  Does it mean your teams are smaller?

• 1:01:33Speaker 01

  I think there's much more to do now. So I think. People can accomplish more individually or in a small team than they were that would require more people than before, but there's at the same time, there's also more to do. So I think the teams are much more ambitious.

• 1:01:50Speaker 02

  You seen any changes in scopes of roles and building teams and how we used to have teams, say, a few years ago versus what ideal teams look like now?

• 1:01:58Speaker 01

  I think we've seen a blurring in the lines between like the typical product development functions, like between like, you know, EM, PM, engineer, designer, et cetera, like.

• 1:02:08Speaker 00

  Yeah, I want to bring up this quote, there will be only four jobs left in tech. There's AI slop cannon, but people who just like throw, burn a bunch of tokens. And then there's, there is SRE, who, people who are more responsible. There's grownups who sell things and then there's hot people.

• 1:02:27Speaker 01

  This is an interesting take. I think my, my suspicion is that there's everything, everyone will be like T-shaped in a way and that like AI will enable everyone to become a generalist. Like, you know, things that like I, I never would be able to like come up with a design before and like even now, I don't have maybe it's like the visual taste required, but I can iterate on something with the help of AI. But then people will have a specialty and that's like the, I guess the straight line in the T or the upward line in the T. And so like you can have a specialty that you're interested in with the help of AI, you can go deeper and become better out over time, but then you'll also be a generalist. And so with that foundation, the what you can accomplish is like almost limitless.

• 1:03:07Speaker 00

  What are you bottlenecked by in terms of specialties? Like, do you need more designers? Do you need more slop cannons? Do you need more hot people?

• 1:03:15Speaker 01

  I think the bottleneck some becomes like sort of like ideas and taste, I guess. Um, I think because anyone can, can build now, I think it really is the era of like bottoms up ambition. And because there's so much to be built, like you're always going to be bottlenecked by, you know, the amount of ideas and amount of things that you're doing at a given time.

• 1:03:37Speaker 02

  Do you think models help solve that?

• 1:03:39Speaker 01

  Models?

• 1:03:39Speaker 02

  Yeah. I mean, I have the example of like, I have a front end design skill that's like, they give me four drastically different examples of what this looks like. Sure, it burns a lot of tokens, but you know, and then I'll mostly just condense down, okay, I like this part, I like this part, let's draw these together. And it's like, yeah, I had a vision, but like, I don't know.

• 1:04:01Speaker 00

  I would say that the one automation that I would love to work and it doesn't work is bring me new ideas, right? Uh, somehow L M's are just not it.

• 1:04:11Speaker 01

  One interesting part of that idea is like, they're not like in a vacuum. It's like not, they usually come from somewhere and like, you know, in, in product development, like they're coming from talking to users or reacting to, you know, friction that you're seeing or feedback, building on some foundation that you already had planned out before, whatever. Um, and so I think that's where like, I think there, there will always be value in, in these like generalists that we talked about, like, you know, closing that loop and, and having, coming up with those ideas that are grounded in, in that feedback or talking to users, whatever it is.

• 1:04:41Speaker 00

  Cool. Uh, you were gonna, you lead the productivity team. How do you define productivity?

• 1:04:46Speaker 01

  I think our mission is to make it possible for people to do things that they weren't able to do before. And right now we're thinking about it from the, the perspective of knowledge work. And so when I look at knowledge work, I think about people are no longer sidelined by their roles, they're no longer sidelined by maybe the, the background or training that they have, like no matter what function you're in, you can suddenly build things and suddenly get access to data that you otherwise might not be able to interpret, et cetera. Um, and then I think that extends to your personal life where we want to give you leverage at the end of the day. Like we want the models and the product to be able to give you leverage so that you can, you know, create time for yourself to do the things that you love.

• 1:05:25Speaker 00

  Does that also translate to a way to measure productivity? Like what is, have you measured leverage?

• 1:05:31Speaker 01

  I think we haven't figured this out yet. Part of the reason is it's so diverse. Everyone has different goals and really the true measurement is like their ability to achieve that goal. Did we help you or did we not?

• 1:05:43Speaker 00

  Yeah.

• 1:05:44Speaker 01

  And it's very difficult without knowing what that goal is upfront and also tailoring your current individual.

• 1:05:48Speaker 00

  And the thumbs up and thumbs down from chat G P T doesn't give you anything, right?

• 1:05:51Speaker 01

  Right. I mean, you don't know if they're thumbs down in the, the content of the answer, the vibe of it, whether it'll help them with their goal. I think that's difficult. Um, but it's something that I think we will need to figure out and the industry at large will need to figure out because, you know, that's how we, we measure success, that this is what we're for.

• 1:06:06Speaker 02

  Do you think it's changed productivity and how you measure it? Basically, you said there's a lot more work that can be done, a lot more scope. Um, has it changed?

• 1:06:15Speaker 01

  I think it was always true that what you really wanted to measure is like, you know, was your team, was the individual, were you personally able to hit the goal? Or are you closer to hitting that, whatever your goal is, right? But I think previously we use proxies for this, so like, you know, code commits or lines of code or whatever.

• 1:06:32Speaker 00

  Story points.

• 1:06:34Speaker 01

  Yeah, exactly. Story points and like.

• 1:06:36Speaker 00

  They're coming back, by the way.

• 1:06:37Speaker 01

  Maybe, I mean, but, but that is for a part of the change. And like, I think with, with AI now, those proxies starting to fall apart, like, you know, and the number of tokens you use or the number of pull requests you make or like no longer like maybe as hyper correlated with that is your team able to hit the goal or are they on, on track to their goals? So I think we'll need to come up with, with new, um, measurements.

• 1:07:02Speaker 00

  For the managers listening, uh, give them one thing to try.

• 1:07:06Speaker 01

  I think for me, what's important is like at bats. Are we as a team building the muscle to have not just quantity of at bats, but quality? Like, are we able to go all the way from like generating an idea, building it out, getting the feedback, reacting to that feedback, actually validating or invaliding the hypothesis, going on to the next idea? Are we able to do that really efficiently? Like that goes to like, you know, the actual like code that's being written or the designs that are being made or specs that are being written, whatever, but also the culture of the team, like, do we have the humility and, and, and, um, are able to like go through that process many, many times and stay motivated and excited throughout that? Um, so that's the thing that like, I think is important now, especially when we're on the frontier of this technology and like, there's so much to build, there's so much to do.

• 1:07:51Speaker 01

  That's probably the most, most important thing that we look at.

• 1:07:54Speaker 02

  Any traps people fall into around measuring productivity, what your teamwork on? I feel like there's a lot of, okay, we added a lot of L M's, we have dashboards for this and that, but not much has changed, right?

• 1:08:07Speaker 00

  That is the trap, yes.

• 1:08:09Speaker 02

  And you know, the, the broader source of the question is for, for the managers and teams building, you know, how, how should they approach this?

• 1:08:18Speaker 01

  I think maybe the trap is like conflating motion and progress. I think motion is much easier now than ever before because of the tooling that we have. But progress requires you to be like very prescriptive and deliberate about like what you're actually trying to achieve. It goes back to our question of measurement, right? Like you were, we were talking about like, can we open AI, like figure out how to measure productivity for, for our users? That's, that's a very hard problem because of the diversity, but like as a team, like you should have a really prescriptive and deliberate view on like what progress looks like for you and for your team. And if you don't have that, then it's very easy to conflate these two things.

• 1:08:57Speaker 00

  I think at bats is a really great thing. I'm, I'm really glad, I like the discussion between motion and progress. I think that's a quote that we're going to feature on the write up. You've been very generous with your time. Thank you so much and congrats on 10 million.

• 1:09:08Speaker 01

  Yeah, thank you for having me.

• 1:09:09Speaker 00

  The next, uh, uh, next one I had a hundred in two months.

• 1:09:12Speaker 02

  Two weeks.

• 1:09:14Speaker 00

  Thank you.