ChatGPT – The Super Assistant Era | BG2 Guest Interview

In this BG2 guest interview, Altimeter Partner Apoorv Agrawal sits down with Nick Turley of OpenAI for a deep dive into how ChatGPT became one of the fastest-growing products in history—and what comes next. They discuss how OpenAI thinks about retention and product metrics, why long-term engagement matters more than raw growth, and how ChatGPT gets the next billion users. The conversation explores the future of AI assistants: moving beyond chat into proactive agents that can take actions, complete long-horizon tasks, and integrate deeply into users’ daily lives. Nick also shares how OpenAI balances product improvements with breakthrough research, how GPU constraints shape product decisions, and why building for both power users and everyday consumers is essential to discovering new use cases. The episode covers the evolution of ChatGPT pricing, the role of partnerships and distribution, and how OpenAI is thinking about scaling access to AI globally. A must-watch discussion for builders, operators, and investors trying to understand the next phase of AI—from chatbots to true “super assistants.” Timestamps: (00:00) Intro (01:00) Nick Turley’s Journey to OpenAI (02:15) ChatGPT’s North Star: Long-Term Retention (04:15) Why ChatGPT’s Retention Curve “Smiles” (06:45) What Drove ChatGPT’s Consumer Breakout (10:15) How OpenAI Gets the Next Billion Users (14:15) When ChatGPT Starts Taking Actions (18:15) Why Coding Agents Came First (21:00) Beyond Chatbots: The Super Assistant Vision (24:00) Power Users vs. Casual Users (28:00) Why ChatGPT Pricing Has to Change (33:45) Partnerships, Distribution, and Product Tradeoffs (37:15) GPUs, Scarcity, and the Cost of Scaling AI (41:30) Shopping, ChatGPT as a Thought Partner, and Code Red (51:45) OpenAI’s Future Interface, Rapid Fire, AI Jobs, and Nick’s AGI Moments Produced by Dan Shevchuk Music by Yung Spielberg Available on Apple, Spotify, ⁠www.bg2pod.com⁠ http://www.bg2pod.com Follow: Apoorv Agrawal @apoorv03 https://x.com/apoorv03 BG2 Pod @bg2pod ⁠https://x.com/BG2Pod https://x.com/BG2Pod

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  ChatGPT originally was entirely free. And the reason for that was that it was intended to be a demo. And we were going to wind it down after a month. We then realized that the demo went viral and people loved the demo. And it was actually a product. And but we realized that to be a product, you can't take the product down every time you're at capacity. So we shipped subscriptions simply because it could shape the demand. It was a way of gracefully turning users away. And we had to turn away someone.

• 0:26Apoorv Agrawal?

  You guys are at 900 million BT active users now. And that growth has been incredible. The next billion users, where are they going to come from?

• 0:35Speaker 00

  We've got about 10% of the world coming to us now. 90% left to go, right? There is so much more opportunity.

• 0:48Apoorv Agrawal?

  Well, Nick, so excited to have you here.

• 0:50Speaker 00

  Thank you for having me, Pooch.

• 0:51Apoorv Agrawal?

  You've had quite the journey from Germany to the US from Brown.

• 0:56Speaker 00

  That's true.

• 0:57Apoorv Agrawal?

  Most recently at Instacart, delivering groceries in 30 minutes. You're now delivering AGI to billions. I'm sure that was a plan all along.

• 1:04Speaker 00

  Yeah, clearly total master plan.

• 1:08Apoorv Agrawal?

  Well, tell us about your journey. What, how did you get to OpenAI? I know it's a fun, fun story. And your, your three and a half years or so at OpenAI, how have they gone?

• 1:16Speaker 00

  The only through line in how I've made any sort of employment decision has been entirely people based. So I don't claim any credit for, for joining OpenAI. We're predicting ChatGPT or anything like it. But I hit up someone who I admire a lot, who I got to know at Dropbox. Joanne, who worked, worked here at the time and she, I asked her to get off the Dolly 2 waitlist. And she told me I had an interview if I wanted to get off the waitlist. So I took the bait and got totally nerdsniped in the process. And here I am.

• 1:52Apoorv Agrawal?

  There you go. The Dolly 2 waitlist will get you.

• 1:55Speaker 00

  It's a great recruiting tool.

• 1:56Apoorv Agrawal?

  Nice, nice.

• 1:57Speaker 00

  We should do more waitlists, probably.

• 1:58Apoorv Agrawal?

  Yeah, yeah, yeah. Well, you know, the big, the big. Supercycle we're in is Chad GPT. Now, I assume over a billion users on the monthly side, nine hundred million weekly active users that recently reported up from zero three and a half years ago. You could have, if I imagine what the dashboard of Nick Turley looks like, it could have users, it could have paying subscribers, it could have daily active users, it could have retention, engagement. I mean, there's like fifteen things, maybe all of them. What is your north star? How do you, how do you, what are you optimizing for? What is Nick looking at on his daily dashboard?

• 2:37Speaker 00

  It's funny, right? Because it's such a young product. It's been, to your point, three and a half years. And this kind of question, it kind of changes as you evolve and you grow up and you ask yourself, you know, what are we really building here? And to this day, right, I want to build a super assistant that can actually help people achieve their goals. And ultimately, the thing we care about is like, is our product doing that? Is it actually helping you do the thing that you're, you know, coming to the product to do? And it's so different for different people, right? Some people are trying to get healthy, other people are trying to start a company, learn a new topic, do their taxes. There's all the different things that you might be doing.

• 3:20Speaker 00

  And the true measure of success is whether or not we're helping you do that. And obviously, we look at WAU in particular because, you know, we want to know if you're coming back to the product. We look at retention. But we, you know, we look at all kinds of stuff in aggregate because really there isn't like this one single thing that you can optimize for.

• 3:38Apoorv Agrawal?

  If you were to allocate a hundred units of points to these metrics, which metric, can you distribute the hundred units across these metrics in order of importance for you right this second?

• 3:49Speaker 00

  It's a good question. I care a lot about long-term retention and I would put all my points there because I'm really proud of the retention stats we have.

• 3:58Apoorv Agrawal?

  Huge.

• 3:58Speaker 00

  But ultimately, the sign of durable values, whether or not people are coming back in three months, because that means you're really solving their problems.

• 4:04Apoorv Agrawal?

  Yeah.

• 4:04Speaker 00

  I think things like revenue, they follow from that.

• 4:07Apoorv Agrawal?

  Yeah.

• 4:09Speaker 00

  Versus like, you know, trying to go on those things directly. And we've had a lot of success making very principled decisions on this stuff. Like, one good example is, you know, GPT-4 used to be behind the pay wall because we couldn't serve it to everyone. And then we had GPT-4, which was a total breakthrough in our ability to inference it. And So we just gave it away for free and that ended up being totally revenue positive and retention positive because it just provided access to the tech. And I think when you make your decisions that way and you focus on the customer, you end up with a great product and revenue obviously falls too.

• 4:43Apoorv Agrawal?

  Phenomenal. Well, it it shows up in the numbers. You know, I posted this chart yesterday on the data that we have, you know, from a third party. The retention curve for ChatGPT are smiling. Look at that, just like that. And that is a rare, that is a very rare occurrence, you know, as we know. And why, why do you think, like, if you were to give us a narrative on that smile curve? What is the, why do these smile curves exist? What are you seeing in ChatGPT that has people who have, who have maybe turned off for a couple of weeks or months coming back? And why are they coming back?

• 5:17Speaker 00

  Look, there isn't one single thing. The way that you build a retentive product is lots and lots of little things and really trying to make it better systematically. I will say that, you know, with AI and in particular ChatGPT, I've found that it takes people some time to really understand all the parts of their life they can delegate. Right? And I think many users for that, it's a multi-month process for them to understand how can this thing help me and what are all the different ways that I can plug ChatGPT into my life? And but, you know, when I think about some of the breakthroughs and levers we've had, things like search and personalization, they, they have helped solve those user problems because, you know, search provides way more daily value to you.

• 6:01Speaker 00

  It used to be that ChatGPT was a pretty worky product, you know, we'd see usage go down on the weekend, we'd use it, you know, go down during the summer months when a lot of people were off from work. And today, you know, we're, we're mobile first, the vast majority of users is mobile. And we see all these personal use cases and I think search was a big investment that got us there. And personalization makes ChatGPT so much more relevant for you, right? Because it gets to know you over time, you get to know it. And those are two things that have materially moved the way that, you know. People come back to the product, but there's lots more to do.

• 6:35Apoorv Agrawal?

  Yeah.

• 6:36Speaker 00

  And, you know, as mentioned, I'm not resting on our retention stats, even though we're obviously very proud.

• 6:42Apoorv Agrawal?

  Nice, nice, nice.

• 6:45Apoorv Agrawal?

  And you know, the other thing that I got wrong about ChatGPT was this is two and two and a half years ago. And I was like, well, you know, let's look at who's going to win this consumer AI race. Typically, these consumer markets are winner take most, winner take all. Look at search, Google has near 90% plus market share, three and a half, four trillion market cap. Mobile, same thing with Apple, social, same thing with Meta. I was like, well, AI, Meta has all the distribution, Google's got all the distribution, they've got three, four billion users. Well, it'd be a flick of a switch for them to roll out their AI, but I was wrong. That's not what happened. ChatGPT turns out, you know, you guys are at 900 million weekly active users now. And that growth has been incredible. Clearly, distribution was not enough, right?

• 7:30Apoorv Agrawal?

  So the same question for distribution. What are the levers for us that have gotten us to the scale? Is it model quality? Is it, you know, product quality? Is it, you know, features? Is it the experience or product improvements like memory and personalization or search? Like, same question, like, what would you say drove historical growth and success?

• 7:50Speaker 00

  We've got about 10% of the world coming to us now, 90% left to go, right? There's so much more opportunity to reach more people and introduce them to the way that AI can benefit them, right? But when I look backwards, and I only say that because like the next billion users might be very different in terms of like how you engage and reach and provide value. But when I look backward, it's been roughly a sort of one-third, one-third, one-third between sort of classic friction removal type of work, like one of the biggest.

• 8:27Speaker 00

  Moments when you look at pure impact was removing the authentication wall. And Sam will say, I told you so because I think that was his feedback from like day one. It was like, you can't, you shouldn't have to log into the chat GPT. But it's the stuff like that that you do for any product and it does matter.

• 8:43Apoorv Agrawal?

  Yeah.

• 8:43Speaker 00

  Some things never change, right? But then another third or so is, are what I would sort of core product investments. And they're really typically things that we've done together between research and product. So search and personalization are really good examples of that where we came together and we figured out not just UI UX evolution, but also how to post train these changes into the model. And it was really the moments when we came together. Another recent example is like we have these writing blocks that render when you ask about queries where you're trying to write with the model. And like putting really good craft into those experiences really matters and our users love it.

• 9:26Speaker 00

  And then another third of the growth has been just model improvements. Like both step changes, like going from GPT 3.5 back then to GPT 4, then going from GPT 4 behind a paywall to 40, suddenly available to everyone, right? But a lot of it is also the iteration that isn't splashy, that doesn't warrant like a named release. And we've, I'm really excited about the updates we just made with 5.3, 5.4, et cetera. Because that is when we like take a lot of user feedback and we methodically address it. And obviously that shows up in our retention as well. So sort of one third, one third, one third between classic friction removal and access, core product investments, and then pure model improvements.

• 10:11Apoorv Agrawal?

  And so the question that I've really been waiting to ask you is how do we get the next billion? And, and you know, talk about that a little bit. There's a lot of, it seems like at least on the outside fog of war, if I was a consumer today in the market to pick my super assistant, you would have a couple of great options, you know? Claude out there, they're having some great traction last couple of weeks. Gemini, mega distribution, Uber distribution, and us, by the, by the, by the leading product today, at least in user numbers, the next billion users, where are they going to come from?

• 10:50Speaker 00

  First of all, just to contextualize that goal, we care about Two things at the end of the day, obviously, reaching more people is really important. It's the direct manifestation of our mission to the world. The more people we can introduce to the benefits of AI, the better that is.

• 11:07Speaker 00

  But we're also really excited to go deeper. And that means taking the same billion years that find value in ChatGPT today. And actually providing more meaningful value in the world, like actually helping them achieve their goals, not just answering questions, right? So I'll talk about how we get to more scale, but I think it's important to remember that, you know, the way this technology is evolving is, you know, we're gonna go beyond pure chatbots pretty fast.

• 11:31Apoorv Agrawal?

  Exciting.

• 11:33Speaker 00

  I think on on scale, it's shocked me how many people have found value in ChatGPT as it works today because I don't think delegation is a natural skill for most. And ChatGPT is a pretty, you know, it's a it's a power tool, right? You come to it, it doesn't tell you what it's for. You kind of have to discover it on your own and you have to use it. And then you'll learn about this prompt that was really cool. And then maybe you you're on Twitter and you learn about another one, or you're on Instagram and you learn another one. But the product, it's like a raw appliance. And I think for two, you know, on one thing we really need to nail as we, you know, reach the next set of users is a product that has a bit more of an affordance. Because I think for most people, they're very, very busy.

• 12:15Speaker 00

  And everyone, I think, in the world has intelligence constrained problems, like problems that more intelligence could help with. But you need to frame that to people. And I still feel like we're a little bit too much like a computer terminal. And it needs to feel more like software, like, you know, or, you know, an operating system of software.

• 12:32Apoorv Agrawal?

  Right.

• 12:33Speaker 00

  So that's one thing.

• 12:35Speaker 00

  Another thing that gets at the same constraint is beginning to be proactive.

• 12:42Speaker 00

  In a world where a lot of folks are, you know, too busy to delegate their problems to AI or don't quite know where to start. I think being able to help you proactively is really, really important as well. But you know, I think all of these are product evolutions that we could make on top of the current tech. And the thing that gets me particularly excited is productizing our next generation tech or reasoning models. Because the truth is, when you look at reasoning in chat GPT today, it's relevant to a very small group of people. It's relevant for the people who are trying to get the most out of chat GPT. But I fundamentally believe that reasoning, it's transformative. And if you can figure out how to productize reasoning in a way that works on people's behalf without them even knowing.

• 13:26Speaker 00

  And that looks very much like, you know, the model doing long horizon tasks on your behalf. It doesn't mean you encounter the concept, it just means it's benefiting you, right? So there's so much work to do and, you know, the product certainly has to evolve to be relevant for this kind of skill.

• 13:42Apoorv Agrawal?

  Yeah, one of the, one of the things that I've been hoping for a while, and you know, Brad made a bet two years ago.

• 13:51Apoorv Agrawal?

  When can chat GPT help me take actions? Can chat GPT help me be more proactive? And I think his bet expired end of last year, so he's, we're very curious. When is that coming? And I'll frame that for you because, you know, with, with, with search engines and Google, you know, two decades ago, you could have gotten the 10 blue links, you could have spent an hour getting the answer. You can now get the answer instantly with chat GPT. And it feels like the next step is, is actions.

• 14:19Speaker 00

  Hundred percent.

• 14:20Apoorv Agrawal?

  And it feels like the next step is, you know, with, you know, Pulse is a, is a great proactive product. That, that feels, you know, it's, I have a pulse that runs weekly. But what I would really like is like, hey, Nick spoke about something and just, just find me. Make sure I know that Nick spoke about this. Or like, hey, this X, Y, Z thing happened that I cared about a lot. When, when is, when is that going to get proactive? What is the modality going to look like?

• 14:44Speaker 00

  Yeah, yeah, so there's, there's two concepts, I think. There's chat GPT doing stuff.

• 14:51Speaker 00

  Rather than just answering. And then there's ChatGPT being proactive. And I think when you put them together, you start feeling like it feels like a super assistant. Because I think these things compound. On the action-taking piece, strictly speaking, ChatGPT can do stuff today. The action space is just very limited, right? It can search the web, which means it can use a search tool or a browser in the same way that a human would. It can make images, it can do all these things, right? But it doesn't have, it clearly doesn't have the same action space that a human with a computer would have. And that is what we aim to build. And if you, timing is everything on these bets, right? And I don't pretend to be great at timing either. When I look at past attempts that we've made, like the ChatGPT agent, for example,

• 15:33Speaker 00

  which kind of has capabilities like this. It was just slightly too early. The models weren't quite good enough to hit real escape velocity. And the problem is if you don't have escape velocities, that users don't learn to trust it. They don't even try. So when you look at a lot of things people were doing in the original version of ChatGPT agent, it was the things that happened to work, like migrating your file server into the cloud or something like that. Useful stuff, but very niche. Yeah. And as this stuff gets better, we just have to get it to a point where people try to use it for real meaningful problems in their life, because then we can start hill climbing. And this has been the magic of ChatGPT,

• 16:14Speaker 00

  where ChatGPT upon launch was good enough to get real attempts at use cases, even if they didn't initially work. Like ChatGPT was a pretty bad writer originally, it was a bad software engineer. But people tried and got enough value out of it that we could take those use cases and make them great. And I do think we're about to get to that point with general-purpose agents, where it works well enough that you get at least partial credit. And because you're getting partial credit, you get really good tasks back. And then the magic begins, because, you know, once you have a set of use cases that you can climb the hill on, we can make them awesome. So on task, I think we're close. But I think even people inside of OpenAI would have had a hard time predicting exactly when this gets good. We've been excited about it for a while. On proactivity.

• 17:01Speaker 00

  Pulse was a really great first step because what we wanted to build was a form factor where you're not prompting the model, like the model's prompting you.

• 17:13Speaker 00

  For the reasons that I described earlier, which is, you know, it's so hard for people to delegate and to figure out what their problems are. What if the AI understood your goals and the things you're interested in and just could start being proactive on your behalf?

• 17:26Speaker 00

  Pulse is limited in the value it can provide for you because it's not connected to your life and it can't take action. So it's producing information for you and people love that. You know, I love that. I've got mine running too. But I think the magic begins when you have actions and proactivity because then it can begin speculatively actually, you know, detecting, hey, you just landed where you were supposed to go. You know, I'm going to call a cab for you. Or, you know, if you're at work, it's like, hey, I proactively ran this analysis because I saw your metrics dropped. So I think these things really compound and we need to nail multiple of the building blocks to really achieve the transformation of the form factor that we hope for.

• 18:06Apoorv Agrawal?

  As you were answering those questions, I now have 15 more questions for you. So I hope you have 15 more minutes. But, okay, one by one, we'll start with what you said on actions and tasks. Got it on timing, tough to say. But is there a shape or ordinality of tasks or agents that you think, hey, this is the kind of thing that's likely to come first whenever it does?

• 18:29Speaker 00

  I mean, the thing that's already come first is the domain-specific agents, right? If you look at what's happening in code, we're fully there. You know, it's mind-bending, but we've got so many engineers who don't open their IDE like ever. And for me, as someone who, you know, used to code and then unfortunately got very, very busy, it's brought me back in the game. So Codex and products like it is clearly a product that has escape velocity where people are absolutely using it for all kinds of agentic work. And if you just take what people are doing and make it work even better, you kind of get all the way there. You know, I won't be surprised if you see this happen for other forms of

• 19:13Speaker 00

  sort of quantitative knowledge work, just because it happens to have the properties that code has. It's testable. You know if it worked or not. It's very RL-friendly.

• 19:24Speaker 00

  But...

• 19:26Speaker 00

  The domain-specific ones already work. I think the thing that everyone's working for is general purpose agents that just kind of work for anything.

• 19:34Apoorv Agrawal?

  Yeah.

• 19:35Speaker 00

  And that's why I think you need to win a consumer because it's very hard to train people into like, okay. It can work. Deep Research was a consumer product and it really was our first agentic thing out there. But I think what consumers want is I can just ask it anything and it'll do what needs to be done without any sort of retraining. And we'll get there, just a matter of time.

• 19:59Apoorv Agrawal?

  At least a psychological goal is flight bookings.

• 20:04Speaker 00

  Totally.

• 20:05Apoorv Agrawal?

  Restaurant bookings, shopping.

• 20:07Speaker 00

  All this stuff. There are so many consumer problems and those are just the type of things that you would kick off, right?

• 20:12Apoorv Agrawal?

  Yeah.

• 20:13Speaker 00

  The minute you have productivity, there's things you don't even think of as agentic tasks. Like you're trying to get in shape. You don't think of that as a task you would delegate.

• 20:22Apoorv Agrawal?

  Yeah.

• 20:23Speaker 00

  Unless you have a trainer, in which case you do, but most people don't, right? But if the E I knew that, it could totally start working in the background for you over very long periods of time and getting you, you know. Here's your fitness plan. Okay, I actually signed you up for this thing. You could imagine it being quite helpful if it's aligned with your long-term interest.

• 20:39Apoorv Agrawal?

  You're gonna give Ozempic a run for the money.

• 20:43Speaker 00

  We gotta be careful what businesses we get into, but hopefully we can help.

• 20:47Apoorv Agrawal?

  That'll be great. Cannot wait. Cannot wait. The second thing you said was proactive users and that might require us to go beyond chatbots. What's an example of a modality that might take ChatGPT beyond a chatbot?

• 21:02Speaker 00

  So chat will always be close to my heart. It's the way we grew up and it's an important modality to stay. Like I think it's less about chat and more about natural language to me where it's the fact that you can express yourself to the machine in ways that are very natural to you, whether that's text, whether that's voice, whether that is, you know, structured UI that is rendered by the model. That is just very, very powerful and that's here to stay. But I think the thing.

• 21:29Apoorv Agrawal?

  S A server.

• 21:31Speaker 00

  That's right. For those of you that know, that's the name of our codebase, short for super assistant server. Because it's proof that this was always the vision and is always the vision. But you know, the. The thing that'll change, I think, is that chat is a great way of expressing your intent. It's a good way of communicating with the machine, but it's not a great output. Where in many cases, what you want back is an artifact. Here's your, you know, plan for your trip. Here is the analysis. Here is, you know, an outcome that I delivered for you. You know, I just made you five bucks. Like, this is what I want my AI doing for me, right? Yeah, totally. I mean, this is what people care about, right? And, and, and I think chat will always be there as the way that you sort of disambiguate your intent and you kick off the task.

• 22:17Speaker 00

  But I don't think it's necessarily the, the, the final deliverable. And I think that's, that's, that's the way in which we can evolve. So hopefully that's a very graceful transition because I'm very lucky and it's hard earned. You know, to, to have a billion people coming to you weekly for a thing that they love. But I think it's a great jumping off point because we have so much unsatisfied intent. Uh, from people where they're trying, clearly trying to do something and ChatGPT is helpful enough, but it could be so much more helpful. And I think that's where we evolve.

• 22:45Apoorv Agrawal?

  Yeah, and you must be sitting on so much of this data where people are showing up to ChatGPT and attempting. As you said, uh, three years ago, they were at least making the attempt.

• 22:54Speaker 00

  Yeah.

• 22:55Apoorv Agrawal?

  So you might have at least a frequency histogram of like, hey, here are all the things that people want to achieve with us.

• 23:00Speaker 00

  We do, uh, we have like really awesome, you know, um, classifiers that run automatically. It's fully privacy preserving, but gives us a sense of, you know, what use cases people have. And it's important, right? Because when you make a new model, we make a model update, you want to know what use cases just got better or what use cases got worse.

• 23:17Apoorv Agrawal?

  Yeah.

• 23:18Speaker 00

  And that's not always, always trivial to figure out unless you have really good analytics on the system. But so much of my learning is actually qualitative where I will just, you know, I have a habit of reaching out to a fairly random set of users who just figure out what they're doing. And I've never worked on a product where three and a half years later, you're still learning every time. Because usually by that time, you know what the use cases are that your product can, you know, deliver on. But our tech is so. Unusual in the fact that I keep learning about something crazy I didn't know was possible.

• 23:47Apoorv Agrawal?

  Wow, that's awesome. But basically, a billion users, I suspect a small fraction of them are power users who are getting maybe thousands, maybe tens of thousands of value on their $200 subscription.

• 24:01Speaker 00

  Yep.

• 24:02Apoorv Agrawal?

  The vast majority is, you know, middle of the pack. And then a few, call it casual users, who are, you know, start using ChatGPT as search, maybe.

• 24:12Speaker 00

  Yeah.

• 24:12Apoorv Agrawal?

  Or teach me about AI, or help me with my homework.

• 24:17Apoorv Agrawal?

  What is your focus? Like, maybe in those constituents, power users, casual users, and early users, or however you frame it, what is our focus on for each of those three factions?

• 24:26Speaker 00

  Yeah, yeah. Well, first of all, I feel accountable to our entire user base. In fact, our non-users too, because products like ChatGPT can have real externalities on all humans.

• 24:37Apoorv Agrawal?

  Yeah.

• 24:38Speaker 00

  But when I think about sort of the way we build, it's really useful to imagine the extremes. One extreme being a user who doesn't care about AI at all, who has a busy life and needs to be convinced of the value that we can provide. Because that forces you to really nail the interface and to expose the capabilities that are hidden in the model in a way that people can actually rock. And then the other useful extreme is our power user base. Because power users are the users who teach us what's possible.

• 25:15Speaker 00

  It's actually impossible for us to do all the product discovery on our own simply because of how empirical this technology is and how much you actually learn post-launch. So building, you know, for each of those extremes can be valuable. But our user base is incredibly diverse and people have so many different use cases. And this is why, you know, I like to look at all kinds of different segmentations, not just frequency, but also, you know, what use cases are you coming to us for? But definitely huge variety in the ChatGPT user base. I look up to macOS, for example, as an example, where it really works for people who don't understand technology at all. It's entirely magical.

• 25:59Speaker 00

  But if you are a power user, you've got terminal, you've got settings, you can configure almost anything in macOS. And it's really beautifully done where the complexity is progressively disclosed. So you can interact with it and love the simplicity of it all, but you can also have got all the knobs and developers love it, right? And so I think this is kind of the inspiration for how we want to be in ChatGPT. That doesn't mean we always live up to it, but it means that building for power users is extremely important. And that's not just a property that I'm think is sort of aesthetically exciting. It's also really important in AI because it's the power users who show you what's possible. They are actually doing the product discovery because it would be impossible for us with such an empirical tech to do all the product discovery on our own.

• 26:42Speaker 00

  So the type of user who subscribes to ChatGPT Pro, who used codecs before it quite worked, who is now the strongest advocate of tools I tokens and teaching us what's possible. That is an incredibly valuable, incredibly valuable member of the community. And it might not show up in your, you know, weekly active users. It's just one number, right? But this is exactly why there isn't like a single North Star. And you really need to need to take these different segments very seriously. So I love building for power users.

• 27:18Speaker 00

  And you asked on, you know, token consumption, et cetera. It's so fascinating to see. There's people who get incredible value out of these products and watching what they do is very informative.

• 27:32Apoorv Agrawal?

  Okay, so we're very focused on the entire user base, learn a lot from the power users.

• 27:39Apoorv Agrawal?

  You know, the other thing I might say is the power users right now are getting

• 27:44Apoorv Agrawal?

  a lot of value, almost too much value. And a lot of

• 27:47Speaker 00

  No such thing.

• 27:48Apoorv Agrawal?

  No such thing. The the the analog that is most common is the Uber and Lyft of the 2015 era. Right? And you know, it took it took a while, but I know you're thinking about it a lot. I know you guys are thinking about pricing quite a bit.

• 28:01Speaker 00

  Yeah.

• 28:02Apoorv Agrawal?

  Maybe tell us a little bit about pricing.

• 28:05Apoorv Agrawal?

  You know, right now, pricing is pretty simple. Is there is there a path for for for folks who are getting a lot of great value to price that product differently? And and and you know, meet them where they are and and and and the other way on the other side?

• 28:17Speaker 00

  And pricing is, there's no world in which pricing doesn't significantly evolve when the technology is changing this quickly. Right? Chat GPT originally was entirely free, and the reason for that was that it was intended to be a demo. And we were going to wind it down after a month. We then realized that the demo went viral and people loved the demo. And it was actually a product. And but we realized it to be a product, you can't take the product down every time you're at capacity. So we, you know, ship subscriptions simply because it could shape the demand. It was a way of gracefully turning users away. And we had to turn away someone, and it felt like the fairest and most equitable way of doing so is saying, hey, you know, if you really need this product, pay a subscription fee and you got it.

• 28:59Speaker 00

  Then we figured out to make the product stable and we had the choice of do we keep the subscription thing? Or do we go back to free? And we realized we had consistently more tech that we couldn't scale, GPT four being the first example, because we had way too many free users to serve GPT four, and we put it behind the plus plan. And so, you know, the way we stumbled into subscriptions was was sort of accidental by trying to just solve for the user. And it felt like the right way at the time to to provide maximal access to our to to to our tech.

• 29:31Speaker 00

  Since then, we've had so many other breakthroughs, including test time compute, where you can scale up intelligence kind of as much as you want, more or less. And it took us, in the entire industry, a little bit of time to turn that into product value, but we're here now, where our power users want to use more and more and more intelligence. And it's possible that in the current era, having unlimited plan is like having unlimited electricity plan. It just doesn't make sense, because people may need a lot, a lot of electricity, and they're getting a lot of value out of that. There's a reason you can't buy that, right? So obviously, I want to be really thoughtful about the way that we evolve our plans and

• 30:16Speaker 00

  SKUs and subscriptions, but I'd be incredibly surprised if it didn't change, given the magnitude and profoundness of the technical breakthroughs that we've had and the product breakthroughs that follow.

• 30:28Apoorv Agrawal?

  Yeah. And, you know, relatedly, so I imagine you're gonna have something for the power users. What about the other side? How do we get the casual users into the wheel and still monetize them?

• 30:44Speaker 00

  As mentioned, our business model will evolve, and the North Star's access, right? We would like to pick a way of providing, we want to provide an offering that maximizes the number of people who can access our most powerful tools. I think for the longest time, that has been subscriptions. Subscriptions have the downside of the fact that in many markets, people don't have credit cards, or they don't use credit cards to subscribe to software. And we're interested in other ways that can maximize access of the tech. Our ads pilots are in that spirit. We really view it as a tool of bringing ChatGPT and our intelligence most broadly to anyone

• 31:28Speaker 00

  around the world. And it is an example of how we constantly need to evolve and figure out the best way to bring the demand in line with what we are able to offer.

• 31:39Apoorv Agrawal?

  Makes sense. Makes sense. You know, the ads pieces has been a tricky one, because, you know, Sam has historically expressed reluctance about ads and...

• 31:52Apoorv Agrawal?

  You've got to maintain a lot of trust while delivering that. So I guess what changed?

• 31:59Speaker 00

  I think we've talked about this several times in my history at OpenAI. And every time it came up, we said if we were to do ads, we'd have to be really thoughtful about the way we do it. So the first thing we did, you know, starting, you know, end of last year was to really engage the company on if we put ads in ChatGPT, how should we approach it? What should the principals weigh? What be? How do you preserve the things that are magical about ChatGPT? While getting the benefits of ads, which, you know, is our ability to bring our most advanced tech to anyone, regardless of their ability to pay. And I really love where we ended up on the principal's side.

• 32:42Speaker 00

  On the experience side, we're very, very early. But on the principal's side, I feel really proud because, you know, it's very important that the answer of ChatGPT be independent. As an example, respecting user privacy is very important. And there's a lot to learn from, you know, the way that tech has evolved over the last few years or really last decade. And I like that the principals are out there before we've even really gotten started. Like, we're very early with our pilots. You know, it's kind of interesting. I was obviously very anxiously and eagerly looking at our support inbounds and data. And the most common inquiry about ads is not, you know, how do I disable ads or turn off ads?

• 33:25Speaker 00

  But it's like, how do I run an ad? Because the entire ecosystem is really excited to be part of the story and to figure out a way to talk to ChatGPT users. So there's a lot more to come, but I'm very eager to get this right.

• 33:41Apoorv Agrawal?

  Yeah, I'm sure you guys will. Switching gears, Nick, something you and I have spoken about a little bit is, you know, distribution and partnerships. There's a couple of big partnerships last year. Apple, Reliance with Gemini, those are two big user bases, right? A lot of India, a lot of the iOS users. Doc, tell us a little bit of how you think about partnerships for ChatGPT to meet the user base and maybe specifically on those two as well.

• 34:14Speaker 00

  Look, I think partnerships are a great way to bring two products together and to, you know, expose something like ChatGPT to people who might not otherwise have encountered it. The thing that I care about most when considering something like a partnership is what is the user experience and can we make it amazing? Because at the end of the day, when you look at what's going on in the market, you can get users to click on things, you can get them to tap any sort of product, especially if it looks like a product they recognize, et cetera.

• 34:57Speaker 00

  But if the experience isn't truly awesome, people will churn or they will, you know, at least not retain in the way that, you know, we've been lucky to retain them on ChatGPT. So for that reason, you know, I'm super interested in paths like that, but it needs to be great. It needs to accrete the user. We are very lucky to have a great brand and a recognizable product for many folks and I want to make sure that anything we do is accretive to all that.

• 35:25Apoorv Agrawal?

  Nick, you are a master of trade-offs. You must be making a lot of trade-offs right now. Tell us about some of the trade-offs you're making. Tell us about, you know, a trade-off that you might be making that people don't appreciate from the outside.

• 35:39Speaker 00

  There are a lot of trade-offs indeed and for different reasons, right? The one I encounter a lot is trading off, delivering on the people, on the use cases that exist in the product today and making them better versus productizing, you know, step-change technology that's going to generate a whole nother set of use cases. Because when you think about how ChatGPT came to be, it was a totally open-ended product. It was basically a user experience around a technical breakthrough. And we couldn't have told you all the ways that people find it valuable.

• 36:19Apoorv Agrawal?

  Yeah.

• 36:20Speaker 00

  But putting it out there was really important because it allowed us to discover and the world to discover what you can do. And then post ChatGPT, we can obviously very systematically go and improve on the things that people actually want to use it for. And when you're at a company in this moment where you both have such amazing traction with what exists today and the most mind-bending breakthroughs on the research side, the balance you have to strike is making the core product you have better today with all the things that matter. Latency, reliability, making the use cases really great that people come to with, you know, providing access to the step change.

• 37:04Speaker 00

  And we try to get the balance right, but we're a small team and we don't always get it right. And for that reason, it's one of the most difficult trade-offs that I have to deal with.

• 37:14Apoorv Agrawal?

  Nick, I imagine one of the hardest trade-offs you guys make here is those GPUs that are melting between ChatGPT, between Codex, research. How do you guys allocate the GPUs?

• 37:26Speaker 00

  That is a very good question, and I'll let you know when I figure it out. Just kidding, you know, we've gotten a lot better at this. I really hope, by the way, to be at a point one day, and I've yet to reach that point, where we don't have to face this trade-off because it's really painful to have real user demand for products that you can't serve.

• 37:48Speaker 00

  Again, if you only ever worked in software, that's an entirely unusual dynamic, right? Where you're just, you know, or you are limited by this zero-sum resource out there. The marketplaces have it, but you know, I think pure software doesn't really have the dynamic, right?

• 38:02Apoorv Agrawal?

  Yeah.

• 38:05Speaker 00

  One thing we try to do, obviously, we prioritize our existing users first. We want to provide a fast, reliable product. And that is critical and table stakes. Then, when you look at new capabilities, the sort of naive business school thing to do would be probably look at revenue, incremental revenue per GPU or something like that. But this is where it's more an art than a science because we often have new breakthrough capabilities that are entirely zero to one. Deep research was one of those. We couldn't have told you is there going to be demand for a, you know, consumer demand for a research product. But if you don't productize it to find out, you will never know. So this is where we have to be a little bit thoughtful on how we balance, you know,

• 38:49Speaker 00

  things that are no brainers that people are really going to love with things that are brand new ideas. Then, obviously, on the research side, there's a reason that Mark has the job he has because a big part of his job is figuring out what research to fund. And, you know, obviously GPU is a big part of that. So very nuanced topic that we're continuously getting better at. But for me, the priority is always on our users.

• 39:13Apoorv Agrawal?

  Yeah. The other takeaway that I had is you don't have line of sight to a time when you won't have that problem.

• 39:21Speaker 00

  It's been so fascinating because, you know, we obviously have been incredibly lucky to encounter more and more users who want to use our technology. But then the value that we're able to provide for each user is going up as well. And GPU consumption correlates pretty well with that value. And when you just look at token consumption per user, especially in the enterprise too, which is, you know, a massive opportunity. You see a lot of very GPU hungry workflows. And yes, demand keeps going up even as prices go down.

• 40:01Apoorv Agrawal?

  This is a fascinating insight. People used to think that humans were, you know, you can't kind of make more humans. Well, it takes nine months and then 19 years. But but you're saying, well, that's actually a more more and more less finite resource than GPUs.

• 40:17Speaker 00

  Yeah, I mean, on the human side, you can hire more humans. And obviously, we've been doing busy doing that and bringing the best talent across functions to open it. In the world with agents, you can also get more leverage per human. Make your humans very effective at their job to do more. But GPUs are zero sum. And if you don't have more GPUs, you really have to figure out how do you make very, very hard trades? And I hate making hard trades for our users. Hence the desire to have more GPUs. But it's useful to start with the most zero sum trade off when you do your planning. So I think starting working backwards from GPUs is usually a pretty, pretty good idea.

• 40:57Apoorv Agrawal?

  Yeah, you know, one of the we have all these external data sources for charts of users and usage and activity and retention and all those things. What we don't have is tokens per user over time. And I bet that chart is like a sweet line going this way.

• 41:13Speaker 00

  I think internal is pretty good. Our internal employees is a pretty good indicator for what's about to happen. And yes, the charts are are are mind boggling.

• 41:22Apoorv Agrawal?

  Yeah, yeah, yeah, fascinating. Okay, a couple of quick ones on the present before we go into the landscape, which is, you know, shopping. You know, we we just moved into a new house. We took some photos and we were hoping that all our furniture would magically appear that chat G P D helped us paint. But, you know, a lot of a lot of recent updates on chat G P D shopping. Tell us, tell us about it. What are you thinking on on shopping as chat G P D as a shopping assistant?

• 41:51Speaker 00

  Shopping is one of those use cases that exists organically in chat G P T today, and they work. You can ask chat G P T about any purchase you might be planning and get pretty excellent advice. But it's also one of those cases where the experience that exists in chat today is it's not it's not the perfect experience that you would want. Because shopping is very visual, for example. So you're going to want to actually see products and images and be able to compare and contrast, not just read, you know, walls of text. And people care about the sources of, you know.

• 42:27Speaker 00

  Where can I learn more about, you know, given product, et cetera?

• 42:32Speaker 00

  And so there's a lot of work to do to make this discovery really, really good and allowing people to use ChatGPT as a, as a, as an assistant to find the right product to buy. And that's where our focus lies, is making that really great and making that really great in a way that works for our retail partners as well. Because as I mentioned earlier, there's huge appetite from the ecosystem to be part of the ChatGPT journey.

• 42:57Speaker 00

  And nailing the discovery piece has been the most promising focus here to date.

• 43:04Apoorv Agrawal?

  Nick, on ChatGPT, you must see a breadth of information. You must see a breadth of use cases that people are doing with ChatGPT. And tell us something about, you know, what does the world underestimate about ChatGPT that you have maybe been surprised by? Or a listener might be surprised by?

• 43:22Speaker 00

  There's been a real change in the way that people think of ChatGPT over the last year or so, where it's increasingly like a true thought partner to people. It's not just a thing that, you know, answers your question, but it's a thing that you can, it's a sparring partner that you can actually think things through with. And that shows up in all kinds of demands, ranging from life advice, where you know, if you've got a relationship problem, you can actually get a lot of value out of ChatGPT just helping you think through how to handle it and how to talk to your partner about it. All the way to a work setting where, you know, you're, you're working on an analysis or you're like trying to figure out how to frame something or you're, you know, trying to build something.

• 44:07Speaker 00

  And ChatGPT really shows up as, as a second brain of sorts. And I think that's qualitatively different in terms of how the mental model it occupies with people. And you see in the usage patterns and the use cases that exist. And I think the more we nail things like proactivity, which we talked about earlier, and tasks, et cetera. I think the more it's going to feel like a teammate in the workplace and like a super assistant at home. And I think that's going to meaningfully change the use cases that people come for.

• 44:37Apoorv Agrawal?

  Yeah, you know, I've been, the, the most high stakes thing I do with ChatGPT is we have a, we have a new baby. And when the baby's crying at three in the morning, ChatGPT, you know, what's going on?

• 44:51Speaker 00

  First of all, congrats. Second of all, I've heard this from all parents in my life. That ChatGPT has become indispensable as a thought partner. And it makes sense, right? If you have a really specific scenario, or you think it's a specific scenario to you, ChatGPT really comes through and can, you know, help you, you know, build confidence. And I think that's such an empowering thing, right? Like, and imagine new parents aren't always the most confident about what is the right thing to do. And if ChatGPT can make you, you know, feel like you are, you have agency and control and, you know, I think that's really valuable.

• 45:28Apoorv Agrawal?

  Yeah, it's huge. Well, thank you for making ChatGPT. It's literally getting me an extra hour of sleep every day.

• 45:33Speaker 00

  Took a village, but that is a great metric. You know, that should be the North Star metric. It's like incremental hours of sleep. That's a great one.

• 45:40Apoorv Agrawal?

  Incremental hours of sleep, incremental hours of joy.

• 45:42Speaker 00

  There you go. I mean, you joke, but like, we talk about this a lot. And because spiritually, that is pretty close to what we hope we can do, right? Is help you reach whatever you consider self-actualization.

• 45:55Apoorv Agrawal?

  Yeah.

• 45:55Speaker 00

  Whether that's sleep or joy or any other goal you might have.

• 45:59Apoorv Agrawal?

  Yeah, yeah, yeah. Well, thank you for, thank you to the village. We're going to switch gears and talk about the landscape.

• 46:05Speaker 00

  Sure.

• 46:05Apoorv Agrawal?

  There's a lot going on, on the field.

• 46:09Apoorv Agrawal?

  You know, how would you frame ChatGPT's differentiation

• 46:15Apoorv Agrawal?

  to people out there?

• 46:18Apoorv Agrawal?

  There's a lot of different products out there.

• 46:20Speaker 00

  Look, it's the best time in history to be a consumer of technology.

• 46:26Apoorv Agrawal?

  It is indeed.

• 46:27Speaker 00

  Because you've got options. And the competition is intense. And I think that's beautiful. And it's actually good for us too, because If you were to pre-mortem why a company like OpenAI does not achieve its mission, it's probably focused because of the sheer number of opportunities that become possible when you approach AGI, right? And having competition and options out there, I think it forces us to focus on our customers too, and on things that really matter, which aren't always the most flashy things, right? Sometimes it's latency, reliability, the quality of the user experience. So I think it's a really good thing. I think the biggest differentiation of ChatGPT is the team behind it because we're not static, right?

• 47:08Speaker 00

  Anything we build will get copied, and sometimes in ways that are high craft, sometimes in ways that are sort of checkboxes. And it's really important to us that we evolve the category and build the super system that we've always imagined. And I think the reason that I have confidence that that's possible at a speed that outpaces

• 47:36Speaker 00

  the dynamic of being copied is that we have an amazing team. And that we have an amazing team across research, and engineering, and design, and all the different functions that it takes to make something amazing. And I think our unique ability has been to bring those functions together to build something that is sort of at the intersection of useful and possible right in that moment.

• 47:56Speaker 00

  So my best answer for you is we keep pushing forward and we hope to be expanding what people think of this product as.

• 48:03Apoorv Agrawal?

  You know, last winter, we had, obviously, what was called Code Red. Google had a great model. There was a lot of, you know, talk about it, Marc Benioff switching very vocally to Gemini, and Austin Delaney ads, and HealthAid and some shopping, basically hit pause on everything, making ChatGPT better. Talk to us about that moment, both about what led to that and what was happening in that moment.

• 48:31Speaker 00

  Yeah, so first off, CodeReds are a tool we use to create focus. As you can imagine, when you're in a place like OpenAI, and this is what makes us special to special to work here, is there are so many different things going on. It's a research lab. We are pursuing many different ideas, right? And there's been these moments where we've wanted the company to come together to solve a problem across boundaries, you know, no matter what your project might have been. And at the last year, we had one of those moments where we felt like we need to show up for our users. We need to focus the things, focus on the basics, like reliability, performance, the way that talking to the model feels, making personalization really great.

• 49:16Speaker 00

  All these elements that our users care about, and I loved it. Because it was really an opportunity to work with a bunch of folks who I don't normally get to work with on making the product great, and we just exited the CodeRed, which we knew we would with the launch of 5.3, which is a great model for the everyday user. It's great to talk to, and 5.4, which is a workhorse if you're trying to do real knowledge work. And undoubtedly, we're going to continue to use the tool of a CodeRed whenever we want to create focus. But I'm excited because I think ChatGPT is in a great spot.

• 49:55Apoorv Agrawal?

  Yeah, so CodeRed is over now.

• 49:57Speaker 00

  That's correct.

• 49:57Apoorv Agrawal?

  It's not the new normal.

• 49:59Speaker 00

  It's not the new normal. We want it to be a special thing, but it is a tool I suspect we will continue to use.

• 50:05Apoorv Agrawal?

  That's great. That's great. And maybe tangibly, if you were to point at how did CodeRed change ChatGPT? Or maybe the ops or how the team operates?

• 50:15Speaker 00

  The thing I try to get faster with the team is focus. So we're certainly more focused than we were six months ago on the things we really want to nail. And some of those things are very behind the scenes, like latency, reliability, those kind of things. And some of those things are like very concerted efforts, like involving ChatGPT into the super assistant. And so focus is the main lasting artifact. And as you imagine, It's hard to stay focused sometimes when there's so much going on in the space. But that's the hard job. And you had asked me about trade offs earlier.

• 50:53Speaker 00

  Getting the team to focus on the things that really matter to users is certainly one of them. That's always worth it.

• 50:57Apoorv Agrawal?

  Yeah, you know, in the back of my mind as I ask you that question. Is all the other founders that are in the arena right now? And just a reminder that, hey, Code Red is a tool for you. Wartime and founder, as you used to call it, is a tool.

• 51:12Speaker 00

  Yeah, I think, you know, every company does it differently in terms of how you how you get stuff done. But I think it's really valuable to have terminology that, you know, means something. That signals to people, it's okay to drop your other stuff. And it's okay to, you know, focus on this thing together, even if that wasn't your original job. So I think it works really well at a place like OpenAI. But I imagine startups would have an equivalent.

• 51:37Apoorv Agrawal?

  Yeah, you know, one of the things that caught everybody's imagination on our team was what Peter was doing at OpenClaw. Incredibly potent to put all the tools together. Obviously, Peter is a great builder. Congrats on bringing on Peter to the team. Tell us a little bit about what Peter is working on and when when might the billions on chat GPT have something to see there?

• 52:00Speaker 00

  Well, first of all, I'm very excited for Peter to be here.

• 52:06Speaker 00

  I was excited to have another German speaker in the house. He's Austrian and I'm German. So we're exchanging guten morgens. But the, yeah, the OpenClaw is so inspiring because

• 52:22Speaker 00

  it brought to life in many ways a vision that we'd had in different forums, admittedly, around this kind of AI that is fully embodied, that, you know, exists across different UIs that can do stuff for you, that has state, that has an interaction pattern that feels a little bit more like talking to a human. Because, you know,

• 52:46Speaker 00

  OpenClaw allows you to interact in a very, very natural way, where you can send many texts back and forth and it's very curt. And so there's a lot of elements of OpenClaw that I think were very clarifying to folks across the industry. And but the best, you know, I'm super excited to just like learn from Peter and bring into the company and figure out what we can do together. So there's a lot more to come.

• 53:11Apoorv Agrawal?

  All right, so now on to the most fun section, rapid fire.

• 53:14Speaker 00

  All right.

• 53:14Apoorv Agrawal?

  You ready?

• 53:15Speaker 00

  Sure.

• 53:16Apoorv Agrawal?

  We'll, we'll start with my most, my, my, my favorite game, which is long, short. Pick an idea, a startup, a, a, a business, a product that you love, you think you're, you're very bullish on.

• 53:27Speaker 00

  Yeah, I'm, I'm, if I were starting a company today, I, I, I'm really excited about these companies that are going into companies and getting extremely hands-on and doing effectively professional services with AI, because we've saturated all the EVS and you need to get proximate to the problems. So that's, it's, it's those companies that, that I'm, I'm paying attention to.

• 53:50Apoorv Agrawal?

  Fascinating. So this is, you know, this is an example, this would be like, hey, you're going and either acquiring or, or, or going inside an operating firm that has scale and, and, and, and a humming engine.

• 54:00Speaker 00

  Exactly.

• 54:00Apoorv Agrawal?

  And making that a more efficient engine.

• 54:02Speaker 00

  Yeah, or, or, or just like, you know, you're, you're doing contracts for, for, for, for customers that have really hard problems and you're actually going in and committing to solving the problem.

• 54:11Apoorv Agrawal?

  And getting outcomes.

• 54:12Speaker 00

  Yeah, because like, you know, there's, there's a reason I think that we've made so much progress on math and coding, but not on many other domains, because those are domains we are proximate to. We as people who work in labs. And there's all kinds of other domains that we are not as proximate to. And if you get proximate, I think you can, you know, build something transformative. And I think this is more important now, because, precisely because, you know, the, the, the easy problems have been solved.

• 54:37Apoorv Agrawal?

  Yeah.

• 54:38Speaker 00

  The obvious problems have been solved by the models. Credit where credit is due, I think Notebook LM is awesome and differentiated and helps me learn new stuff. I think it's great.

• 54:47Apoorv Agrawal?

  It's so good.

• 54:48Speaker 00

  Yeah.

• 54:48Apoorv Agrawal?

  It's so good.

• 54:49Speaker 00

  I think this is the example of you can innovate and you can build something totally different. It's awesome.

• 54:54Apoorv Agrawal?

  Yeah, yeah, yeah. It's so good, particularly for, I found it for some more technical learning to be a very approachable way to, to.

• 55:02Speaker 00

  Totally. And it's really cool. I feel like AI. An underrated capability of AI is to just transform things into a different medium. And I think that's so important for learning. We just launched these like dynamic math blocks, which allow you to visually understand math inside ChatGPT. Learning is obviously a big use case for us too. And I think just being able to transform things from text to visual, you know, soon from visual to video, and like all these different media is amazing because people have such different ways of processing information. And some people are like auditory learners, some people are visual, some people like reading. So I think that's really magical and a great, great angle to take.

• 55:44Apoorv Agrawal?

  Yeah, amazing, amazing, amazing. You know, one of the things I think about a lot is education and education for kids now in school. The world's changing so fast. I'm not sure our education system's changing that fast.

• 56:00Speaker 00

  Yeah.

• 56:01Apoorv Agrawal?

  What advice would you have for students who are in school now, you know, who might have to adapt faster than the system around them might adapt?

• 56:11Speaker 00

  It's a really good question and something that I've thought a lot about myself. And, you know, the, I think the most important perma-skill in this era is curiosity, I think. Because if the machine can answer all your questions, you better have good questions. And the only way to have good questions, I think, is to pursue the things you were actually excited about from an early age and throughout your entire life.

• 56:44Apoorv Agrawal?

  Yeah.

• 56:44Speaker 00

  And I reflect on this because the only reason I'm here and working on this stuff is because I thought it was neat. When I got, you know, nerd-sniped in the interview process, right? And it was like, this is so cool. And so no matter what you're doing, I think that's an important skill is to be curious and learn to stay curious. And I think I'm confident that if you foster that skill, you will know how to adapt to, you know, an evolving landscape of tools and AIs and jobs. So that would be my advice.

• 57:17Apoorv Agrawal?

  Yeah, curiosity has always been the perma-scale. Our friend Bill Gurley wrote about it in his book, Running Down a Dream. But curiosity.

• 57:25Speaker 00

  You're gonna have to check that out.

• 57:26Apoorv Agrawal?

  Yeah. What is a job that gets more valuable, not less, as a?

• 57:34Apoorv Agrawal?

  AI gets better as AGI arrives.

• 57:37Speaker 00

  Well, I think maybe the easy answer is being an entrepreneur. Because it's the best time to build ever. In terms of like being able to self-actualize your idea. But maybe one that is maybe not obvious is I think writing, actually.

• 57:55Speaker 00

  Is very important and it's not because the AI can't write. You know, AI will become amazing at writing, just like any other domains. But because I think the skill of writing forces you to be very clear of what you have to say. And even though prompt engineering is obviously going to go away and has gone away to much extent. The idea of expressing what you want to a machine. Requires you to be a pretty good writer and a very precise writer. So I would say that that is a, you know, in any profession that involves very clear writing and therefore thinking, I think is.

• 58:35Speaker 00

  Well set up.

• 58:36Apoorv Agrawal?

  Yeah, hundred percent. Honestly, I mean, this is the whole thing about slop, right? There's just so much.

• 58:43Speaker 00

  That's the other thing. I think there's going to be a permanent need for. High quality, trusted, authoritative content. And tools like ChatGPT can help you discover that content.

• 58:54Apoorv Agrawal?

  Yeah.

• 58:54Speaker 00

  But I think the need for amazing content is also here to stay.

• 58:58Apoorv Agrawal?

  And final question, what has been your AGI, feel the AGI moment? When did you feel it?

• 59:03Speaker 00

  I've had so many, honestly.

• 59:07Speaker 00

  And it's.

• 59:09Speaker 00

  It's definitely not stopped.

• 59:13Speaker 00

  A few weeks or so after I joined. OpenAI GPT4 had finished training. And I remember trying it out and it actually. It didn't impress me at all. And nor anyone else that week. Because it kind of didn't work. And it's because we hadn't figured out how to post-train it. And I think seeing it go from kind of, wait, is this really a thing? Or was GPT-3 kind of it? To wow, actually, this is an entire step change with what felt to me at the time, who didn't understand much about AI at all, as like some tweaks or some, a little bit of final stretch work, was profoundly humbling because you can realize that you might, it might not look like we are close

• 59:59Speaker 00

  to really powerful, useful AI, but we probably are.

• 1:00:04Speaker 00

  And then the moment that, you know, really, you know, there was two things that GPT-4 did that felt like AGI to me. One is it could do poetry, and I didn't think it was possible for an AI model to do poetry. It just kind of fundamental philosophically, it just didn't feel like in scope. And then the other one was it could produce code that actually worked and compiled. And then my next moment where I stared at the ceiling just in awe was when I realized GPT-4 could just simulate an entire computer terminal. Like a full computer with commands, etc. And I'm like, wait, how would this be imbued in a language model? And there's been so many moments since then, honestly. Like reasoning was a moment.

• 1:00:44Speaker 00

  One of the moments was when, I think Mark and I were giving a demo of reasoning in front of the whole company. And this was a moment where we were still trying to kind of find use cases that were hard enough for the AI, for the reasoning to, you know, make a difference. We're way past that point, we know. But at the time, you know, I think we were having to do a puzzle in front of everyone. And I think one of the moments that maybe totally fueled the AGI is like, we were in the middle of the demo and everyone started laughing and I was like, wait, what is funny? And then I stared at the screen because we're showing this chain of thought as it was streaming out of the model. And the model swore and said like, oh damn it, may have to adjust because it realized it had made a mistake in the puzzle.

• 1:01:28Speaker 00

  And the fact that it did that, but particularly the fact that it did that in a way that was entirely emergent from the, you know, RL process, completely blew my mind and, you know, made me, you know.

• 1:01:41Speaker 00

  Feel quite humble about what else these models might be able to do. So that was one of those moments.

• 1:01:47Apoorv Agrawal?

  Yeah.

• 1:01:47Speaker 00

  And then most recently, watching people use codex. Like watching people have walk around with their computer open because they don't want the task to end.

• 1:01:58Speaker 00

  Watching people who have never coded in their life make stuff and bring ideas to life. That feels like an AGM. So honestly, it's just accelerating for me and it doesn't wear off at all. And everyone has a different thing, obviously, but those were some of mine.

• 1:02:16Apoorv Agrawal?

  Yeah. You know, it's 10 years ago, there was a product called Kite. I don't know if you remember, it was for software engineers. It was like an AI coding product. That's when I felt the hunger for personally I. And nothing happened for 10 years and then everything happened in the last 10 months.

• 1:02:33Speaker 00

  The timing thing is really hard because it's actually quite possible to predict where things will end up. I think in terms of the kind of product performance factors you're going to have. But to know when it happens, it's really hard for me to make statements on anything between sort of eventually and in three months.

• 1:02:49Apoorv Agrawal?

  Yeah.

• 1:02:50Speaker 00

  Because of all the ambiguity around.

• 1:02:53Apoorv Agrawal?

  Well, that's a tight enough window, you know, now in three months is a tight enough window.

• 1:02:56Speaker 00

  Three months is pretty, pretty okay. Try to stick to the three month plan more or less.

• 1:03:03Speaker 00

  Though, you know, my team would probably tell me we don't, but I try. But yeah, anything in between three months and eventually is difficult.

• 1:03:10Apoorv Agrawal?

  Yeah. Yeah, yeah. Well, thanks for doing it. You've got a lot going on. This was a total treat. We're so excited to see all the great products you release for us. And if there's ever we can do anything to be of help, let us know.

• 1:03:23Speaker 00

  Awesome. Thanks very much. Thanks for having me.

• 1:03:24Apoorv Agrawal?

  Of course, man. This was fun.