Ep 64: GPT 4.1 Lead at OpenAI Michelle Pokrass: RFT Launch, How OpenAI Improves Its Models & the State of AI Agents Today

In this episode, I sit down with Michelle Pokrass, who leads a research team at OpenAI within post-training focused on improving models for power users: developers using OpenAI models in the API and power users in ChatGPT. We unpack how OpenAI prioritized instruction-following and long context, why evals have a 3-month shelf life, what separates successful AI startups, and how the best teams are fine-tuning to push past the current frontier. If you’ve ever wondered how OpenAI really decides what to build, and how it affects what you should build, this one’s for you.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Jacob Effron?

  Michelle Pokrass is one of the key people behind GPT-4.1 at OpenAI. As a post-training research lead, she played a crucial role in making these models so much better for developers. I'm Jacob Effron, and today on Unsupervised Learning, we dug into everything GPT-4.1 and more. Some of my favorite parts from my conversation with Michelle include the current and future state of agents, whether future models will be purpose-built for different groups. We talked about RFT and what it will mean for builders, and tactics for figuring out what's just out of reach for the models versus way in the future. We also talked about how companies can set themselves up for success with rapid AI progress, and what kind of founders will win at the app layer. And we finally hit on what's next for OpenAI's agent products. It was an awesome episode with someone who's helping define the cutting edge. Before we get to the episode, I just have one plug.

• 0:43Jacob Effron?

  If you're enjoying this on Spotify or Apple Podcasts, please consider leaving a rating on the show. Ratings help us grow, which helps us to continue bringing on the best guests and keep the conversations coming. Now here's Michelle Pokrass.

• 0:57Jacob Effron?

  Well, Michelle, thanks so much for coming on the podcast. Really appreciate it.

• 1:00Speaker 01

  Yeah, thanks for having me. Very excited to be here.

• 1:02Jacob Effron?

  Yeah, there's a ton of different things I'm excited to explore with you today around GPT-4.1. You mentioned the model has like more of a focus on real-world usage and utility and less on benchmarks. And I feel like that's definitely resonated in the Twitter discourse and just people playing around with the model. How do you actually go about making that happen in practice?

• 1:19Speaker 01

  Yeah, it's a good question. I mean, the real goal of this model was something that's a joy to use for developers.

• 1:27Speaker 01

  Often, you know, and we're not the only ones who do this, but sometimes you optimize a model for benchmarks, and it looks really great, and then you actually try to use it and you stumble over basic things like, oh, it's not following my instructions, or the formatting is weird, or, you know, the context is too short to be useful. And so with this model, we really focused on what have developers been telling us for a while now that they want, and how can we reproduce this feedback? So a lot of the focus was on, you know, talking to users, getting their feedback, and then turning that into an eval that we can actually use during research. So I would say there was like a pretty long lead up before we even got to model training. We were just kind of getting our house in order on evals and understanding where actually are the biggest problems in our models.

• 2:10Speaker 01

  And so we actually put this in the blog post, but we have an internal instruction following eval. It's based on real API usage. It's based on what people have told us. And this was kind of one of the north stars while developing this model.

• 2:22Jacob Effron?

  Yeah, I'm curious, because I've heard you talk about this idea of picking evals and, you know, basically going to startups and people that are building on top of the API and saying, what are the things that the models can't do? And let's try and hill climb on those. How do you go about figuring out? I'm sure like everybody has their pick of like, you know, 15 things they want you to optimize for. How do you like go about figuring out what are the evals that matter? And any learnings on that, like over the course of building this?

• 2:43Speaker 01

  Yeah. I will say it's actually more of the opposite problem. They're not coming to us with like, oh, I have these 100 evals. Please fix all of these. It's more like they're saying, ah, it's kind of weird in this one use case. And then we have to be like, what do you mean by that? We like actually, you know, get some prompts going and figure it out. So I'll say a lot of the legwork has been just like talking to users and really pulling out the key insights. There's actually an interesting insight I got recently from talking to a user where it turns out our models could do better on kind of, sometimes you want to tell them, ignore everything you know about the world and only use the information in context.

• 3:25Speaker 01

  This is something we would never see in an eval, like AV, GQ, none of them look at this. But for this specific user, what's most important is the model will attend only to the system instructions and ignore everything it already knows. So, so yeah, back to like the question of how do we determine what's most important? Basically just see what comes up over and over again in themes with customers. And then we also, you know, use our models internally and we have a sense for where they're not doing well. And we also have internal customers building on top of our models. So basically, all of these things put together make it, that's how we determine which set of evals to really go after.

• 4:01Jacob Effron?

  Do you have a request for evals for our listener base? Like, are there some areas where you're like, "Oh, we really would love more, you know, examples or things to test around certain areas?"

• 4:09Speaker 01

  Yes, yes, always requesting more. I'm always pitching, like, we have this eval product where you can opt in, and you get free inference on the evals, in exchange, we get to use them. But in particular, the things I'm interested in are more long-context, like, real-world evals. It's really hard to make a long-context eval. Like, synthetic evals are nice to, like, target really niche use cases, but if you want to get, like, holistically, "Does this work in long-context?", we're kind of, you know, we could use more of those. And then the other one is instruction-following. This is, like, the hardest thing to define in ML, I feel. Everyone is like, "The model didn't follow this instruction, it's not good at this." But people actually mean hundreds of different things, and so, anything more there, I'm always interested.

• 4:54Jacob Effron?

  Did you have any favorite, like, random evals that emerged in this process? I mean, you mentioned, obviously, the, you know, some examples already, but like, any that, you know, were surprising, I guess, and things that weren't working, or you thought were, like, particularly fun ones to hillclimb on?

• 5:07Speaker 01

  This is interesting. Like, we tested a few different versions of 4.1, and with real alpha users and got their feedback. And one customer just really preferred the first version over the fourth one, which is the one we ended up shipping, and they were the only user to feel this way. And all of the evals were up and to the right between these, and we just could not figure out what it was. And, you know, it was just some really niche use case that wasn't covered anywhere.

• 5:33Jacob Effron?

  Hard to please everyone with these models.

• 5:36Speaker 01

  It's nearly impossible, but if you make something that follows instructions pretty well, then you can try to please more people by teaching them to prompt better. And then, you know, the fine-tuning offering, I think, is a really great way of pleasing more people.

• 5:50Jacob Effron?

  100%. Well, we'll definitely dig into both of those aspects, you know, here. I guess, like, I'm curious, you know, the model's been out for a few weeks now. I'm sure you had all these, you know, and you were obviously testing this with plenty of people, so you had some sense of how people would use it. But then, it's always fun to get it in the wild and see, you know, all sorts of unexpected ways. Any kind of, like, unexpected things that it's been able to, like, bridge or solve that have been kind of fun to see these last few weeks?

• 6:13Speaker 01

  Yeah, I've really loved seeing a lot of the cool UIs people have been building. So actually, this is something we snuck in near the very end of the model is like much improved UI and coding capabilities. So I've seen really cool apps there. I've also loved seeing people make use of Nano. It's, you know, small and cheap and fast. And I saw, I think Box has some product feature where like you can read 17 pages of docs and

• 6:41Jacob Effron?

  I know Aaron tweeted like some results using the using the models and it was pretty impressive uplift in the in the core product.

• 6:47Speaker 01

  Yeah, it's very cool to see like the hypothesis behind Nano was can we just spur on a ton more AI adoption with models that are cheap and fast? And it looks like the answer is yes, like people just have demand at all points in the cost latency curve.

• 7:01Jacob Effron?

  I feel like that that answer seems to have generally been yes throughout this. You know, you guys are always cutting prices and it seems to always keep spurring spurring more demand. You know, I feel like you've been acknowledged by Sam, I know, by all sorts of folks that's like, you know, really one of the ringleaders of making this whole thing happen. What is actually involved in like shipping a model like this end to end? And what work are you guys doing behind the scenes to kind of like make this happen?

• 7:23Speaker 01

  Yeah, it's a great question. So obviously, there's a large team behind the scenes. And so we have basically these three models are each kind of a semi new pre-trained. So we have the standard size of Mini and the Nano. So really great work from the pre-training teams and to

• 7:44Jacob Effron?

  What does a semi new pre-training mean?

• 7:46Speaker 01

  Yeah, it's a good question. I mean, it's kind of like a we call it a mid-train. It's a freshness update. And so the larger one is a mid-train, but the other two are new pre-trains. And then my team works a lot on post-training. So we've been focusing a lot on, you know, how do we determine the best mix of data? Or how do we determine the best parameters for RL training? Or how do we determine the weighting of different rewards? And so back to like how this all came to be, I think, we started realizing, you know, a lot of developers had a lot of pain points with 4.0. And we went, I would say like, three months in on evals and figuring out what the real problems were. And then I would say the next three months was kind of a flurry

• 8:28Speaker 01

  of trading. And so we would just run tons of experiments, like, how does this dataset work? Or like, what if we tweak these parameters? And then that all kind of linked up with these new pre-trains. And then we finally had like about one month alpha testing where we were trading stuff really rapidly and getting feedback and trying to incorporate that as much as possible.

• 8:47Jacob Effron?

  You know, a part of this is it was gathering these evals. Like, does it feel like that set of evals is still relevant? Or is it now like you have to go gather a whole new set of stuff that like, you know, maybe is the right stuff to hill climb on for, you know, improving upon 4.1?

• 8:57Speaker 01

  Yeah, I think the shelf life of an eval is like three months, unfortunately. Like, progress is so fast, things are getting saturated so quickly. So we're still on the hunt, as always. And I think we always will be.

• 9:10Jacob Effron?

  I mean, one of the things that obviously is so clear in the model is that you improved instruction following, you improved long context, obviously both incredibly beneficial for agents. You know, I think our listeners are always trying to figure out, like, where are we with agents? Like, how do you characterize today? Like, what does work, what doesn't work? Like, what is kind of the state of the field post, post 4.1?

• 9:27Speaker 01

  I think where we are is that agents work remarkably well in well scoped domains. So, you know, a case where you have all the right tools for the model, it's fairly clear what the user is asking for. We see that all of those use cases work really well. But now it's more about bridging the gap to like, the fuzzy and messy real world. It's like, the user typing something into the customer support box, you know, doesn't actually know what the agent can do. And the agent maybe is missing an awareness of its own capabilities. Or maybe like, the agent isn't connected enough to the real world to know, you know, a certain piece of information. So I think, honestly, I think a lot of the capabilities are there. But it's just so hard to get the context into the model. And then one area

• 10:12Speaker 01

  I do think we can improve is like, ambiguity. Like, we should make it easier. For developers to tune, you know, if it's ambiguous, should the model ask the user for more information, or should it proceed with assumptions? It's obviously super annoying if the model is always coming back to you and be like, should I do this? Are you sure? Like, can I do this? I think we need like more sterability there. But yeah, I would say.

• 10:32Jacob Effron?

  We've all, we've all worked with interns like that before. So I get that there's a fine balance to strike.

• 10:37Speaker 01

  You want some delegation, but not too much.

• 10:40Jacob Effron?

  It sounds like, you know, the underlying capabilities of the models, you know, in many ways aren't being fully shown just because we haven't connected enough context in or tools into the, to the models themselves. And it seems that there's a lot of, a lot of improvement on, on just doing that.

• 10:52Speaker 01

  Yeah, exactly. Yeah, I would say when we look at like some of the external benchmarks for function calling or agentic tool use, I, when we actually dig into the failure cases, like where our models are graded incorrect, I see that they're mostly misgraded, or maybe it's ambiguous, or maybe they're using a user model and the user model isn't following instructions well enough. And so we're actually struggling to find cases where the model actually just does the wrong thing. There, there obviously are those, but most of the benchmarks, I would say, there are saturated.

• 11:25Jacob Effron?

  I imagine like over the next six, 12 months, a lot of that stuff gets added in, you know, more, there's more tools, more context. You know, I feel like one of the, the gaps remains kind of like longer term, you know, task execution. Like, how do you think about what needs to be done to kind of continue making progress toward some of these longer, like more, you know, ambiguous, you know, many step tasks?

• 11:42Speaker 01

  Yeah. And I think we need changes like on the engineering side and the model side. So on the engineering side, we need APIs and, and, you know, UIs where it's much easier to like follow along with what the agent's doing, a summary of what they're up to, a way to like jump in and, and change the trajectory. We have that in Operator, it's pretty cool. You can kind of jump in and like steer. But you don't have that as much for other things in our API. And so I think I think that's a core capability on the engineering side. And on the modeling side, I think we need more robustness, like when things go wrong. Obviously, sometimes your API will have a 500 and the model will kind of get stuck. And so I think we're hoping to trade in more of more robustness and like grit is another way we think about it sometimes.

• 12:27Jacob Effron?

  Another part of the of the models that I think everyone's noted on, and obviously you have in the benchmarks, is just how much better they are at code. And so I guess, you know, to start there, like, how do you kind of, you know, characterize where we are with like what, you know, where we are with AI code? Like, what works, what doesn't?

• 12:41Speaker 01

  Yeah, totally. So I think where we are for code is that 4.1 and some of our other models are remarkably good when the problem is like locally scoped. So maybe you're asking the model to like change some library and all of the files are, you know, near each other and it makes a lot of sense. But we see like the Sweetbench tasks that we're missing, for example, are those where the model really needs global context. And it needs to like reason about many various parts of the code. Or maybe there's like some extremely technical details in one file and you're trying to pass them into another. So I would say like, we're still improving of that global understanding.

• 13:24Speaker 01

  I also think we've made a really big improvement on the front-end coding. But I still would love to continue improving like we should. We should not only produce front-end code that's beautiful, but like a front engineer should be proud of it. And there's some linting stuff there and code style is another top focus area for us. And finally, I think another thing we're always going to keep improving is like changing only what you ask for and not everything else. Like, the model should adapt to the style of your code and not kind of inject its own style too much. And on our internal evals, we see, you know, it went from, I think, 9% to 2% from 4.0 to 4.1.

• 14:04Speaker 01

  These like irrelevant edits, but obviously 2% is not zero, and so it's something we're going to continue improving.

• 14:11Jacob Effron?

  Yeah. What does that mean for how then you end up using it in your kind of day-to-day coding?

• 14:15Speaker 01

  Yeah, I manage a team now, so there's not

• 14:19Jacob Effron?

  Yeah, alas, the inevitable trajectory of doing well at these companies.

• 14:24Speaker 01

  But I do use Codex, and I have honestly still been using GitHub Copilot. It's still a great product, and I also dabble with WindSurf and Cursor. So in and out, but Codex is really cool, the way it does stuff independently. And I think the main model I use there is O4 Mini, just for speed.

• 14:43Jacob Effron?

  Obviously, you've alluded to this, there's lots of benchmarks, and I feel like people are always debating, are benchmarks still relevant? I think you guys even added some into 4.1. I think there's been this feeling in coding for a while, for example, benchmarks don't tell the full story, and you know it when you use it. To what extent is that true? And what's your overall view on the state of these benchmarks today and how useful they are?

• 15:05Speaker 01

  Yeah, I do think Sweetbench is still a useful benchmark. The actual differences from a model that can achieve 55 versus 35 are staggeringly different. I think the ADR evals are still super useful. But then there's ones that are just fully saturated and not useful. Basically, you gotta use the most out of an eval during its lifespan, and then move on and create another one. And so I do it.

• 15:32Jacob Effron?

  The three-month shelf life definitely is tough.

• 15:35Speaker 01

  Yeah, there's gonna be successors to Sweetbench once that's saturated, for sure.

• 15:39Jacob Effron?

  Yeah, I mean, one thing I think that's so interesting about 4.1 is that I think you guys have been very explicit, like this was built for developers. And there's all these evals you did to make it better for the things that developers were asking you for. And it does beg the question, how does the OpenAI model family evolve from here? Because obviously, you could imagine a pre-trained model that's post-trained for different end-users, or domains, or tasks. I'm sure you guys learned a ton, kind of building this model for this explicit end-group. How do you think about that?

• 16:05Speaker 01

  In general, my philosophy is that We should really lean into the G and AGI and try to make one model that's general.

• 16:16Speaker 01

  And so ideally, I think going forward, we're going to try to simplify the product offering, try to have one model for both use cases and, you know, simplify the model picker situation in ChatGPT as well. But for 4.1, we thought there was like a particularly acute need and we thought we could move a lot faster at this problem if we could decouple from ChatGPT. So this let us, you know, train models, get feedback much quicker, ship on a different timeline. And it also let us make some interesting choices with model training. So we were able to remove some of the data sets specific to ChatGPT and we were able to like upweight the coding data significantly. And so this is stuff you can do when you're kind of targeting a separate domain.

• 17:01Speaker 01

  But in general, I do expect us to simplify. And I think the models get better when like the creative energies of all researchers at OpenAI are working on them, rather than, you know, the subgroup focused on the API right now.

• 17:13Jacob Effron?

  Well, it also seems like there's been massive like cross-domain generalization anyway, where, you know, in general, it feels like, you know, putting it all into one model has been beneficial. But it's interesting, obviously, that this, this has been such a success with that more targeted approach.

• 17:24Speaker 01

  Yeah, there's room for both, I think. Like, sometimes it makes sense to eject and ship thing for a user you know really well.

• 17:31Jacob Effron?

  Do you think that, do you think that's something you guys might do again?

• 17:33Speaker 01

  Yeah, I think it's possible. I mean, we, we don't, like, we make a lot of changes on the fly, like, as we see what demand is there, and it's definitely possible.

• 17:42Jacob Effron?

  Well, one thing I obviously hear from folks all the time is you guys ship models very rapidly. I know the, the naming has always been debated ad nauseam about how many different models there are. I feel like companies are trying to keep, you know, stay on top of like what the cutting edge of model capabilities are. You know, any best practices you've seen from like, you know, what, what companies do to like just stay on top of, you know, it feels like a new model drops every like month in this space. And, you know, how would you be thinking about it if you were at one of the users of these APIs?

• 18:09Speaker 01

  It's all back to evals, unfortunately. Like, the most successful startups are the ones who know their use case really well, have really good evals, and then they can just spend, you know, an hour running evals on the new models when they drop. There's also, I think, like, the customers are really successful are the ones who can switch their prompts and their scaffoldings and tune them to the particular models. So that's what I would recommend. And then the other thing is to Build stuff which is maybe just out of reach of the current models. Or maybe it works one out of 10 times and you'd love it to be nine. If you have these kind of use cases in your back pocket, the new models drop and things just work, then you're first to market.

• 18:52Jacob Effron?

  Do you have a heuristic you use for what's just out of reach? Obviously, I feel like it's hard to tell sometimes, like if it's, you know, how soon some of these things might work.

• 19:00Speaker 01

  Yeah, I think if you see like significant improvements in fine tuning, like let's say you're getting a 10% pass rate, you can fine tune it to 50%, it's probably not good enough for your product yet. That's something that's right on the cusp in a future model, a few months from now will probably just crush it.

• 19:17Jacob Effron?

  No, that makes a ton of sense. I mean, obviously, you mentioned kind of like the, you know, being able to switch the prompts and the scaffolding. I think one thing that like I think a lot about on the investment side is, you know, there's lots of companies that, you know, the models are able to do what they're able to do. There's all sorts of scaffolding they build, you know, based around those limitations to make the products work today. And then it feels like, you know, you guys released the next great model and some of that scaffolding just gets obviated. It's like, okay, cool. Like the models are way better at like following instructions. I don't need to do all this hacky stuff because you have this long context window now. Given that, how do you think about when it like does and doesn't make sense to like build some of the scaffolding? Or like what set of scaffolding makes sense for these folks, you know, for people to focus on?

• 19:55Speaker 01

  I like to take this back, I guess, to like your reason for being as a startup. Like your reason for being is to ship value to your users and make something people want. And so I think it is super worth it to build the scaffolding and like make your thing work. You basically are doing like a few months of arbitrage before this capability is available more easily. But I do think it's important to keep in mind future trends. So like maybe build the rag thing for now, or maybe like have your instructions five times in the prompt, although not with 4.1. But just be prepared to change things, but know where things are going. So, you know, I think context windows are only going to keep improving. I think reasoning capabilities are only going to get better. Instruction following is only going to get better.

• 20:40Speaker 01

  And so just have an eye to where those trends are going.

• 20:43Jacob Effron?

  Yeah. Any other like, you know, for where things are going, like tips for folks?

• 20:48Speaker 01

  Yeah, I think multimodal is another one. The models are getting so natively multimodal and easy to use in those.

• 20:56Jacob Effron?

  Yeah, I feel like that's been a pretty under discussed part of a 4.1. It's pretty impressive multimodal capabilities.

• 21:01Speaker 01

  Yeah, honestly, huge shout out to our pre-training teams because these new pre-trains have just significantly improved upon multimodal. And I think we will continue to see these improvements. But so many things that didn't work in 4.0 just work now because, you know, the models have gotten better there. And so it's worth it to connect the model to as much information about your task as possible, even if you're getting bad results today, because tomorrow it'll get better.

• 21:30Jacob Effron?

  And you mentioned like fine tuning. I mean, I think it's interesting, right? I feel like we've gone through this journey with fine tuning where, you know, early on, I feel like folks, a lot of folks were like, I don't know how helpful this actually is. And then it feels like there's been a renaissance of fine tuning with these newer models and how helpful it actually is. Like, I guess I'm curious what you've observed. Like, does that arc ring true to you? And like, how should people be thinking about this? And should more people be revisiting their prior assumptions around fine tuning?

• 21:53Speaker 01

  Yeah, I think I would bucket fine tuning into two camps. The first is fine tuning for speed and latency.

• 22:01Jacob Effron?

  Yeah.

• 22:01Speaker 01

  And so this is still, I think, the workhorse of our SFT offering.

• 22:05Jacob Effron?

  Yeah.

• 22:05Speaker 01

  So 4.1 works well, but you can get it at, you know, a fraction of the latency. But then, you know, I think we haven't seen too much of fine tuning for frontier capabilities. You can, could maybe get them in a really niche domain with SFT. But with RFT, you can actually push the frontier in your specific area. And the fine tuning process is so data efficient that you can just make deal with like 100 samples or something on that order.

• 22:35Jacob Effron?

  Yeah.

• 22:35Speaker 01

  So our RFT offering is actually shipping to GA next week. I guess your readers, your listeners will probably hear about it when it's out. And we're really excited about that. There's some use cases where it works really well. For example, like teaching an agent about how to pick a workflow or how to kind of...

• 22:56Speaker 01

  You know, work through its decision process. Then there's also some interesting applications in deep tech. Where, you know, maybe the startup or organization has data that other folks don't have and it's really verifiable. And from that, you can get the absolute best results with RFT.

• 23:13Jacob Effron?

  I think one thing, you know, that I've been struck by at least is like, it feels like across the board, the number of examples you need is not massive, right? I think in the early days, people, you know, were like, oh, well, like some of these companies sit on tens of thousands of examples and, you know, they'll just be able to, you know, outcompete. And it's like, you know, it feels like, I mean, the data really does matter, but it's maybe to the tune of a lot less examples than, than folks might have previously thought.

• 23:34Speaker 01

  Yeah, I think these two trends are making fine tuning more, more interesting, where like it's extremely data efficient. And also, RFT is basically the same RL process we use internally for improving our models. So we just know that it works remarkably well and it's, it's less fragile than SFT. And so, yeah, for those reasons, I think it's, it's going to be really useful for deep tech and some of the hardest problems.

• 24:01Jacob Effron?

  Is this the kind of thing you think everyone should play around with? Or like, you know, is it like, I mean, obviously, there's some cases the models can do, but let's take, you know, almost anything that maybe they aren't as accurate as folks want. Is it like worth trying this for, for, you know, for, for any of those cases?

• 24:14Speaker 01

  I think my mental model is if it's a stylistic thing, then you should probably use preference fine tuning, which we launched.

• 24:20Jacob Effron?

  Yep.

• 24:21Speaker 01

  Somewhat recently. If it's more simple, like, you know, maybe you want Nano to classify things and it gets, you know, 10% of cases wrong and you can close that gap with, with SFT, that's great. But then for the things where just no model in the market does what you need, then you should turn to RFT.

• 24:39Jacob Effron?

  It's not like you were kind of alluding to the fact that there's like some things, especially when they're verifiable, that like make this easier to do. Like, do you have like any rough, like rules of thumb you use for like when RFT, you know, the, the types of domains or the types of problems that RFT will be like particularly effective for? Or like what these more easily verifiable domains are? It's like everyone's asking this question now outside of, outside of code and math.

• 24:59Speaker 01

  Yeah, I think there's stuff in like chip design or in biology, just like.

• 25:08Speaker 01

  Stuff like drug discovery, I think those sorts of of things where maybe you need to explore, but the things that work are easily verifiable. I think those will be good applications.

• 25:19Jacob Effron?

  Certainly, chip design is that. I feel like drug discovery perpetual awesome use case, but sometimes it takes 10 years to figure out if it actually actually works in people. So the feedback loop is always obviously interim steps in between. But it does kind of beg the question, I mean, it's like, you know, you see in 4.1, obviously, these multimodal capabilities, you know, you talk about kind of the ability to use R F T for, you know, for biology. I guess there's always been this question of like, you know, are there going to be like standalone types of foundation models, like a robotics foundation model or a biology foundation model that is like nothing or has something to do, but like it's kind of a separate class of models like what's your kind of view on that? Does it feel like it's, you know, you kind of mentioned the G and A G I before, like, does it feel like we're converging in that in that aspect?

• 26:01Speaker 01

  I kind of do. I think generalization, you know, improves capabilities a lot. I think it remains to be seen with robotics, like, you know, I guess we'll know empirically if the best robotics products are their own models. But I do kind of think, and I think the trends I see here internally are combining everything just produces a much better result.

• 26:22Jacob Effron?

  Everyone's teased that you'll have soon, like, you know, one model that will like pick up behind the scenes for people what to what to use. But obviously, we don't have that yet today. And so, you know, I'm curious if I'm a company and, you know, figuring out like which, you know, obviously, I'll probably test a bunch of them. Do you have any rough rules of thumb on like which models people should be choosing for the different things they're trying to do?

• 26:42Speaker 01

  Yeah, totally. It's a pretty tough decision tree, so I'm excited. We're going to simplify it. Here's how I think about it. In chat G P T, obviously, in chat G P T now. And so my

• 26:54Jacob Effron?

  You and me both.

• 26:56Speaker 01

  Yeah. My main model there is 4.0, and I use 4.5 sometimes for writing or creative stuff. And then 03 is what I use for like the hardest math problems or like, I don't know, it's filing my taxes and I wanted them done right. So that's You know, somewhere where I'd use O3. Does that line up with you? Are those the models you use in chat?

• 27:16Jacob Effron?

  I wasn't sure the models were good enough yet to trust my taxes to them. So I haven't yet done that. But maybe I should. If you're saying it's good enough, that's great. Next year, I will totally go ahead and do that.

• 27:23Speaker 01

  I'm more double checking my CPA. Verify with the trusted SARS.

• 27:28Jacob Effron?

  But yes, that definitely lines up on the kind of consumer side. And then I'm curious like for the enterprise users, like, you know, how they, you know, obviously, I feel like you always want to go as fast and cheap as you can. But I think folks are still trying to figure out exactly like when to reach for each different kind of model.

• 27:44Speaker 01

  Yeah, totally. So yeah, how I think about it there is developers should just start with 4.1. See if it works well for their use case. If it does, then you're looking for faster. Then I would look into mini and nano and fine tuning those. Obviously, mini next and then nano as the smallest model. And then if some things are just out of reach for 4.1, then I would push for O4 mini and see if you can kind of, you know, get the sufficient like reasoning capabilities out of it. And then you go to O3. And then if that's not working, then you go to RFT with O4 mini.

• 28:18Jacob Effron?

  I guess on the other side of using these models, one thing I always enjoy is like the prompting guides you guys release behind alongside these models because it's always kind of like funny sometimes, counterintuitive, like the different things that help on the prompt side. Like, any particularly favorite things that have emerged is like, oh, that's actually a really helpful way to prompt, you know, 4.1?

• 28:35Speaker 01

  Yeah, I think we found XML or like structuring your prompts really well. Works super well. The other thing is just telling the model to keep going.

• 28:45Jacob Effron?

  I liked that one.

• 28:47Speaker 01

  It's something we're hoping to fix for the next one. But it is remarkable like how much better performance you can get by telling the model like, hey, please don't come back to me until you've solved the problem. So yeah, those are interesting and somewhat counterintuitive.

• 28:59Jacob Effron?

  How do you go about like, so like, yeah, you've seen that keep going thing. And obviously, in your cookbook shows like a big impact. How do you then go about like, you know, incorporating that into the next generation of models such that like that isn't a thing anymore?

• 29:11Speaker 01

  Our post-training process can be pretty sensitive to the exact mix of data used. So, you know, you can imagine a post-training process where you train the model on one diff format. And then, you know, your users are using totally different diff formats and the model's a bit lost. Whereas for 4.1, we trained the model on like 12 or so different diff formats, everything we could think of. And so our goal is to really put out something that works really well and even document maybe the best one. So our prompting guide has, you know, diff formats. We found that worked well. We also want it to work well out of the box for developers who aren't going to read our docs, which I like, you know, recognizes most. You want

• 29:50Speaker 01

  it to work anyway, even if you're not using the best. So we focus a lot on general prompting and general capabilities. And this way, we don't, you know, kind of burn in the model a specific one.

• 30:04Jacob Effron?

  Yeah, though keep going is a great thing to say to our team internally too. So, you know, definitely it helps across the board. You've obviously mentioned evals as one thing that like the most sophisticated companies do well. I'm curious if like there's something that you've, you know, either maybe some of the OpenAI products or some techniques that like a select few companies are using really well. And you're like, God, I just wish that like thousands of companies were using this or thinking about things this way.

• 30:27Speaker 01

  Yeah, I think some of my favorite developers to work with are those who know their problem really well and actually have evals for the whole problem, but can break them down into specific subcomponents. And so they can tell me things like, the model got better at picking the right SQL table by this percentage, but it got worse at picking the right columns by this percentage. And it's like, wow, this level of granularity really helps you tease out like what actually is working and what isn't. And then, you know, they can tune specific parts of this. So I guess like making your system modular and easy to plug different solutions into, I think that takes a little time upfront, but makes you move faster in the long run.

• 31:10Jacob Effron?

  I guess a question people are always asking is like, how much AI expertise will like the, you know, will like the leading AI app companies need versus just like being good engineers that take your models off the shelf and know their end customer? Like, do you think long term, you know, being able to kind of like have a sense of what data to apply on the fine tune or like, you know, tweak your evals? Does that end up being a really important skill set for the app players? Or is it really like, no, they can kind of take the models mostly off the shelf or, you know, a basic fine tune and that kind of core AI research capability may be less important?

• 31:40Speaker 01

  Yeah, I'm really long generalists. So I think people who understand the product are really scrappy engineers who can do anything like. I honestly don't think you'll need that much expertise to combine these models and these solutions in the future. So yeah, I'm definitely much more bullish when I hear about a team of like scrappy hackers than, you know, a bunch of PhDs with only like research publications under their belt.

• 32:08Jacob Effron?

  And there's so many exciting areas to continue pushing these models forward. Like what future areas of research are you most excited about to like make these models better?

• 32:16Speaker 01

  I'm really excited about using our models to make models better. And so this is particularly useful in reinforcement learning when we can use, you know, signals from the models to figure out if the model's on the right track. Yeah, I'm also excited. This is like a more general research area, but we're working on improving our, our speed of iteration. So the more experiments you can do, that just like the more research gets done. And so it's a real focus right now to make sure, you know, we can run our experiments with the fewest number of GPUs and get, you know, you want, you basically want to kick off a job and know when you wake up in the morning that you know if this thing is working or not.

• 33:00Jacob Effron?

  Is that just like a pure infrastructure problem or like, you know, like the latter part?

• 33:04Speaker 01

  Not really. You also need to make sure that kind of the things you're training are at sufficient scale to get signal on what exactly it is you're experimenting with. So also some interesting ML problems there.

• 33:17Jacob Effron?

  Yeah. And then in terms of like using the models to make models better and kind of signals if you're on the right track, like where, where are we in that? Like, does that work or like, you know, are we still kind of early, early innings of that?

• 33:29Speaker 01

  Yeah, it works remarkably well. I think synthetic data has just been like an incredibly powerful trend. So yeah, excited to push this more, but every more powerful model makes it easier to improve our models in the future.

• 33:44Jacob Effron?

  You guys have also shipped some really interesting, you know, agents, I think, deep research probably most famously, a product I use all the time, you know, and basically, you know, as I understand it, like using reinforcement learning like on, you know, a tool or a set of tools, right, until the model gets really good at using it. How do you imagine that like type of approach scaling for agents at large? I guess it's kind of like a sub-variant of the question we were talking about earlier of like building these like specific models for, you know, end users or specific, you know, specifically doing RL on tools versus like the G of, of generalization here?

• 34:14Speaker 01

  Yeah, so I think deep research is like zero to one, or deep research in operator, like zero to one or two, where you want to train the model like really deeply on this specific thing. But I think what we've seen with O3 is that we can just train the model to be great at all kinds of tools. And actually, learning to use one set of tools makes it better at other sets of tools. So I don't expect too much of just like one tool-specific training going forward. It's like we've kind of proven that out, and now we can incorporate those capabilities broadly. And actually, that's one thing people really love about O3, is that it can do a lot of

• 34:54Speaker 01

  deep research, like all of those capabilities, but quicker and, you know, you get it. You can really go for deep research when you want like the absolute best report, but if you want something somewhere, you know, in between, then O3 is a great fit for that.

• 35:09Jacob Effron?

  Yeah, and as the kind of general models, you know, get better at using tools and, you know, and doing some of these tasks, you know, are there areas that you think will like be easier or harder? I mean, obviously, you guys have publicly said you'll have a coding agent. You know, I don't know if there's like, as folks are thinking about again, like what's on the, you know, you know, what capabilities are sooner rather than later? Any just like mental model you use of like, yeah, I think these things would be, would come before the next set of things?

• 35:34Speaker 01

  Yeah, I think, I mean, yeah, coding is obviously coming soon, given like sweep bench numbers are already exceeding, you know, what a lot of humans would get there. So I think the ability to supervise these long runs is there. In terms of other stuff, I think, like, long workflows. So what's interesting about O3 already is that when it calls developer-specified tools, they're already part of the chain of thought of the model. So the model can You know, use the thoughts of the previous tool call and the output and think some more about what to do.

• 36:13Speaker 01

  And so I think because of that, the agentic, like maybe customer support or other sorts of capabilities, I think are, I think personally are there and it just needs to be hooked up with everything to make a cohesive product.

• 36:27Jacob Effron?

  Yeah, I mean, it seems like in many ways, like the capabilities of these models like exceeds like the actual like nitty gritty, just implementation of like, yeah, hooking them up to things, getting them, getting enterprises ready to use them in some way. But it's like, you know, I, I think there's always this big debate of like, if you stopped, if you completely stop model progress right now, is there like just tens of trillions of dollars of value to be extracted from me, from just like, you know, from, from these models? And it seems like you're very much in the, in the camp of yes.

• 36:53Speaker 01

  Yeah, I mean, I think, if you think about like the capabilities overhang of the internet, it's, we still haven't saturated, like things are coming online, like internet is still eating the world. And I think for AI, like we haven't even saturated capabilities 3.5 turbo, like I think, still think they're billion dollar companies started that only need that level of capabilities. And so now with 4.1 and these reasoning models, like I think we have, you know, if, if we truly stopped right now, I think we'd have 10 years of, of building at least.

• 37:24Jacob Effron?

  Sam's obviously talked about combining the model families into, into this GPT-5 that will, uh, will, will probably end the really fun, you know, point this and point that and all that. But like, what, what actually needs to be done to like combine this into, into like a model, a single model?

• 37:38Speaker 01

  It goes back to like, what are the models good for? So right now, the 4.0 series is really great for chat and most users in the world use 4.0. So they love the way it matches like tone and style preferences and it's a great conversationalist. It's good at like figuring out deep conversations with people or like, it's, it's kind of a good sounding board. But, you know, 0.3 has a very different skillset. It can think through problems really hard. You don't really want the model to think for five minutes when you say hi. And so I think the real challenge facing us on post-training and research more broadly is like combining these capabilities. So, you know, training the model to be like just a really delightful chit chat partner,

• 38:21Speaker 01

  but also know when to, you know, reason. And this kind of plays into 4.1 a bit. I mentioned that we downweighted some of the chat data and upweighted coding to make coding better. So there are some like zero sum decisions in that sense, where you have to figure out what exactly you're tailoring the model for. So that's the real challenge in GPT-5 is like, how do we strike this right balance?

• 38:46Jacob Effron?

  Yeah, I mean, it's so interesting. I feel like some reason people have been drawn to, you know, different models in the past has been like intensely like personality based. I like the personality or vibes of this model. And I'm struck by, I mean, in some sense, trying to combine it into one model, you get like a median personality. And back to the earlier question of like, I wonder whether, you know, longer term folks will want like, you know, maybe they accomplish this through prompting or like, you know, through kind of like learning about you. And then the models themselves have all these personalities within them and can kind of emerge. Any thoughts on that?

• 39:13Speaker 01

  Yeah, I already think we're going in this way a bit with enhanced memory. So I think like my chat GPT is so different from like my mom's or my husband's. So I think we're going in this direction already. It's just becoming so much more useful, the more it knows about you. But also the more it knows about you, the more it can like adapt to the things you like. So I think that's actually going to be a really powerful lever for personality in the future. But we're also going to make it more steerable. So we want, you know, you can already use custom instructions and tell the model like, hey, I don't like capital letters, or please, you know, never, never ask follow up questions. I don't like that. So I think we're going to lean more into steerable there. I think

• 39:55Speaker 01

  everyone should be able to kind of tweak the personality that they want. But yeah, I'm curious, like what's, what kind of personality are you looking for?

• 40:04Jacob Effron?

  It's kind of like I'm still, I'm still discovering, right? Well, I like kind of like the banter is fun, right? And like a little like, you know, kind of like hanging out with your, your kind of like fun and quirky and like, you know, kind of like almost takes risks sometimes and like the stuff they're saying type friend. I feel like I always, always enjoy that. I guess I'm also curious just to kind of hit on your personal journey at OpenAI. Like, obviously, you've You've been, you know, you've done a ton of different roles within OpenAI. You've also like, you know, the company has had, I mean, probably a million different subchapters of like growth and experience in your time there. Like, maybe just talk a little about your like personal journey there and like also, you know, how is it kind of, what feels similar and different from like, you know, the days, early days you joined to like now like leading this large team here?

• 40:48Speaker 01

  Yeah. So yeah, I've been here for two and a half years and I joined on the API team on the engineering side. Actually, a lot, a lot more of my background is, is engineering. I, I worked at other companies like Coinbase building their like high frequency, low latency trading systems. So a lot more focused on like backend distributed systems. But I did study AI in college and I worked with some professors there on research projects and I actually remember using OpenAI Gym at the time, which was super cool. But yes, I, I was here for like a year and a half working on engineering and then I, I, it kind of seemed like it made sense to focus more on the model side for, for the API specifically.

• 41:31Speaker 01

  So there wasn't really enough of a focus on improving the models for developers and I kept hearing like folks wanted something like structured outputs. And so that was kind of the first foray into doing research here. It's like training the models to do that and building the engineering systems. And then after that, I kind of formed this team and then moved over to research. And actually recently rebranded my team a bit and we, we focus now on, on power users. So it's the power users research team and it's, the reason for this like rebrand is that we don't just focus on the API. Obviously developers are some of our, you know, most discerning power users. These features that, you know, other users don't know about. They know about prompting our models the best. They know the capabilities the best.

• 42:16Speaker 01

  But there's also power users across, you know, ChatGPT. There's some in free, there's plenty in plus and pro.

• 42:23Jacob Effron?

  I'm kind of insulted I haven't been reached out to as a ChatGPT power user. I thought I might have hit the threshold, but I guess, I guess there's probably some people that use a lot more.

• 42:30Speaker 01

  I mean, yeah, we we get a lot of signals from people who are using our models in this way. But also, this is like, the reason it's interesting to focus on power users is because the things that the power users are doing today are going to be the things that the median users are doing a year from now. So we just learn a lot from being on the frontier and figuring out what we can do to make the models better for them.

• 42:51Jacob Effron?

  And I guess like, what's it been like, obviously, you know, over those two years, I feel like the organization has changed a lot, both in size and the scope of things you you work on, like, what kind of feels still the same and what's really different, you know, these days?

• 43:04Speaker 01

  Yeah, I think the pace of shipping is the same. It's actually remarkable, like, how how an organization this large can can move so quickly. I think some things are different is you just definitely can't have context on everything going on at the company anymore. It's like, you know, it used to be more possible to like, have pretty good state on all the cool projects going on and read all of their research updates and and be intimately familiar. But now you you kind of just have to tolerate that you can't know everything cool going on anymore.

• 43:36Jacob Effron?

  Totally. Well, we always like to end our interviews with a quick fire round where we get your take on some overly broad closing questions. And so maybe to to start, would love your take on just one thing that's overhyped and one thing that's underhyped in like, the general AI discourse today.

• 43:50Speaker 01

  So yeah, overhyped, I think benchmarks, like, like I mentioned, a lot of like, the agentic ones are saturated, or people release like, the absolute best numbers they get, but, you know, realistic numbers are different. And then underhyped, I mean, the corollary of that is like, your own evals. And so, using, you know, your real usage data to figure out what's working well, underhyped.

• 44:17Jacob Effron?

  Awesome. What's one thing you've changed your mind on in the AI world in the last year?

• 44:21Speaker 01

  Yeah, this is back to fine tuning, but I actually used to be more of a fine tuning bear, because it's kind of like, You know, it's a few months of arbitrage, but is it really worth the time? But I actually do think RFT is worth the time for for these like specific domains where you need to push the frontier.

• 44:38Jacob Effron?

  Yeah, was there like one particular fine tune that convinced you or like, you know, was it over time, having seen this?

• 44:44Speaker 01

  I think the cool thing now is that, like, you know, our previous post-training stack or like the 4.1 stack is a lot more than just SFT. And so like, we weren't shipping how we trained our models. But with RFT, we, you know, it's it's basically similar algorithm as our reinforcement learning. And so that's why I think it's like, it's a big shift where you can actually kind of get the capabilities that we can elicit ourselves.

• 45:08Jacob Effron?

  Do you think model progress will be more, the same, or less than last year this year?

• 45:12Speaker 01

  I think it'll be about the same. I don't think we're slowing down. I don't think we're in a fast takeoff at the moment, but it's going to continue to be fast and there will be a lot of models.

• 45:24Jacob Effron?

  I realize I can't ask you to pick a favorite, but I'm curious that like, you know, you imagine this like class of of kind of harder to to solve problems, you know, maybe, you know, beyond enterprise apps, like any kind of like consumer products or things that you're like most excited about outside of of OpenAI? Or things that you use in your in your kind of like day to day life?

• 45:44Speaker 01

  Yeah, I use a lot of stuff that that is like AI based. Like recently, I've been using Levels and they are like have a pretty cool AI focus there. I think Whoop has some very cool like health insights as well.

• 45:58Jacob Effron?

  Yeah.

• 45:59Speaker 01

  Yeah, I think taking AI out of just the digital world is super cool.

• 46:04Jacob Effron?

  Well, this has been a fascinating conversation. I want to make sure to leave the last word to you. Where can folks go to learn more about you, For1, anything you want to point our listeners to? The floor is the floor is yours.

• 46:15Speaker 01

  Yeah, totally. Thanks. So, yeah, we put out a blog post for For1 if you want to read more about it. I'm also on Twitter and I love hearing feedback from users, like developers, power users. So if something isn't working well on our models and you have a prompt that can show it, please email me. I'm first name at openai.com and I just I love getting the feedback so we can make models better.

• 46:38Jacob Effron?

  We'll have to get you on again to talk about like the weirdest email you got from us of like, you know, a obscure use case prompt.

• 46:43Speaker 01

  Yeah, I've already gotten some good ones.

• 46:46Jacob Effron?

  Yeah. Well, Michelle, thank you so much. This was a, this was a ton of fun.

• 46:49Speaker 01

  Yeah, thank you so much for having me.