Ep 9: Open AI VP of Product Peter Welinder on OpenAI’s Strategy and AI’s Future

Jacob sits down with OpenAI VP of Product & Partnerships Peter Welinder and guest host Rob Toews (Radical VC Partner) to discuss OpenAI’s strategy, how they think about what they will/won’t build, the future of open source models and when we’ll reach AGI.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:02Jacob Effron?

  This is Unsupervised Learning, Red Points AI Podcast, and I'm Jacob Effron, one of your co-hosts. Today, I'm joined by guest host Rob Caves from AI from Radical Ventures, and we had a fantastic guest today in Peter Welinder, the VP of Product and Partnerships at OpenAI. Peter is responsible for the OpenAI products you know and love, GPT-4, ChatGPT, GitHub Copilot, and we had a fascinating conversation, talked about everything from how OpenAI thinks about what they will and won't build with regard to agents, developer tools. We talked about when Peter thinks we'll reach AGI, and we hit the future of open source models and when they will and won't be good enough. Definitely tune in for this one. Peter, thanks so much for coming on.

• 0:39Speaker 01

  Thank you so much for having me. Super excited to be here.

• 0:41Speaker 02

  I want to start with a big picture question that a lot of folks on the investment side, on the startup side, kind of throughout the AI ecosystem are grappling with and thinking about and talking about these days, which is this question of where value will ultimately accrue the most in the AI ecosystem. And you can, you know, there are a lot of different ways to divvy up a mental model of the AI ecosystem, but I think, you know, people often think in terms of the model layer, or the infrastructure layer, and the application layer, and where the most interesting opportunities arise. I'm curious to hear how you think about that topic and what your mental model is, given where you sit at OpenAI. Obviously, OpenAI is at

• 1:24Speaker 02

  the very front of the field in terms of building world-leading models that are powering a lot of this transformation, but at the same time, OpenAI also obviously makes possible all sorts of end-user applications across different sectors and industries. And you guys have an investing arm that's investing in a lot of different application layer companies. So I'm curious how you think about things when you, when, when you hear the question around where, where will value ultimately accrue in the world of AI?

• 1:52Speaker 01

  Yeah, it's a, it's a, it's a great question. I, I think this, the, this, the simple answer, I think, is that it's, I mean, it's going to be in a lot of those areas where most of it will accrue. I think I suspect that most of it will accrue at the application layer across a lot of different applications. And in fact, like very much the strategy or approach we have at OpenAI is like, you know, our mission is essentially, you know, build it, make sure it's safe, make sure it benefits all humanity. And I think central to that is really enabling as many builders as possible

• 2:37Speaker 01

  to build products on top of this technology. So like, we've made a conscious choice to make sure that we don't have a lot of like value extraction at the model infrastructure layer, like really trying to kind of keep prices really low to enable way more development. Like, and you can actually see this, it's like we made some pretty dramatic price cuts over the years. Like, I think we made around like about a year ago that we cut prices by like 70%. And we released a model that was like 10x cheaper, deeply 3.5 turbo than the original 3.5 model. It was also much faster. A lot, kind of, both the research and the engineering that we do is to kind of, you know, bring down

• 3:19Speaker 01

  prices, because it just broadens the access to these models and the applicability of these models to many more applications. And I think

• 3:35Speaker 01

  that's like a super critical component. And tooling is obviously a super important component. I think we spend less of that time on that. Like, we, there's just so many, the right tools for this technology just haven't been invented yet. Like, some of them are probably in the early stages. And we have no illusion that we will be the ones that kind of come up with those. Like, I would much rather have the developer ecosystem figure out what are the best tools and actually build those. And because like when we release a lot of these models, we put in a lot of effort, obviously, to kind of make sure that they're safe, responsibly deployed. But also in the way to kind of apply these models, like the kind of the What you can apply these models to, like, we don't know that beforehand.

• 4:18Speaker 01

  Like, we're like figuring this out together with the community as we go in some sense. Like, and it's like, it's much better to have like, you know, we have, we have about 400 people at OpenAI. You know, there's going to be like millions of people that can, that are already building on top of this technology, right? Like, so, and they are the ones that are going to come up with all those interesting ways, those interesting applications, those interesting tools that you can, you can kind of use to kind of be even more productive. Obviously, if you have like a factor of, you know, uh, over, uh, over a thousand, 10,000, you know, there's just going to be way more innovation in, in that space. And, and so we really want to enable that.

• 4:56Speaker 01

  I, I do think at the end of the day, what you see, where, where you will see like a lot of the value capture is probably going to be at the application layer where people, like companies are able to kind of build really, uh, kind of great applications with some of the standard things you have in any business. Like, you know, the, the standard kind of competitive modes around network effects, around branding and so on. That's like really where you often have the opportunity to kind of do, do a lot of that value extraction. I don't think it would be very much different here. It's just kind of another technology where a lot of the same, uh, principles, um, apply. And in fact, we would be very happy with that in some ways. It's like, uh, uh, there is this like Bill Gates quote, like, you know, you know, you have a platform where, um,

• 5:39Speaker 01

  where, where the value generated on the platform is much bigger than the value that's actually captured by the platform provider. You know, I, I think that's, that is, that is kind of the aspiration here is that we get to that point where it's, it, there's just a tremendous value being generated on top of, of this technology.

• 5:54Speaker 02

  That makes a ton of sense. There's, there's one quick follow up to that. Could you envision, or do you envision OpenAI starting to build more and more like first party applications for folks or staying more at that platform level?

• 6:07Speaker 01

  We really want to stay at the platform level, at the more general level. Like this is like, obviously, we have ChatGPT, which is a kind of a product that's more focused to consumers. But even that application, it's a very general application. It's sort of like, you know, it is there to really answer more generic questions. It's not meant to be like the best AI teacher in the world, or the best, you know, AI doctor, or lawyer, or whatever. Like, there are companies who are really building on making, taking those applications one step further. And that will involve not only kind of the intelligence part or the reasoning part that we can provide with our models,

• 6:47Speaker 01

  but it also provides like integrations to other products, to like connecting it with proprietary kind of content, like training it specifically with the expertise of people in those domains and so on. And so the way we really look at this is like, we want to provide a platform for companies to kind of build those experiences. And probably also kind of connect it to ChatGPT. Like, the first step in this direction is like plugins in ChatGPT, where you can actually connect ChatGPT to other kind of external services and so on. And I think that's a direction where we go into even more.

• 7:28Speaker 01

  So in some way, we really want to be a platform layer, but like at different, at different, different days, both for the kind of the raw reasoning, but as well as the kind of providing companies with kind of access to actually reach, reach consumers with their products.

• 7:48Jacob Effron?

  Yeah, no, I mean, you mentioned kind of plugins with ChatGPT, and we'd be remiss not to mention. I mean, the pace at which you and the team are shipping product right now just feels insane, right? I mean, plugins, web browsing, mobile, it's, I imagine, you know, it probably only scratches the surface too of all the potential ideas that get thrown around internally. There's so many different directions you can take a product like ChatGPT. How have you kind of been prioritizing what to build? And like, how do you kind of see this product and kind of the plugin ecosystem you've built evolving?

• 8:17Speaker 01

  I think a lot of like the way we approach these things are first.

• 8:23Speaker 01

  Kind of having a, trying to have a fairly high focus on kind of the models themselves. Like this is where a lot of our efforts going like, I think OpenAI took this approach, which is a bit different from other research labs, where we just made a really big bet on language models. And just how general they would be. And you know, this is a bet at many different levels. Everything from like ensuring that most of the people within OpenAI were focused on large language models. Most of the compute, we had most of the GPUs were being spent on training these models. And, and, and that's sort of like kind of a more of a top down decision that we made at OpenAI. That this is like, this is our bet and we're not going to spread across too many other bets.

• 9:08Speaker 01

  We have a few other things in, in kind of parallel that we think at some point we'd kind of tie into the broader picture. But, um, you know, we used to have more bets, like we used to have robotics bets where we kind of, we used to have kind of, um, bets on say, you know, beating world champions in Dota two. These were all to kind of really push technology directions really, really far for us to learn, to see how viable they were. But as we kind of gain more confidence in one direction, we put most of our efforts into that. And I think this, um, this is sort of continued in that the reason that we can kind of move fast with a lot of these things like browsing and our, um,

• 9:48Speaker 01

  and, and plugins and so on. It's like we're building on top of these language models like, and they are incredibly flexible. And at that point, if you have that like kind of platform layer of just the, the really flexible language models, then the only other thing you need is really smart people and like driven people. And we've been very lucky to kind of just have some incredible, both researchers, engineers and product people at OpenAI, who can take like just have these ideas and can take them and, um, and kind of take them from the research state, stage all the way to product where I think a lot of, a, a, a kind of a big difference from also other kind of more research oriented organizations or organizations that have research labs is that I think a lot of the.

• 10:29Speaker 01

  People who are more research focused at OpenAI are also very motivated to actually get their stuff out in the hands of users and learn from that. And and I think that's how we we can see a lot of these these these products like or these these features like browsing and plugins and so on. It's really came through researchers having a vision of like how you can build on top of these language models. And then put them out in the project and then we try to build a product that has the flexibility where it's easy to kind of deploy that, you know. Plugins and it's was kind of a way to kind of unify a bunch of different things we were doing around browsing, around code interpreter and like connecting to external APIs. We just tried to look for these kind of common patterns that will enable the research teams to get their stuff out faster into the hands of users.

• 11:14Jacob Effron?

  Yeah, I mean, you know, one one interesting area I'd be curious for your thoughts is it does feel like, you know, over the last months, you know, autonomous agents have become all the rage, right? And this is like big next step change in like capabilities of large language models. I'm sure you guys have been thinking about that and wondering how you see that, you know, potentially being part of of of the product here.

• 11:34Speaker 01

  I mean, in some way, we already have it with plugins. It's just like a little mini agent that at this point kind of you give it a task and it just connects to a number of APIs in a certain sequence to kind of carry out some tasks. If you want to kind of, you know, create some recipe on a topic and then you want to kind of put that on a shopping list and Instacart is on it. You can use our plugins to kind of do do those things. And so it's it's fairly straightforward to kind of play that out towards like, well, you wanted to kind of do more complicated tasks. Only check in with you when it gets the the the kind of model gets a bit uncertain on what it's supposed to do. And then suddenly you have a much more agent like kind of behavior.

• 12:13Speaker 01

  I think, but this is also like where where I think that it is really great to have this. The API aspect of our product, which enables developers to really push this much further and like explore much faster than we can. So I I I just think it's like incredibly inspiring to look at these things like, you know, R G B T, A B A G I, these these kind of these kind of projects and see how like what people are doing with them. I would say like my my current sense is like just with plugins, it's sort of still early, like these things aren't working perfectly. They are like they're a bit rough around the edges, but. You know, you can see the potential in them like they just need to be a little bit more robust, a little bit smarter.

• 12:57Speaker 01

  And we all know that this is like going to be happening over the next kind of next kind of year or two that like we're gonna have another generational models that are gonna be way smarter. And and and then a lot of these things will start to work. So I think um what we're really optimizing for here is like, you know, we we try to really think about it. We're here, we're like, we're an AGI company. We like, we try to look at a lot of what we're doing from the lens of like, what will this product look like when we have AGI? You know, what does it mean? Like, it's when you have an AI, you can actually ask it just as you would like, ask a human to do something. The human would go off and do things and come back and check with you. Like, it's like you hire a new employee to do like, you know, to say,

• 13:39Speaker 01

  you know, you hire an employee to code your iOS app and you give them some vague directions. They come back with questions and they go off coding, then they show you something, you give feedback on that. It's sort of like, you know, if we're really AGI, that's sort of probably what sort of what we're what we're gonna have in in a couple of years, right? And and so um I I think you should kind of expect the product to evolve in that direction.

• 14:03Speaker 02

  That makes a ton of sense. Yeah, I I share your excitement around the concept of agents. It does feel like a natural step on the path toward more generalized intelligence. One topic I'm very curious, Peter, to to hear your thoughts on is around open source AI and and open models. There's obviously been a massive explosion of activity in this space just over the past few months, in particular after Meta's release of Llama and all the kind of progeny of Llama that have that have emerged. I'm curious here how you think about how open source, open source models fits into the broader AI and LLM ecosystem,

• 14:48Speaker 02

  in particular with the perspective that at least at this point, OpenAI's most advanced models are closed source and and and OpenAI's business models based on that. I'm curious how you envision the kind of closed versus open AI ecosystem playing out.

• 15:03Speaker 01

  Yeah, this is this is a really interesting topic and and maybe like, maybe this this is where I think I might have like a more kind of unpopular opinion, honestly. Kind of in the sense that I I I think that there is a lot of

• 15:20Speaker 01

  There's this kind of growing belief that, you know, at some point like these these open source models would essentially catch up with with the models that OpenAI and others have. And and so that's that's kind of where this would go. I think I think this is true in the very long run. You know, because I think we should expect just like the, you know, everybody. The the sort of the the compute we have in our phones and our devices to just keep on getting better. I mean, we have Moore's law, but there's there's other things here that will happen that will probably accelerate that over time. But I think that's going to play out in a slightly longer time horizon. But but in in the kind of, I I would suspect that at least.

• 16:05Speaker 01

  I I think there's always going to be a category of models and or like maybe more accurately like AI systems. That will be way, way better than the open source kind of versions of those same models. Like, we've sort of seen that play out in a lot of other areas like, you know. You know, desktop Linux has been a thing for like, you know, thirty years, and it's never really kind of caught up to like, you know, Mac OS X and and iOS and and Windows and so on, like, because there's six, there's never enough kind of investment into a lot of the details and and and and so on to kind of get it all the way there. And so there's going to be something similar here. That said. I it's and and and I think the other trend here is that

• 16:49Speaker 01

  the amount of capital and just like amount of engineering that goes into both training these models, but that also essentially doing inference in these these models, like running them at scale and so on. It's it's very non trivial and it's just hard to replicate at at a at an open source level.

• 17:11Speaker 01

  And and so the incentives are definitely going to be like the companies that are investing a lot of this, they're not going to want to kind of open source it. I think that's just that that doesn't make sense from an investment standpoint. It also, there's also, it's also questionable from um from a kind of a safety standpoint. Um, I think what is important is that, you know, that there's broad access to these models, and that's very much what we're focusing on at OpenAI. But like, I'm not sure that um, I'm more skeptical that this would happen through open source. That said, that doesn't mean that I don't want to kind of be sort of all poo poo on open source. I'm actually very excited about the open source development. Like, I think there's a ton of

• 17:51Speaker 01

  energy in that space, and there's like both in terms of like seeing just like how like, I think having open source models is extremely useful for pushing the research forward, for like having people just try out new approaches, like, you know, you know, different kinds of ways of training these models for applying them in different ways. There's also, there are also going to be like product areas where it makes sense to have kind of a an open source model that can be, or like rather kind of, I think it's more like having kind of maybe the way of putting it kind of smaller models, like if you want to put them on a device or on the edge or kind of in different kind of on-prem deployments and so on, like you

• 18:34Speaker 01

  kind of have to use kind of um, kind of more an open source kind of approach. And I think it has a role there, but if for the applications where you really want the best stuff, like the like where you don't need to want to worry about the reliability of the model, like how smart it's gonna be, I just want the smartest thing that's actually gonna work the best, then I I think, you know, the the most rational thing is going to be to pick one of the kind of the more proprietary proprietary models, and those we just keep on getting better and and probably be like, you know, a couple of years ahead of of open source for the foreseeable future.

• 19:11Speaker 02

  Yeah, yeah, I think it's a very fair perspective that there will be this persistent performance gap potentially between the best cutting-edge closed-source models from places like OpenAI as opposed to what you can get from open source, and that the the frontier is constantly moving, and it's likely to be the kind of the proprietary research AI research organizations that are advancing that frontier.

• 19:37Speaker 01

  Actually, I have a comment on that. Actually, one more comment on that, which I think is important, is that I think one interesting thing is that there is an open question about like, how smart does the model need to be for certain tasks? Like, if you're doing something like just summarizing something, then, you know, you may not need the smartest model. But then, you know, the problem that we often see kind of customers facing who are working with these open source models is that products tend to go towards generality over time, and then you just want the models to be generally smart. And then, so if you have a very specialized thing, then it makes sense to kind of have an open source model. You fine tune it specifically for that that purpose. But then, as you, but then that just means that you will have a very narrow product.

• 20:20Speaker 01

  And if you try to go slightly outside of the domain of that, you're gonna hit an issue. And that's really where like the the generality of these models come in. And so, for example, even for like a customer service bot, like if you wanted to kind of just be able to kind of deal with all the edge cases and so on, right? You'd need the smartest model. And but there is a question about like how

• 20:41Speaker 01

  every time you have a, every time you can make a smarter model, it's going to be able to cover a few more use cases. And so I, I, I do think there's going to be applications, continued to be applications, where probably an open source model is going to be sufficient for for that application. So there's, there is that, that dimension to it as well.

• 21:01Speaker 02

  Yeah, yeah. This, this question relates to that point and to how much generality you actually need for a given commercial product. But I'm opening it, obviously, is many things as an organization. You're, you're a business, you're also a research organization for CRI AGI. I'm curious, even granted that open source may not be able to keep up with closed source in terms of performance, like, do you, do you think that this movement of open source models could present or does represent a meaningful competitive risk to open AI from a business perspective? And from like an addressable market perspective?

• 21:39Speaker 01

  I don't think so. And the reason is that I think that most of the value is ultimately going to be in the smartest models. Like it's sort of like, you know, like, give you an example of like,

• 21:52Speaker 01

  are you, are you, when you hire people, you always try to hire the best people, right? I think it's going to be the same thing as you're building these sort of products. You'll want the smartest models because those are the ones which are going to give you the best products. And, and, and we will, so we will see the same thing play out here. It's just like, if you're picking like a model that's a bit dumber, you're going to be at a competitive disadvantage with the companies that are picking a smarter model. And, and so our bet has sort of been that that's where like most, most of the value over time will just accrue in, is in the smartest. That's where you can tackle the most economically valuable problems and so on.

• 22:35Speaker 01

  Um, and, and, and I think like, what we will essentially see is like first, first we will see kind of this working for more and more kind of, I guess, professions, like where you can start building, say, you know, co-pilots for different professions like lawyers and medical providers and, and so on.

• 22:54Speaker 01

  And over time, it would sort of probably go even beyond that. It's sort of like, well, if you then want kind of science, AI scientists that come up with new drugs or come up with, um, you know, solutions to climate change is on that's a, a, a whole new level of like economical output you can get. And, and we, we would just kind of see that kind of moving higher and higher over time. So that is going to be, continue to be, that continues to be the way we think about it. Um, and, and so our effort is always going to be put mostly, mostly on that. There's, there's some auxiliary stuff that you, that are useful with smaller models. Like, give you an example, using models for, smaller models for embeddings for it, for retrieval, for grounding these models can be,

• 23:39Speaker 01

  it's useful to have a, to have a smaller model for that. And those can actually be open source and there's some advantage with having those, those open source. And it's, you know, it is because, um, In that case, like latency and so on is much more important often. And so you like having a smaller kind of model is important for that. Also, it's kind of making sure that you have a little bit more control over like owning, like recomputing embeddings can be expensive. So probably you want an open source model. So we, you know, I can see us kind of, and we already do this. Like we, we, we look at a lot of what are sort of models that can kind of enable more

• 24:24Speaker 01

  kind of smarter model use in some sense. And, you know, our Whisper models is an example of that, where we open source the model, because it actually allows people to now do very accurate transcription of audio. And now they can feed that into large language models and get like, build many more applications that it couldn't do before, because like the, the, the, the speech recognition stuff wasn't accurate enough. And so that's, that's, that's a really, where we, we, we just want to enable that as broadly as possible. We don't think we will make any meaningful amount of money on, on speech kind of recognition. But we want to enable more use cases. And, and I think that's, that's often how we pick where we make our open source investments.

• 25:08Jacob Effron?

  Yeah, it's an interesting point in like, you know, obviously companies will try to optimize, you know, costs and latency. But, you know, I feel totally hear you on the idea that even if there's orchestration to route some tasks to distilled models and some tasks to the cutting edge ones, the real economic value accrues to, you know, the state of the art models, right? I mean, those are the most sophisticated tasks that will be routed their ways. So I think it makes a ton of sense. And I mean, you mentioned some of these like enterprise, you know, use cases. And obviously, I think one of the most exciting things about being in the space is it just feels like every day there's, you know, new announcements and new ways companies are working with these models. I feel like you're on the ground seeing this and working with these folks. And I'd just be curious, like, obviously, I feel like we've gotten some level of enterprise adoption. I'm sure there's, you know, still 99.9% of it to go.

• 25:54Jacob Effron?

  So as you think about kind of some of the gaps today that you're seeing that are, that are preventing folks from, from really going as full steam as you'd like with some of these models, like what are you seeing on the ground?

• 26:04Speaker 01

  I think probably the biggest gap right now is the problem of hallucinations. These models like hallucinating facts. So you can't entirely trust them. And we hear this over and over again, and it, you know, and the truth around this is like, it is like, it's so one of these things where we're enough on the cutting edge in this, this, um, this field generally, where this is an open research problem. How do you solve this problem of models not hallucinating? We, we, we put in a ton of effort into our models to make them more and more robust to this. And I think the way that more and more companies are getting, um, uh, getting around this is basically grounding

• 26:44Speaker 01

  the models in, um, kind of external data. So for example, if you're asking a question and, um, you have some internal company documentation where you wanna, which, which you can use to answer that question. You can use this kind of embeddings to, um, approaches to where you embed the question, you embed all your documents, you have a vector database, vector search database where you find the relevant documents, you give those to the language models, and then the language model essentially picks the relevant information from those documents to summarize an answer to the question. And then like you, you, you can get around these, these problems of the model hallucinating. You actually tell them all, like, if you can't find the answer, just say, I don't know.

• 27:25Speaker 01

  You know, so it's, it's kind of funny. You just, you just kind of instructed to do that and it works really, really well. And, um, but I, I, I think this has been kind of probably as, as I've been looking at applications, that the hallucination issue has probably been the number one issue.

• 27:41Jacob Effron?

  Yeah, it's interesting. I mean, back to the earlier conversation we're having, having about what you guys build and what you don't and kind of some of these developer tools emerging. I mean, obviously there's a ton of developer tools that have emerged around getting around this. There's hacky ways I've seen different enterprises get around this. Like, is this something that you kind of wait for, you know, the larger ecosystem to build tooling around, or is it something that like, OpenAI just eventually solves themselves? Like, how do you think about that?

• 28:04Speaker 01

  I, I think it would be like, it's, it's going to be something that. Where the way we can make decisions on that is like we're going to be listening to our developers. It's like, where are your obstacles? Like, what is like preventing you from being successful? And what can we do to enable it to kind of move faster? That's like one aspect. So then, you know, if there's certain pieces of tooling that are like all of our developers need, then, you know, we will probably build some of that tooling, at least to some extent. So that at least it's easy to kind of get off the ground and kind of get going, developing quickly. I think that, but I think the other thing that we're thinking about is sort of like,

• 28:42Speaker 01

  what, what are the areas where we are, you know, we are really kind of the best in the world at doing things? And I think it's like training these models, making sure inference is really, really good. And so that's, that's where we've got to put most of our efforts because that's where we can provide the most value in substance. So we would always prioritize that, that higher. It's just like, you know, if, if I look at all the things that people are asking us for, probably like lower prices, higher reliability, lower latency, you know, we, we're always going to be prioritizing that, that, that first. And then, because there's plenty of other companies that are building really good tooling on top of that.

• 29:24Speaker 01

  But unless you have those fundamentals, that, that key kind of intelligence product, like that tooling doesn't matter. Yeah.

• 29:34Speaker 02

  Peter, another topic I'm very curious to hear your, your thoughts on. And there's been more and more discourse around this topic in recent weeks and months. There's, obviously AI is a technology with tremendous upside potential. We all believe deeply that has the capability to improve humanity and civilization. But it's not without its risks and downsides as well. And there are a number of different downsides and concerns around AI that, that people have been spending time thinking about, from misinformation and deep fakes to bias in AI. Job displacement is obviously a big, a big topic. And then even the, you know, the, the bigger picture, you know, could a runaway powerful AI pose an existential risk to humanity itself?

• 30:19Speaker 02

  I'm curious, from your perspective, which risk or concern around AI do you think is the most important one? And the one that we like, we collectively should be taking the most seriously and thinking the most about?

• 30:34Speaker 01

  Yeah, I would say like my take on this is like, if I look at like the first few risks that you listed, I think those are all pretty surmountable, you know. I think the, and, and, and a lot of them aren't, a number of them are to do with the models, but they're also on distribution channels, like, you know, misinformation and so on. Like, you know, you can't really do that. Misinformation becomes a problem when you do it at scale. And, and, and so you need to kind of go via some of the existing kind of platforms and so on to do that. Whether that's like, you know, social media or, or, or kind of email or something like that. So there's already kind of, and there's already a lot of like infrastructure in place to protect against that.

• 31:18Speaker 01

  I think similarly with, with bias, a lot of our, our take of this is that it's impossible to not have bias in your models. And what we want to do is like really provide the, like the, the tools so that you as, as a product developer can, or as a user can kind of instruct the models to have the biases that, that you want. Obviously with, within some bounds, but like, but we want those to be very broad, you know, like they shouldn't have a particular political orientations on like, it should really be up to the users to, to pick the behavior of the model that, that they want. I think, I think the risk that we like paying too little attention to is probably the one around like us.

• 32:04Speaker 01

  Right now we're like, we're going towards AGI, but I think the next step is going to be super intelligence. Like these models are going to get really, really smart and you know, beyond the smartness, smartness of humans. We don't know what timeline, but it's like, that's a pretty big deal, you know. So it just feels like, and there's just a surprisingly little amount of research being done on like, how do we ensure that that outcome is good? Like we, we, we have teams at OpenAI who are doing it, but it's sort of, there's no like super intelligence, safety department at universities and stuff like that, you know, it's sort of, and if this is, this, I do think that this has, this can both be. This can sort of be existential for humanity.

• 32:48Speaker 01

  And so I think it's really important that we think about it, both in terms of the techniques, the technical aspects of how do we ensure that these models are aligned, that they are, that we as humans are able to control them. But also that we, there are also kind of other areas around regulation to ensure that we as, that the government understands and at a worldwide scale understands when we're getting into, when we're getting close to this, like which companies are doing it, like how much computer they're using to train these, the models that would be able to, to kind of go beyond AGI in some sense. And I think it's one of those things where it's sort of like,

• 33:30Speaker 01

  we've talked about it a lot at OpenAI and, and you know, it's, it's, it hasn't been taken very, very seriously. And I think superintelligence will still be one of those things that isn't taken very, very seriously. But I think it will be a very serious topic within the next decade. So, and I, I think the sooner we can have much more kind of serious debates and so on around that, the better.

• 33:54Speaker 02

  Yep, yep. Two quick follow-ups for me on, on that. It's a fascinating topic. The first, I know that you don't know and nobody knows, but everyone still loves a good prediction. So I'm curious, like what, what's your best guess as to the year that we are most likely to reach, to reach superintelligence? And again, I know this is just speculation. I know you're like, you don't really know for sure, but I'm curious what you would say.

• 34:18Speaker 01

  Yeah, it's a good question. Like I,

• 34:25Speaker 01

  I think the,

• 34:27Speaker 01

  I had sort of, you know, we, we sort of have a definition at OpenAI around AGI, which is like autonomous systems, AI systems that can do economically valuable work at the level of humans. And I think we have a shot at doing that, you know, between before 2030. I, you know, the exact probability, I think there's definitely some probability that there's something unknown that we don't know about that. That would prevent us from getting, getting there. There can also be like external kind of events in the world. That happens, you know, maybe whether it gets huge like GPU ships, a shortage or, or something like that. That would prevent us from going there, but things seem to be moving in a, in a kind of, um.

• 35:09Speaker 01

  Almost like automatic fashion at this point. There's just so many much innovation in the field. That this is, it feels like we're really on a trajectory. Um, you know, I, I would say like, you know, this is speculative, right? I, I don't know. I definitely don't know for sure. But it does feel like something is very different. I, I, I've been in this field for like 15 years. This is not what it felt like, you know, 15 years ago. If I was like, you know, I have no idea like. How we could even get there. Now it feels like it's sort of on the path of kind of just happening. And, you know, I'm not the only one saying this, like privately, as I talk to many people who have been in the field for much longer than me, they, they say the same thing. It's like something is very different now. Than how it felt like 15, 20 years ago.

• 35:47Speaker 01

  And so that makes me feel like there, I think we have, I wouldn't be surprised if we have kind of something that resembles EDI. By the end of this, this decade. Uh, and then like super intelligence is sort of interesting in how you define that. It's like, well. Can you do it, run it? Can you run this AI much faster than humans can think in a much more paralyzed fashion and so on? Like different. Dimensions, you can think about super intelligence. You can do many more experiments than a human could do and so on. I mean, some of those aspects will probably be possible. There's also some of the cars that we get there and it's like. We have AGI, but it's like too expensive to run and we need to wait another like, you know, five to 10 years before we can actually run it economically. There's, that's a potential outcome. Um, but, but, but I think like 2030 is probably around the time where I imagine that we will start.

• 36:33Speaker 01

  Really seeing kind of the early signs of super intelligence. It's, that's my, my speculative bet and would probably, you know. There's some chance I would need to eat that up in 10 years.

• 36:44Speaker 02

  Yeah, no, I mean, you're totally right. It's incredible how just what it feels like in this space today, how quickly things are moving. It's, you know, it's impossible to keep up with. It's exhilarating and also overwhelming. One more question for me on this topic. Jeff Hinton recently made waves and got a lot of headlines when he publicly said that he's worried about the potential existential negative consequences of super powerful AI. And so to your point, and others, Yashua Benjio and others have voiced similar concerns. To your point around the unpredictability of what AGI and then especially superintelligence will look like.

• 37:24Speaker 02

  How worried are you about like an extreme downside scenario where like the continued survival of humanity is at risk? And if that is a risk that you take, that you see as credible at all, like what role do you think OpenAI as the leader in the field should or is playing there?

• 37:44Speaker 01

  Yeah, I definitely, I think I'm probably a little bit more on the optimistic side. I think that we will be able to kind of figure this out as humanity together because, you know, a little bit like we haven't had a nuclear war, you know, because we're like, we have some level of self-preservation. I think there would be something similar here where I think as we get closer to AGI and superintelligence, we will tread much more carefully. And in fact, but I think like the way to do this is you kind of need to build up the right organizational processes, the right kind of frameworks to think about this and so on. This is very much has been the strategy from OpenAI for the past couple of years is that

• 38:28Speaker 01

  we think it's really important to get these models out when the stakes are low. So we learn from all the kind of the risks that are kind of arising when those takes are low, like things like misinformation, things like biases and so on. Like, you know, how do we, how do we tackle these? Because if we can't tackle those, then I feel like not very confident that when we get to like superintelligence, we will know what to do. And again, it's not only about the technical approaches. It's even like, how do we have like the right kind of? Kind of organizational structures for making decisions around when to deploy things into the world and what are the safeguards and so on. This is not stuff that you can just kind of figure out. Once you get there, if you've trained them, now we have super intelligence. Now let's figure out how to make it safe.

• 39:12Speaker 01

  That seems like really, really off to me. As our approach has really been like, let's gradually figure this out. And if we feel like the things are kind of getting a bit too unsafe, then we will hold back on releasing it. We did hold back on releasing GPT-4 for almost half a year just because we wanted to kind of get much more clarity in the potential downsides of this new model. And I think this is how I would, I think we, I think if we set that example at OpenAI, I think others would also, you know, it's easier to set that examples if you're sort of one of the leaders in the field.

• 39:54Speaker 01

  Because then that adds some accountability to others as well. And we hope through some of our actions that we can kind of add that accountability elsewhere. And I think at the end of the day, then the chances of this being a really positive outcome, which I think it can have, right? Let's also talk about the upside, right? If we actually get super intelligence where you can solve all the problems we're facing today, like climate change, like cancer, you know, things around aging, all the things that happen as you age, you know, it's like kind of more abundance of and higher standard of living for everyone. Like this is the upside of this technology, right? And so I think it's like, it is important that we get it out. We just need to do it carefully.

• 40:37Jacob Effron?

  Yeah, I mean, one thing I was struck by that you said earlier is obviously there, you know, you guys are doing a lot of work on this at OpenAI, but there isn't, you know, you mentioned these kind of lack of like super intelligence safety departments at universities, right? And folks thinking about it. And, you know, I do wonder like if you were, you know, if you were in a different life, running, you know, a super intelligence safety department at a university with a large budget, like what would you kind of focus on?

• 41:00Speaker 01

  Yeah, that's a great question. I think. I think there's a few different areas. Like, I think there's a lot of work that can be done on like, understanding what's going on in these models. They're still fairly black box. So interpretability. There is a number of like, uh, people who are doing this, like, um, and you know, there are people like, um, Chris Ola, who used to be at Dobrers, now at Anthropic has done incredible work on that. There's a number of others who, like, where you kind of go deep into understanding kind of why are certain activations within these deep neural networks happening? And like, how can you, uh, use those to understand what's going on into the models? In the models? I think that's one area where there's probably a lot of work we can do. Like, and it's probably an area where we can move much faster. This is what's happening in neuroscience.

• 41:44Speaker 01

  That's happening kind of more computational neuroscience and so forth for a long time. But like, you can move much faster if you do this on these, these models. And in fact, like, we, we are collaborating with researchers who, who want to do this on, on our models.

• 42:00Speaker 01

  I think that's one big area. I think there's also others around, like, I think just kind of defining kind of what, what is alignment? Like, how do you set up, like, what, how do you specify goals? How do you specify the guardrails? And, and so on, like, kind of being much more, getting much deeper in the specification is another area, which I think is going to be quite important. I think has so far been under invested. It's like, oftentimes you hear very vague statements around alignment and, and so on. We need to get way, way more crisper on this. Uh, and this is probably like, you know, collaboration between technical people from social sciences, from philosophers and so on, really getting much crisper on this stuff. And, and then there are the technical approaches of like, you know,

• 42:45Speaker 01

  there's lots of cool ideas here and I don't know which ones will prevail at the end of the day. But it's like, everything from like shaping the reward functions. We used to train these, these models with reinforcement learning to having kind of, you know, one model oversee.

• 43:00Speaker 01

  The the the the actions of another model and being able to report, you know, what that model is doing is on. There's lots of different approaches here and I I I I wouldn't pretend that I'm by any means the best, the the the expert on like the right investments here. But I I I I know that these are all areas where where we are investing and others are investing in. And I think as as as a kind of society, we should probably invest much more resources in having many more smarter people be incentivized to think about these problems and think about solutions.

• 43:32Jacob Effron?

  So I know that all makes a lot of sense. Um, you know, switching gears entirely, one thing I I was really curious to ask you is obviously you kind of lead product for ChatGPT. I think, you know, it's amazing just to see the different ways people are using it, you know, I feel like where my mother uses it, you know, everyone, everyone's using it in their own way, right? Uh, one thing I was really curious about is how do you guys use it like internally at OpenAI?

• 43:52Speaker 01

  Oh, I think I think one really cool thing about chat GPT is that it has such a broad set of use cases. We see this in our users. It's not like a use users come in with one use case in mind. They usually have a whole like, you know, a dozen or more uh kind of pieces in their workflows where they go to chat GPT, whether it's like answering questions, summarizing things or um or or doing things like helping with writing and so on. I think internally, definitely we see a lot for for coding, you know, people use it to kind of debug i- issues, put pacing it like stack traces and stuff like that. Uh, I personally use it a lot for um

• 44:31Speaker 01

  for actually for helping me write, you know, both in terms of like being a bit of a helping with writer's block, but also kind of improving my writing, like having like, you know, I would ask you to be like, hey, provide a few uh, you know, suggestions of how I I can improve this or like I, here's an email. I don't really know how to write it, but here's approximately the point I want to get across. Give me a first draft, you know, I think those are the things where I in the past I would I would just kind of get stuck and now I can just ask the model.

• 44:59Jacob Effron?

  I love that.

• 45:01Speaker 02

  One last big picture question for me, Peter. I'm always so curious to hear how a lot of the sharpest minds in AI respond to this question. I'm curious to hear your perspective. What's one thing that you believe strongly about AI and its future that you think is really important? That you think most people would disagree with you on?

• 45:21Speaker 01

  Well, I still think the thing I would go back to is like, I do think that we're now at a stage where we should be thinking more about the implications of superintelligence, to be honest. And I don't, I don't think this is like something that is just kind of generally accepted that we're going towards that. It's sort of like people are now waking up to the fact that these models can do a lot of the things that us humans can do. But it's not quite there and feels like, okay, it can do all these things, but probably won't be able to do X, Y, Z. And every time we've had those X, Y, Z things that, well, the models won't be able to do that. Like within a short amount of time, the models are able to do that. And, and, and so I think taking kind of superintelligence much more seriously.

• 46:02Speaker 01

  I do think that's like, you know, I, I, I think this is one of those things where it's definitely, it's not, it's not something we've, that, that everybody has a line on that this is something we should take seriously quite yet. People have very different timelines and so on. And I think we should, we should be acting on a potentially much shorter timeline and do that.

• 46:25Speaker 01

  Basically, what if the timeline is much shorter and, and have a plan for that?

• 46:31Jacob Effron?

  I mean, it feels like, you know, in zooming out in the ecosystem today, it really feels like it's, you know, OpenAI's world, right? I mean, you obviously at the forefront of everything. It's every month, there's some new product release that everyone is, is losing their mind over. You know, it's not hard to paint the path where you guys continue kind of leading the field and, and setting the direction of where things are going. I guess if you were to zoom out in five years from now, you know, OpenAI wasn't in that incredible leadership position. Like, what do you think would be the most likely reason why?

• 46:58Speaker 01

  I think the thing that I'm most paranoid about is like I one of my primary focuses is making these models much, much better. And that's like where we put a lot of our, our, our efforts, both in terms of better in terms of capabilities, but in terms of safety. It's on the thing I'm always worried about is that we kind of lose touch with our users. Like, I really think like, we need to continue kind of just having a really good relationships with the developers or building on top of our models. And I, I, I always feel like worried about there's this tension that we feel with our developers. It's like they, they develop something that's like really good. And then we release a model and it kind of replaces some of the stuff that, that developers have been building on. It's like, it's sort of like, it's, it's not intentional. We want to do

• 47:42Speaker 01

  that. But it's just like the models just get generally better at stuff. And I think that always worries me. Like, I, I, I want us to kind of like really start, uh, uh, really be in tandem with our developers to, to create like a great experience for them. And I think, um, I, this has been particularly on my, my mind now or the past couple of months, because we've seen a lot, a ton of growth. But that also means that we're now able, less able to attend to all of the developers. Like, I, I remember when we, when we shipped the first version of this API, I knew all our customers by, uh, you know, by name, but first on a first name basis and, you know, worked with them all these like channels. And now we have like millions of customers. And it's like, it's just like, how do you scale just a great, great customer experience? I think that's

• 48:26Speaker 01

  going to be really important for, for this to actually be something that people want to embrace and build on over the, the coming, coming years. So I think if we don't do that, I, that, that, that's sort of risking the whole mission in some sense. I'm very worried about that.

• 48:41Jacob Effron?

  Well, Peter, this has been a fascinating conversation. I feel like we could, uh, you know, keep talking to you for hours, but then we'd be, we'd be like risking lengthening that timeline towards superintelligence. So, uh, you know, we won't do that for the world. Um, but you know, we usually ask people just, you know, where people can go to learn more about their work. I will say that I think every listener of our podcast is probably quite familiar with your work. So I'll just leave it to you. Anything you want to plug or point people to, or, or any, any kind of follow ups from our conversation you'd like to leave our listeners with. I'll give you the last word here.

• 49:08Speaker 01

  Well, maybe like one thing I would say is that I think definitely the audience we have now is huge with chat GPT and so on. But I think a lot of people who are using GPT probably don't know that we are, we put quite a lot of effort into our blog, where we try to really, you know, demystify a lot of the technology that we're building and the way we think about things. So for people who want to learn more, I would really encourage people to go and check that out. Where we, we tried to talk about this, how do we think about things as we go closer to superintelligence? How do we think about safety? How do we think about, how do these models work? What are the technical approaches underpinning them and so on? It's a great place where, where we, we try to make these, these, these blog posts kind of, kind of approachable

• 49:51Speaker 01

  from a more, more general kind of technical audience. So I encourage people to go and check that out.