Open AI leadership shuffle, new diffusion models, and starting the cult of Q*

OpenAI’s leadership has taken us all on a rollercoaster so it’s great timing for another host-only episode. This week Sarah and Elad get into what has been going on at OpenAI and what the turbulent leadership changes tell us about the importance of good intent and good incentives when building these influential companies. They also talk about innovative products coming out of Pika Labs, why people are moving away from diffusion models to LLMs, and how, in AI investing, the ASP is the opportunity. Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil Show Notes: (0:00) Recapping the OpenAI saga (9:56) AI video products (16:14) Moving from Diffusion Models to LLMs (19:47) The beneficial margins of AI investing

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:05Sarah Guo?

  Hi, NoPriors listeners. Time for a host-only episode. This week, Elad and I are going to talk about what's going on at OpenAI, of course, video, Q-Star, what might be next in research, and some predictions. Okay, Elad, we have to start with the saga from this past week. What is your take on the outcome and the second-order effects?

• 0:28Speaker 01

  From a second-order effect perspective, this seems like overall positive news for everybody involved. So it looks like on the OpenAI side, they're back to being in a really positive, stable situation. I think they still have the leading model in GPT-4. They've reworked the board, which seems like a positive thing. So imagine if this had happened two years from now, three years from now, et cetera. So it seems like it would increase the stability of the company and governance and a few things like that, or the nonprofit and the company and governance. So as an external viewer, it seemed like a painful thing to go through, but the flip side of it is it seems like they're moving forward and moving ahead in a positive way. And then in parallel, I think it may have ramifications to other areas that we can talk about if useful. What are the second-order aspects of this?

• 1:13Speaker 01

  But it'd be great to hear what you think.

• 1:15Sarah Guo?

  Yeah, I think the first obvious lesson is that governance matters, right? And this isn't an area where I think most companies are that experimental, but I think a lot of entrepreneurs are likely to think twice about placing their destiny in the hands of groups with explicitly mixed incentives now. I'd say generally, nonprofit governance is not every organization, but as a class, known to be kind of abysmal, right? Because performance is hard to measure objectively, and so it often ends up more about politics and specific relationships and status games than outcomes. The clarity around how much any board matters was kind of a wake-up call for people.

• 1:58Sarah Guo?

  The second lesson that a lot of people will take away, or I think should, from this whole saga is that money matters, right? The factors of production are labor and capital, and compute is the AI-specific form of capital. Microsoft holds the compute here and that clearly matters. This is amazingly well managed and supported by Si and Kevin. And then the class of like really special labor here, the team rebelled and the board obviously underestimated the level of support that Sam and Greg had from them. And then I think one thing that is often unsaid, because it's a little bit less idealistic, is that a lot of OpenAI people were very upset this last week about not just the destruction

• 2:42Sarah Guo?

  of the mission, which I think was absolutely like genuine, but also destruction of the value they built and been promised a piece of with the $86 billion tender offer. It's just a reminder that labor and capital are like leveraged, they're stakeholders. And there's no, there's no free lunch or control without skin in the game. And I think there likely shouldn't be from the view of many of the people involved in this.

• 3:09Speaker 01

  Yeah, I think you're raising an important meta point, which is basically what are the incentives that different organizations have in place? And ignoring OpenAI, I think there's a lot of boards which added board members for reasons that were politically motivated or motivated by different regulations getting passed that force certain board changes, et cetera. And you also see that in executive teams. And I think it's really important for people to go back and rethink, okay, who should be on my board and why? What are they representing relative to the board? What expertise do they provide? What insights are they bringing? Or what strategic views are they bringing? Same with your executive team. And also, what are their incentives? And you know, there's this, this view that's kind of been moving around Silicon Valley in terms of the professional managerial class, right?

• 3:50Speaker 01

  People who have alliance not to the organizations they work at, but external incentives. And those external incentives could be speaking at TED or going to Davos or getting kudos or an award from a specific organization versus doing what's actually their duty as a representative of the very shareholders of a company in the context of a corporation. And so there are these fiduciary duties that may be being breached by other incentives for different actors who've been added over the last, you know, five, 10 years to boards, to executive teams, et cetera. And I think it's really worth rethinking, like, who do I want on board and why? And it also comes back to some of the companies that have been resetting things relative to politics.

• 4:34Speaker 01

  I think Shopify did a great job, for example, of saying, we're a performance-based culture. We're focused on, you know, a very specific mission. We don't want that mission to creep. We're about, we're not a family, you know? Like, if your uncle shows up drunk and does something bad, you forgive him. If a board member shows up and does that, then, you know, you don't want them on your board, right? They're being irresponsible. There's also that broader context of like, how do you want to think about alignment, incentives, culture, motivations? And, you know, is this a good moment in time to sort of pause and rethink some of those things relative to your own company?

• 5:09Sarah Guo?

  Yeah. One friend at OpenAI who, I guess, publicly declared that this reignited their belief in clear incentives and some, and good intent and capitalist structures. That has actually seemed somewhat radical in

• 5:27Sarah Guo?

  in many Silicon Valley companies over the last few years. But I think that is going to get rethought when you see what happens when there are unclear or misaligned incentives.

• 5:35Speaker 01

  Yeah, there's two actually really good quotes to that. There's one, which is something which I'm going to get wrong, which is something along the lines of like, capitalism is the best way to take care of people that you don't know. You know, it's the means of actually growing the pie in many cases and providing for others through those sort of incentive of markets. But the other one is the Charlie Munger-ism, and unfortunately Charlie Munger passed away earlier today. And obviously, he was sort of a giant of industry and he had this great saying, which was, anytime I think I understand the importance of incentives, I realize that I'm underestimating the importance of incentives.

• 6:04Sarah Guo?

  If we just think about what the other second order, like more commercial effects are, I do think that there is an interest in owning models more, in open source models and in at least understanding like reliance on a single vendor.

• 6:24Sarah Guo?

  What do you think here?

• 6:25Speaker 01

  Yeah, I think there's a couple of people who have built solutions during the last week that, for example, Brain Trust now has a AI proxy where you can use the OpenAI SDK to effectively query multiple different models, including Mistral and Llama through Perplexity, as well as a variety of other things, GPT-4, GPT-3.5, I think potentially Anthropic. And so it just allows you to be able to both load balance your queries or prompts, but also interchange models more easily. So you can actually look at performance across them. I think Chima similarly has done something over the last week that they've released that helps with some of the proxying and other things. And so I think there's solutions like that that have started to be accelerated in a market that would have happened inevitably.

• 7:09Speaker 01

  Right? I think everybody, the journey that I see people often take is they'll prototype on GPT-4, they'll look at how good it is, and then they'll either keep it on GPT-4, particularly if they need like advanced channel logic or other things, or if they need very high throughput and performance and low costs, then sometimes they'll either switch to GPT-3.5, or they'll see if they can fine tune something, right? And that's the only people with enormous scale. Like, I don't see very many fine tunes happening in general unless, you know, somebody has enormous scale and/or proprietary data they just don't want to get out, right? So they'll fine tune Mistral or Llama or something. So already, I think people were thinking about that. And then that means you need to build an orchestration layer, you may need the proxy, you may need a variety of things.

• 7:45Sarah Guo?

  So The dimensions that people were evaluating an LLM provider on, or whether or not they wanted to control or host or fine tune themselves, just became more clear, right? Where like reliability became more important, but the reliability latency, cost control, capability questions were sort of naturally there. And to be clear, like OpenAI leads on capability in many areas, in unique capability in some, right? Like cogeneration, GPT-4V, right? You can do amazing things with that, and people should go build on those tools. I

• 8:29Sarah Guo?

  think the ecosystem will mature and OpenAI has a great partner. But I think the questions are just much more obvious for anybody relying on these models now.

• 8:38Speaker 01

  Yeah. And I think honestly, a lot of the bigger enterprises I knew always wanted to make sure that if they really needed to, that they could second source something. So I don't think this is a new thing. In other words, you know, one could argue that no matter what OpenAI does, there'll always be at least one or two other suppliers or vendors or partners for advanced LLMs, simply because the market always wants an alternative, even just for negotiation leverage. And so if you look at other markets, for example, in the router world, one of the main reasons Juniper exists is because everybody wants to make sure that they can push on Cisco for pricing. And so they always want to have a second source. That's why Juniper is always 10% to 20% the size of Cisco, right? It's just second sourcing or AMD versus Intel for a very long period of time.

• 9:24Speaker 01

  So I think often markets will end up with other players just because big enterprises always want to have that option if they need it, even if it isn't as good. And if anything, I think OpenAI kind of emerges more stable through this in ways that people didn't expect, simply because there's going to be more stability at the board level in a way that people didn't understand perhaps that there could have been instability, right? This is a strengthening event and a focusing event for the company, at least from what I can tell extra. Do you want to talk about Pika and video?

• 9:56Sarah Guo?

  Yeah, yeah, there are a couple of really amazing launches happening in the video space. What's the cause for this? Like, we suddenly have text to video generation and avatar cloning in different ways. What do you think is going to happen in this space?

• 10:12Speaker 01

  Interesting shift has been happening because basically if you go back a year and a half, Midjourney launched, Stable Diffusion came out, Dolly 2 came out. And there's a whole wave of people saying that they were going to go build on diffusion models. And ImageGen was like the thing that everybody was going to go do for like two months. And then ChatGPT came out and then everybody was like, oh, my God, I need to go work on LLMs and language and natural language and NLP and all this stuff. And so the entrepreneurial ecosystem went through this sort of zigzag where everyone was going to do ImageGen and a bunch of companies started going down that direction. And then the LLM stuff really kind of was substantiated through ChatGPT. And then most people went that way. And a handful of founders stuck around on the diffusion model side. And diffusion models are really popping up. And obviously there was like Image Transformer and a bunch of other stuff,

• 10:57Speaker 01

  but they're mainly being used for ImageGen, for video and for audio, actually. And so there's a wave of people who've continued to work and crank on this. And they're starting to come out with really interesting products. For example, Pika is a great example where it was two Stanford PhD students who'd been working on diffusion models for some time. And they made this really amazing, creative text-to-video engine. There are other companies like a HeyGen or Synesthesia or others that are doing, let me use these diffusion models to clone an avatar, to generate an avatar of a person, so that they can either go into the metaverse, L L Suck, or alternatively, they can use it for marketing purposes, they can use it for internal training, they can use it for all sorts of applications.

• 11:40Speaker 01

  And then there's some really cool, like, audio-based things coming out, too, which I think are starting off more sort of tools to create music, or to simulate voice in the context of a soundtrack, or you like make EDM and you want to add voice to it, right? And you can just now kind of do some really interesting things there. So it seems like there's this really interesting renaissance that's happening in part due to diffusion model work. And in part due to a handful of founders not getting distracted by L L M's, which are super exciting, obviously, but wanting to do things in video. It's a really exciting trend, and I'm guessing the success of some of these companies and their traction and growth is going to pull more people over to work in this area again. I think it was just an area of less emphasis for the last year relative to language.

• 12:23Sarah Guo?

  One of my favorite dynamics that happens in sort of technology ecosystems is that once people show that something is possible, like, a lot of talent floods in, right? And you kind of get, or you get a lot of competition, but you also get innovation coming in waves. That could be with Mistral developing open-source models that are actually interesting from a reasoning perspective at relatively small parameter size. Or it could be Demi and Chenling and the Pika team creating text-to-video generation models that are really interesting, quite efficiently, from a training perspective.

• 13:07Sarah Guo?

  And I know we're both investors here, but I've seen a huge wave of people interested, as you said, in media diffusion of different kinds now that they know it's possible.

• 13:18Speaker 01

  And it has real benefits, right? Because it's very cheap to do. And from a data set perspective, the data is reasonably straightforward to get. I mean, it's hard to get, but it's not as hard as, you know, the entire internet and transcribing voice from videos and all the rest of it. The original stable diffusion model supposedly was trained on like 600k of GPU.

• 13:38Speaker 01

  And these models cost in the millions to train, not tens of millions at least initially, right? And so that's another big difference relative to the really big foundation models and language models and all the rest, right? And so you can actually imagine that in the language world, you're going to have a lot more platforms that people build on. In the diffusion model world, image, video, et cetera, you're going to have more people kind of grow their own, right? And people should still potentially try things on stable diffusion first just to test it out. It's back to the, you know, no GPU before product market fit, but they can still train their own model in a very economic way relative to like the amount of money a startup would raise. So I think it is a more accessible thing in some sense, unless you just go and build on somebody else's LLM, which people should do for most things initially as well.

• 14:20Sarah Guo?

  Yeah, one of the things I think is really interesting about this space as well is we've actually had leading researchers like say, you know, we're still very, very early in video. Video generation is so hard. It's data intensive. The data is like, as you said, it's not the whole internet, but it's problematic in that a lot of the training has happened on short clips. People aren't sure how to caption. It's expensive to generate. You have like complex like sliding window approaches and others to try to deal with the like temporal coherence problem. There are many more unknowns about how to progress this type of product technically.

• 14:55Speaker 01

  Do you mean video specifically?

• 14:56Sarah Guo?

  For video specifically.

• 14:57Speaker 01

  Yeah, because the teams for all these things are actually quite small, right? The Pika team is reasonably small. The Mid Journey team for a long time was tiny. And so I actually think this is a good example where you can do a lot with very small teams. And to your point, there's all sorts of technical challenges. But the reality is you can get to the cutting edge with like a handful of people. In these fields, which isn't necessarily true as much for, you know, other types of models or certain types of models at least. So I do think it's striking how few people you actually need to do something really interesting here. And to your point, there's other challenges coming. Three, four years, maybe it becomes harder.

• 15:28Sarah Guo?

  I agree with you. You don't need more than a handful of people. Like there's empirical evidence now. Maybe a slightly different point, which is instead of like having to have a certain size of team to go deliver an LLM at scale, there are more degrees of freedom in how you would innovate technically in this area. And there's more disagreement on like how to progress. And I think that's actually just interesting for startups.

• 15:51Speaker 01

  Yeah, I think they should just use Q star.

• 15:54Sarah Guo?

  That's probably.

• 15:55Speaker 01

  I think that's, that's the main solution to most problems, I feel, in AI today.

• 16:01Sarah Guo?

  Okay, well, do you want to give me some investment advice, given Q star? Should I go home?

• 16:05Speaker 01

  Yeah, you should do mainly Q star centric companies. And so, you know, if they're doing Q star, you do it. If they're not doing Q star, you don't do it. So that's one big piece of advice.

• 16:15Sarah Guo?

  I think one of the things I would like to go back to and talk about is your point of view on like, hey, a bunch of people moved away from diffusion models to LLMs.

• 16:23Speaker 01

  Yeah.

• 16:23Sarah Guo?

  One of the reasons that people moved away from diffusion models to LLMs is because there's a lot more sort of text and code in enterprises that is obvious, right? Working with images and video and audio felt more verticalized, like where the B to B use cases. And I think what we're seeing increasingly is the creative fields are, are actually pretty commercial, right? So one of the things I'm most inspired by, and I think there's a lot of money in, is if you look at mid journey, one of the biggest knocks on them from investors or naysayers early on

• 17:08Sarah Guo?

  was, well, like, how many people want to make images?

• 17:13Sarah Guo?

  That's not a hobby, that's not a social network. Like, what percent of the population are artists? And this was clearly wrong, just in terms of the scale that Midjourney has already reached. There's probably two pieces here, right? Like, one, these tools like Pika and Haagen and audio generation and Midjourney, they make the pie bigger for creative fields, especially since if you look at Pika or Haagen, they're really focused on all creators, rather than just like the film industry professional. And like, if you go look at the Midjourney use cases, and then I suspect the Pika and Haagen use cases over times, they're very commercial, right?

• 17:55Sarah Guo?

  Like, a lot of the things that you named or that people are experimenting with are really about communication, marketing, and advertising.

• 18:03Speaker 01

  Yeah, I think if you just look at it as market cap of incumbent, right? Adobe's a 280, 300 billion dollar company. Like, that's huge, right? And so I don't think the creative world is small, right? I think a lot of creator economy companies have failed in the past, which is a different thing, depending on how you think of the creator economy, right? In terms of, you know, celebrity-based marketing or whatever. But if you actually look at enterprises, obviously, they use enormous amounts of imagery and video and other things to reach with and interact and brand and, you know, associate with their customers. And you look internally, you need to create imagery for slides or for other things and communication and, you know. And so to some extent, you're kind of looking at different proxies and you say, okay, well, what are some of the proxies on the, um, on the text-based side?

• 18:46Speaker 01

  And you could say, oh, you add up Microsoft and, you know, a few other companies like that, and you're kind of getting a rough proxy for some form of text. Not really, but, you know, I'm just simplifying things dramatically. And then you add up Adobe and a few other companies, and that's your proxy for, for ImageGen, right? And so I think, um, both are big. And then the question, I think, always with the diffusion model-based companies was, where are the biggest application areas? And the application areas also were a little bit driven by, well, where will incumbents play a role? Where would you get blocked by other companies in the ecosystem versus, you know, it's a natural new greenfield thing? Most things aren't truly new capabilities. Most things are just like making certain things dramatically easier. There's the, the duality of that.

• 19:29Speaker 01

  There's the market expansion, where people can do this thing. And then there's a value contraction. Hey, you can do this at a tenth or a hundredth the cost. Yeah. And so often in markets like this, you see both of those things happen at the same time. You're simultaneously growing the market and shrinking it.

• 19:47Sarah Guo?

  Yeah, I mean, you have the, um, the, um, famous, uh, strategy like your margin is my opportunity, right?

• 19:55Speaker 01

  Uh, I was gonna say, I mean, well, these are actually higher margin things. I think really what you're doing, if you look at the sort of, um, uh, my understanding, I need to look up these numbers again, but I think it's something like software spend is like, I don't know, I'm making it up, half a trillion dollars, 500 billion dollars a year. And then services spend is like three to five trillion a year. Right? And so really what you're doing is you're taking services revenue, which is very people intensive and low margin, and you're converting it into higher margin software revenue, but less of it. So maybe you take that five trillion and you turn it into two trillion, but it's 80% margin, two trillion versus 30% margin, right? The margin dollars actually expand. And you see that in other industries. That's kind of what Anduril is doing in defense, right? They're taking a

• 20:39Speaker 01

  cost plus model, right? You buy a drone from Lockheed Martin for a million dollars and you, you get paid by the government 5% cost plus, which means you get 5% as your margin. So you make 50K off of it. And Anduril will sell the same drone or, you know, better drone for a hundred thousand dollars with 50% margin. I'm making up the margin, right? But that's 50K. And so you're making the same margin on a 10th of the price. And so I think one of the ways I think about Anduril as a company is they're taking very bad, low margin revenue from other defense companies and turning it into higher margin, healthier revenue, right?

• 21:15Sarah Guo?

  Yeah, uh, I'm gonna edit the quote and just say like, your ASP is my opportunity, right? There's a, like a democratization that happened in the like latter half of the SaaS revolution, or really most of the SaaS revolution, which is instead of there's 100 very large enterprises that have some sort of CRM because it costs X dollars to, um, deploy and implement Steeble, then you have, you know, 10s of thousands of companies who can buy Salesforce. And then companies that figure out how to efficiently distribute, um, SMB SaaS on the internet, even though that, that is still hard. I, I think one of the

• 21:55Sarah Guo?

  things that is, um, interesting here is, let's just take video generation as the example, the ASP of, you know, hundreds, two thousands of dollars an hour.

• 22:08Sarah Guo?

  Make it single-digit dollars to generate per minute and expand the audience to many more people, right? So that's the democratization that is happening. Somebody told me a joke the other day, like somebody really negative on AI investing. There's only five businesses in AI that have breakout traction right now. Foundation models, waifus, mid-journey, copilot, and inference platforms. I think there's both truth in it and like also it's not that funny of a joke because you're like, it's true. Like there is a set of things that people are figuring out that are really early, many of which feel pretty different than like, yes, it's it's useful

• 22:53Sarah Guo?

  to look at the incumbent vendors like Adobe, but you're not fighting really the video editing software spend. You're eating into the production spend.

• 23:04Speaker 01

  Sure. The early internet version of this, by the way, is there's only five things you do on the internet. You go to Yahoo to look for links. You buy pes dispensers and auction on eBay. You buy some books on Amazon. And then there's probably like some like two bullshitty companies that you would have quoted as like high-grade things, right? So if you were looking at the internet circa 96, 97, whatever, you probably would have had a pretty short list of real use cases and then a bunch of stuff you just thought was kind of dumb, right? And you'd be like, look, you're not changing anything. Like you're still using Microsoft Office and you're still using whatever shrink wrap software. You're still watching TV, right? And so I feel like we're kind of in that era of AI. The stuff that's going to happen right now is the really easy, low-hanging fruit and then a bunch of dumb things are going to get built that aren't going to work. And dumb

• 23:49Speaker 01

  is not meant in a pejorative way. It just means like, it's very hard to tell what's actually a good idea in a new market like this. And that was true of the internet and that was true of mobile and that was true of cloud. There's a lot of these like waves where there's a bunch of stuff that gets built, right? So I think it's the same thing, right? It's a very positive sign. This has been such massive traction in such a short period of time for so many companies, if you think about it. It's actually kind of amazing. So I'd actually take the other side of that, but I totally get the point.

• 24:20Sarah Guo?

  Yeah, well, with the AI-focused fund, I agree with you.

• 24:24Speaker 01

  Are you gonna change the name of your fund? You should call it like Conviction Star or something.

• 24:28Sarah Guo?

  Conviction Star? Yeah.

• 24:30Speaker 01

  But spelled with a Q, like conviction.

• 24:32Sarah Guo?

  Conviction Star. Okay, LPs, you heard it here first, Conviction Star.

• 24:36Speaker 01

  Very exciting. I should send you a T-shirt.

• 24:38Sarah Guo?

  Thank you, please do. That'll actually be the swag for No Priors Season 1. If you're a guest, you're gonna get the No Priors Tequila and then a Conviction Star T-shirt.

• 24:48Speaker 01

  Yeah, that's very good. I'm very excited about the tequila. Anybody on the podcast has to have a tequila. We could actually call the tequila Conviction Star with a Q.

• 24:58Speaker 01

  That'd be pretty amazing. I'm serious.

• 25:00Sarah Guo?

  Okay.

• 25:01Speaker 01

  It could be like a Q-shaped bottle. You know how they have like the really cool bottles for different things?

• 25:05Sarah Guo?

  A lot Gil, guys, are No Priors branding.

• 25:08Speaker 01

  Brand marketer. Yeah, I think I'm moving to LA and starting the brand.

• 25:12Sarah Guo?

  If you haven't yet been a guest, please write into the show and, you know, for the low, low price of one GPU, we'll ship you a bottle.

• 25:21Speaker 01

  Yeah, we're looking for a brand marketer to join the team, too.

• 25:25Sarah Guo?

  We're not, actually.

• 25:26Speaker 01

  If you work for Mr. Beast, just call me.

• 25:29Sarah Guo?

  A lot's gonna do it. Okay, A lot, thank you for joining me on No Priors.

• 25:34Speaker 01

  Thank you for joining me.

• 25:35Sarah Guo?

  And I look forward to getting my Conviction Star T-shirt and tequila.

• 25:41Speaker 01

  Exciting.

• 25:42Sarah Guo?

  Find us on Twitter at NoPriorsPod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.