OpenAI Sora 2 Team: How Generative Video Will Unlock Creativity and World Models

The OpenAI Sora 2 team (Bill Peebles, Thomas Dimson, Rohan Sahai) discuss how they compressed filmmaking from months to days, enabling anyone to create compelling video. Bill, who invented the diffusion transformer that powers Sora and most video generation models, explains how space-time tokens enable object permanence and physics understanding in AI-generated video, and why Sora 2 represents a leap for video. Thomas and Rohan share how they're intentionally designing the Sora product against mindless scrolling, optimizing for creative inspiration, and building the infrastructure for IP holders to participate in a new creator economy. The conversation goes beyond video generation into the team’s vision for world simulators that could one day run scientific experiments, their perspective on co-evolving society alongside technology, and how digital simulations in alternate realities may become the future of knowledge work. Hosted by: Konstantine Buhler and Sonya Huang, Sequoia Capital

• 0:00Speaker 00

  For OpenAI across the board, it's really important that we kind of like iteratively deploy technology in a way where we're not just like dropping bombshells on the world when there's like some big research breakthrough. We want to co-evolve society with the technology. And so that's why we really thought it was important to like do this now and like do in a way where, you know, we've hit this, again, this kind of like GPT 3.5 moment for video. Let's make sure the world is kind of aware of what's possible now and also, you know, start to get society comfortable and like figuring out the rules of the road for this kind of like longer term vision for where there are just copies of yourself running around in Sora and the ether, like just doing tasks and like reporting back in the physical world because that is where we are headed long term.

• 0:54Speaker 01

  Today on Trading Data, we sit down with the team behind OpenAI's Sora, Bill Peebles, Thomas Dimson, and Rohan Sahai. You'll hear about spacetime tokens, building internal world simulators, and how optimizing for creation instead of consumption is just better for social platforms. This conversation goes way beyond video generation and into questions about how society will co-evolve with powerful simulation technologies. We promised that this was an actual, real-world conversation and not a video generation, but we don't know how to prove that to you. Let's jump in.

• 1:27Speaker 01

  Hey, guys. Thank you for being here at Sequoia. Congratulations on Sora. Thank you. Maybe you could tell us a little bit about yourselves and how you got to OpenAI and Sora?

• 1:36Speaker 00

  Yeah, I'm Bill. I'm the head of the Sora team at OpenAI. I had a pretty traditional path, came through undergrad doing research on video generation, then continued that work at Berkeley, and then started at OpenAI working on Sora from, you know, the first day I joined.

• 1:51Speaker 02

  And I'm Thomas. I work as an engineering lead inside of Sora.

• 1:56Speaker 02

  I have a bit of a longer story, but I worked at Instagram for about seven years, doing some of the early kind of machine learning systems and recommender systems there. But it was a very tiny company. It was about forty people. Then I quit, did my own startup for a while, which was Minecraft and the browser, which we've talked about a couple of times. And I think that OpenAI noticed that we had a very cracked product team there. And so they acquired our company. And I've been bouncing around different products inside of OpenAI and on the research side as well on post-training. But super happy we landed kind of together on Sora to bring this thing to life.

• 2:29Speaker 01

  It was a really cool product in between two, like the global illumination product.

• 2:33Speaker 02

  Oh, yeah, I still believe in it.

• 2:35Speaker 01

  Yeah, me too.

• 2:36Speaker 03

  Awesome. I'm Rohan. I've been at OpenAI for about two and a half years. Started as an IC on ChatGPT. But then, as soon as I saw the video gen research, I got quickly Sora-pilled and made my way over there. And so currently lead the Sora product team. Before that, just startups, big companies within kind of the valley, a bunch of random stuff.

• 2:55Speaker 02

  Yeah, cool.

• 2:57Speaker 01

  Well, Bill, you are the inventor of the diffusion transformer. Can you tell us what that is?

• 3:03Speaker 00

  Yeah, so most people are pretty familiar with autoregressive transformers, which is the core tech that powers a lot of language models that are out there. So there you generate tokens one at a time, and you condition on all the previous ones to generate the future. Diffusion transformers are a little bit different. So instead of using autoregressive modeling as kind of the core objective, you're using this technique called diffusion, which at a very high level, basically involves taking some signal, for example, video, adding a ton of noise to it, and then training neural networks to predict the noise that you applied. And this is kind of a different kind of iterative generative modeling. So instead of generating token by token, as you do in autoregressive models, diffusion models generate by gradually removing noise one step at a time.

• 3:45Speaker 00

  And in Sora 1, we really kind of popularized this technique for video generation models. So if you look at all of the other competitor models that are out there, both in the States and in China, most of them are based on DITs diffusion transformers. And a big part of that is because DITs are a really powerful inductive bias for video. So because you're generating the whole video simultaneously, you really solve issues where quality can degrade or change over time, which was kind of like a big problem for prior video generation systems, which DITs ended up fixing. So that's kind of why you're seeing them proliferate within video generation stacks.

• 4:17Speaker 01

  When I try to visualize it, I mean, for each diffusion, you have a matrix of pixels. And then you do the entire video at the same time, which you can basically see as different frames, I imagine. Can you visualize that as, you know, a matrix of matrices that basically transforms over time?

• 4:32Speaker 00

  Yeah, it's a good question. So we really kind of consider things at the granularity of like spacetime tokens, which is sort of like an insane phrase. But, you know, whereas, you know, for example, characters are very fundamental building block for language. For vision, it's really this notion of a spacetime patch, right? You can just imagine this little cuboid that composes both X and Y, like spatial dimensions, as well as a temporal locale. And that really is kind of like the minimal building block that you can like build visual generative models out of. And so diffusion transformers sort of consider these, almost you can think of it like voxel by voxel. And, you know, in the traditional versions of these diffusion transformer models, you have all of these little spacetime patches talking with all of the other ones.

• 5:16Speaker 00

  And that's how you actually are able to get properties like object permanence to fall out because basically you have full global context of everything going on in the video at every position in spacetime, which is like a very powerful property for a neural network to have.

• 5:30Speaker 01

  Mm hmm.

• 5:31Speaker 00

  Yeah.

• 5:31Speaker 01

  And is that the equivalent of the attention mechanism? Is the object's movement throughout the video?

• 5:37Speaker 00

  Yeah, that's right. So in our like Sora 1 blog post, video generation models as world simulators, we kind of laid out some visuals, which sort of go into exactly your point here, which is really, attention is like a very powerful mechanism, right? For sharing communication, like sharing information across spacetime. And if you represent data in this way, right, where you patchify it into a bunch of these spacetime tokens, as long as you're, you know, properly using the attention mechanism, that allows you to transfer information throughout the entire video all at once.

• 6:05Speaker 04

  What are the biggest differences between Sora 1 and 2? And I remember with the original Sora 1, you already seeing kind of emergent properties where the more you scale, the more it's able to do things like understand physics. What is, is Sora 2 purely a function of scaling or what are the biggest differences?

• 6:20Speaker 00

  Yeah, that's a great question. You know, we've spent a long time really just doing like core generative modeling research since the Sora 1 launch to really figure out how we get the next step function improvement and video generation capabilities. We really kind of operated from first principles, right? So we really want these models to be extremely good at physics. We want them to kind of feel intelligent in a way that I'd say like most prior video generation models don't. So by that, I really mean, you know, if you look at kind of any of the previous set of models that were out there, you'll notice a lot of this kind of like effects that happen. Like if you try to do any sort of complicated sequence of like physical interactions, right? For example, like spiking gymnastics, classic. Riding a dragon like you did. Riding a dragon? That was fun. That was, that happened for real, actually, Constantine.

• 7:06Speaker 00

  You know, there are like very clear problems with the past generation of models that we really like set out to solve with Sora 2. And I think one thing that's really cool about this model compared to prior ones is that when the model makes a mistake, it actually fails in a very unique way that we haven't seen before. And so concretely, for example, if, let's say like the text input to Sora is a basketball star wants to like shoot a hoop, right? Shoot a three throw. If he misses in the model, Sora will not just like magically guide the basketball to go into the hoop, right? To be over optimistic about respecting what the user asked for. It will actually defer to the laws of physics most of the time and the basketball will actually like rebound off the backboard. And so this is a very interesting distinction, right? Between like model failure and like agent failure.

• 7:50Speaker 00

  Agent as in the agent that Sora is like implicitly simulating as it's generating video. And we haven't really seen this very unique kind of like semantic failure case in like prior video models. This is really new with Sora 2. It's kind of a result of, you know, just the investment we put in like really doing like the core generative modeling research to like get this massive improvement in capability.

• 8:09Speaker 04

  Okay, so not purely a function of scale. You're actually, you know, there's, there's some concept of agents implicit in this. There's, there's things you're doing beyond just scaling.

• 8:18Speaker 00

  Well, the, the notion of agents, I'd say is actually mostly like implicit from, from scale. Like, you know, in the same way where we kind of showed that object permanence, right? Begins to emerge in Sora 1 pre-training once you hit some like critical flops threshold. We see similar kinds of things happen as we like push the next frontier, right? So you begin to see these agents act more intelligently. You begin to see the laws of physics be respected in a way that they aren't at like lower compute scales. How does the concept of a spacetime latent patch relate to a spacetime token, relate to object permanence and how things move through the physical world? Yeah, that's a great question. So I'd say spacetime patch and spacetime token are more or less synonymous with one another. I'll use them interchangeably.

• 9:01Speaker 00

  You know, what's really beautiful, right, is when people started scaling up language models from like GPT-1 to GPT-2 to GPT-3, we really began to see the emergence of like world models internally in these systems. And what's kind of beautiful about this, right, is there's incredibly simple tokenizers that actually go into like creating the data that we train these systems on. But despite this very simple representation, right, you know, like BPE, characters, what have you, when you put enough compute and data into these systems, like in order to actually solve this task of predicting the next token, you need to develop an internal representation of how the world functions, right? You need to like simulate things. And like, you know, the models will make lots of mistakes right now at like low compute

• 9:43Speaker 00

  scales, but as you continue pushing it from three to four to five, you just see these internal world models get more and more robust. And it's really analogous for video, right? And in many ways, more explicit. So I think it's easier to picture what like a world model or a world simulator looks like with video data, right? Because it is literally representing like the raw observational bits of like all of reality. But what's really remarkable, right, is because these spacetime patches are just this like very simple and like highly reusable representation that can apply to like any type of data, right? Whether it's just like video footage of like this set, whether it's like anime, cartoons, like whatever it is, you're just able to build like one neural network that can operate on this vast, extremely diverse set of data and really build these like incredibly powerful

• 10:25Speaker 00

  representations that model like very generalizable properties of the world, right? It's useful to have a world simulator to predict like how a cartoon will unfold. And likewise, it's useful for predicting how, you know, this conversation might unfold. And so that really puts a lot of optimization pressure on Sora to like grok these like core fundamental concepts in a very like data efficient way.

• 10:43Speaker 01

  Did you have to put effort into selecting the data such that it reflected the physical world? For example, I'd imagine if you have data from the physical world, it all abides the laws of physics. But you mentioned anime, that might not always abide in the laws of physics. Did you have to be selective or did it naturally find patterns that separated that out?

• 11:03Speaker 00

  That's a really great question. We did spend a lot of time, you know, really thinking about. You know, what does like the optimal data mix for like a world simulator kind of look like? And to your point, you know, I think in some cases we'll make decisions that, you know, maybe are for like making the model really fun. Like, for example, people love generating anime, but you know, do not necessarily like perfectly represent like the laws of physics that are like directly useful for like real world applications. So like to put it another way, right? I think in anime, there are certain primitives that are simplified that are actually probably useful for understanding the real world. You know, people still locomote through scenes, for example. But like if there's like some crazy dragon that's like flying around, that's probably like not so useful for like rocking aerodynamics or something. Dragon Ball Z is more or less how I learned athletics. You know, there you go.

• 11:48Speaker 00

  Motion and Super Saiyan. I think it is an interesting question like that I do not know the answer to whether somehow like pre-training on simplified representations of like the visual world, whether that's like sketches or like some other modality. Like, you know, makes you more efficient at like rocking these concepts. I think it's actually a very interesting scientific question that we we need to understand better.

• 12:10Speaker 04

  Do you think we're close to exhausting the number of pre-training tokens there are out there? Or do you think video data is just video is just so massive and it's actually one of the more untapped vats of data?

• 12:19Speaker 00

  Yeah, the way I kind of think about this is the intelligence per bit of video is much lower than something like text data. But if you integrate over all of the data that really exists out there, the total is much higher. So to directly answer your question, you know, I think it's hard to imagine ever fully running out of video data. There's just like so many ways that it exists in the world. That like, you know, you will be in a regime where you can continue to just like add more and more data to these pre-training runs and continue to see gains for like a very long time, I suspect.

• 12:48Speaker 04

  You think we'll ever discover new physics? There's the LLM world of, you know, Einstein thinking the whiteboard. It's all I'm thinking. There's also just the if you develop a perfect simulator and you just simulate physics better and better, you you might learn things about the world that we haven't learned yet.

• 13:02Speaker 00

  I totally think that this is like bound to happen one day. And like, you know, I think we probably need. Even like, we probably need one more step function change, I'd say, in like model quality to like really get to a point where, for example, you can think about doing like scientific experiments in the models. But like, you could imagine, right, one day, you have a world simulator that is like generalized so well to the laws of physics, that like, you don't even need like a wet lab in the real world anymore, right? You can just like run biological experiments within Sora itself. And like, again, this needs like a lot of work to like really get to the point where like you have a system that's robust enough to do this reliably. But you know, internally, like again, we've used Sora 1 as kind of being like the GPT-1 moment for video. It was like really the first time things started working for that modality. Sora 2, we really view as like GPT-3.5, uh, in terms of like it really being able to like kickstart,

• 13:47Speaker 00

  you know, the world's creative juices and like really like break through this kind of usability barrier, where we're seeing like mass adoption of these models. And we're gonna need a GPT-4 breakthrough to really get this to the point where this is useful for like sciences as we're seeing now with GPT-5, right? Like, I feel like every day on Twitter, I see another like convex optimization lower bounds get like improved by GPT-5 Pro. And I think eventually we're gonna see the same thing happening for the sciences with Sora.

• 14:11Speaker 04

  Do you think you need physical world embodiment to get there? Or do you think a lot of it can be done in, effectively in Sim?

• 14:16Speaker 00

  You know, I am like, I'm always amazed, you know, every time we like push another like 10x compute into these models, like what just like magically falls out of it, um, with like very limited changes and kind of like what we're training on and like the fundamental like approach to, to what we're doing.

• 14:34Speaker 00

  You know, I suspect some amount of like physical agency will certainly help. I have a hard time believing it will like make you worse at like, you know, modeling like collisions or like something else. Video only is like quite remarkable though, and I wouldn't be surprised if it's actually kind of like AGI complete for like building like a general purpose world simulator.

• 14:51Speaker 01

  So for this concept of a general purpose world simulator, a world model,

• 14:54Speaker 00

  Yeah.

• 14:54Speaker 01

  where you can do science experiments in that world, do you think that video is the soul? Or some combination of video and text are the combined data inputs? And you train it on this type of, uh, this type of model? Or is it gonna be, does it have to be based on more structured laws of physics that are understood and laws of biology that are understood?

• 15:20Speaker 00

  I think it probably depends a lot on, um, the specific use case you're kind of envisioning for, for the world simulator, like. For example, if you just really want to build like an accurate model of how like a basketball game is played, I actually think like only video data and like maybe audio as well, like kind of sufficient to to build that system.

• 15:37Speaker 01

  Not of me playing basketball. That would be an inaccurate, very bad player of basketball.

• 15:42Speaker 00

  You know, uh, you actually like Sora's current understanding of how people play basketball, Konstantin, may be at your level.

• 15:49Speaker 01

  Wow, okay, that makes sense.

• 15:51Speaker 00

  It's possible. It's possible.

• 15:54Speaker 04

  I think he just dissed you.

• 15:55Speaker 01

  I liked it. It's accurate.

• 15:56Speaker 00

  But it's better than mine, Konstantin. For once, that was like a Sora 1 situation. You're at Sora 2.

• 16:01Speaker 01

  We'll toss some hoops. Is that what they'll say?

• 16:03Speaker 00

  You know, I'm down. Okay, I'm down. Yeah. Shoot some hoops.

• 16:06Speaker 01

  Thanks.

• 16:08Speaker 01

  Thomas's first, first statement in the podcast.

• 16:10Speaker 00

  But I'm also at your level, so I'm sorry.

• 16:13Speaker 00

  You know, I think it is an interesting question, like what are all of the modalities that should be present in like this kind of general purpose system? Like certainly, you know, if you add more modalities, I have a hard time believing it will like decrease the intelligence. I also think there's an argument to be made that, um, just, you know, adding more and more does not provide like significant marginal value compared to like, you know, full mastery of like video and audio, for example. I think it's an interesting open question. I'm not, I'm not actually sure right now. Um, and it's something we need to understand more.

• 16:41Speaker 01

  So cool. Sonya a minute ago mentioned Einstein at a whiteboard. And obviously that makes me think of you, Thomas, and your hair.

• 16:49Speaker 00

  Me too.

• 16:51Speaker 01

  It had, it had to come. Like if, if any hair gives the feeling of space-time tokens, it's definitely, definitely yours. At some point, you know, Bill, you're the creator of this revolutionary technology that has changed the way that AI video is created. At some point, you from Sora 1 to Sora 2 said, hey, altogether, you said, there needs to be an application around this. There's some benefit to an application. You brought together some of the best product people in the world. How did that crew come together at OpenAI?

• 17:23Speaker 02

  Yeah, it's a, I mean, the story is never as linear as you might think it is. So I think that, I mean, we've had a product team on Sora since the get go. Rohan was like spearheading that effort in the Sora one days. But I think Bill's right when he says it was really like a GPT one kind of moment. We're seeing pockets of very interesting things there. But the models were not like, you know, models without sound, videos without sound. It's like a very different kind of environment. And so we're working on that surface, mostly targeted on, on kind of like a prosumer demographic. And separately, I mean, Rohan, probably go into more details of all that. Separately, we're also just kind of exploring different social applications of AI inside of OpenAI and like what that could look like.

• 18:05Speaker 02

  We had a lot of prototypes, most of which were quite bad. And when we started to see some of the magic was actually with ImageGen before it had been released, we were playing with it internally in a social context. And the social context was really interesting to see that what people were doing is he'd sort of like take an image and then you'd have like a chain of remixes of that image. Where like, I don't know, there was a, it's a duck and then now the duck's on somebody's head. And now everything's upside down and they're smoking a cigarette. Like, just a lot of weird things. It's a crazy party. Yeah, it's like, and we were seeing this, we're like, oh, this is kind of like a very interesting thing. That's like, you know, nobody can really do that with like social media because it's so hard to create something or riff on something. It like, it's such a high barrier to entry action.

• 18:50Speaker 02

  Maybe you have to get a go, get a camera set up and not, it's not just like thinking of the idea. There's actually a lot of things involved. And so we were like, okay, this is a very magical behavior. How can we kind of productize that behavior? And we were mostly thinking about it away from Sora. Some of the Sora research was still ongoing and I mean, there were signs of life, but it wasn't like quite there yet in productized form. Bill probably had it in his head somewhere. It's like, I can see the future, but that's fine. I'm a little bit more, can't, can't quite see the future yet. So, so we were just exploring that. I think we tried a few things and then at some point the, the research was really just showing very clear value of even iterative deployment style value of like, oh, this is something that people will really want. And so.

• 19:33Speaker 02

  So we went into this project like two or three months ago. Yeah. It wasn't very long. It was like July 4th. July 4th, yeah. Yeah. That's when you disappear, Thomas. That's when I disappeared, yeah. Makes sense. Yeah, so and we just kind of locked in like, okay, we're finally doing it. You know, that's always a moment. And we started with without any magical features, just like, okay, let's just try to get native video environment where you can hear the audio fullscreen.

• 20:03Speaker 02

  And we did some quick generations. Things were showing very, they're very cool, very fun, very interesting. And because of that ImageGen experience, we sort of had thought like, okay, what's the magical here, magical thing here is that like barrier to entry is very, very low for creation. Coming from Instagram, that's like, it's impossible to get people to create on Instagram, and that's the most valuable thing that people do. So what does that unlock? And it's like, okay, well, that remix thing from ImageGen, that kind of could still apply here. And so we brainstormed all these things about how could remixes work? And like, what does a remix mean here? One of those was this like cameo thing, which I think also Bill had in his head, but this it wasn't. It was in the ether. It was in the ether for sure. But we just were like hacking together things on the product.

• 20:48Speaker 02

  It was, I'll see if this works. I, I didn't think it would work at all. But it was on the list and there were a few other things on the list. Some of them were pretty crazy. It was like a. Why didn't you think it would work? I am bad at predicting technology. Like, it wasn't super clear to me that you could like, you know, take a likeness of a, of a person and have that kind of imagined into a video form. And whether it would work or not. And so we had early prototypes of different things of like people reacting in the, in the video corner or stuff like that. But when, when we saw cameos just start to work and even playing internally, like. Rohan, do you remember that day where we're like?

• 21:27Speaker 03

  Yeah.

• 21:28Speaker 02

  Feed is entirely cameo.

• 21:29Speaker 03

  Yeah, it's entirely. It just went from, you know, we didn't have that feature. Once we had that feature, product market fit on the team, all, everything we were generating was all of each other.

• 21:38Speaker 04

  You must have seen the meme potential.

• 21:41Speaker 03

  I mean, yeah, that's, I think at first, we were just like, this is hilarious, this is amazing. And then like a week later, we were like, this is still all we do. There's something here.

• 21:52Speaker 02

  Yeah. I mean, at first, we were actually a little bit like, is this good?

• 21:55Speaker 03

  Yeah.

• 21:56Speaker 02

  Hey, the cameos, it's just all cameos now.

• 21:58Speaker 03

  Does anyone else care about this?

• 21:59Speaker 02

  People care about other people doing stuff? And we kind of got to the point where we're like, no, no, this is actually good. Like, it's actually, it feels like I'm coming back to see. And it really humanized it a lot, where like a lot of AI video is just kind of static scenes that are quite beautiful, quite interesting, might have extremely complicated things going on. But they lose that human touch and it really felt like it was coming back into it. So.

• 22:26Speaker 03

  Another learning from ImageGen too, like ImageGen took off and had viral moments because I think you could put yourselves in these scenes in accessible ways that weren't possible before. Obviously, this massive like, put me in a Ghibli scene. People taking selfies with their idols and stuff like that. And so the kid, once, you know, once you actually kind of thought about it, it's like, yeah, cameo feature makes a lot of sense. You put yourself in all these scenes, that's way more exciting, you and your friends. It's novel. It's like, that's something you could do before.

• 22:53Speaker 02

  Yeah. And then that combined with remixes, I mean, cameos kind of remix to begin with, but then you start to think about, okay, well, now I can riff on Rohan doing something.

• 23:01Speaker 03

  Yeah.

• 23:01Speaker 02

  Or whatever it is, like with Bill, had he wrapped in an action figure package and it was.

• 23:06Speaker 00

  It's been remixed like, yeah, an insane number of times.

• 23:09Speaker 02

  Thousands of times, yeah. So like, just very, very crazy things that kind of go on and very emergent. A lot of stuff that I would have never thought of.

• 23:16Speaker 00

  Actually, how many generations of you guys have been like publicly posted at this point?

• 23:20Speaker 03

  I have no idea.

• 23:21Speaker 02

  Uh, I know I'm 11,000 or so.

• 23:23Speaker 03

  I was like, a little less than that.

• 23:25Speaker 04

  Wow.

• 23:26Speaker 02

  Yeah.

• 23:26Speaker 00

  Yeah.

• 23:27Speaker 04

  What has surprised you about the types of users that are really sticking with Sora? Who is it really a hit with?

• 23:32Speaker 03

  If you just go to the latest feed, which is just like the fire hose.

• 23:36Speaker 01

  Astronaut mode.

• 23:37Speaker 03

  Of everything. Yeah, it's spacetime Thomas mode. It's wild out there, but that gives you a pretty good snapshot into like just everything happening. I mean, I think we have like almost seven million generations happening a day. So you can imagine there's just a ton of information there. It's one of my favorite ways to just get product feedback. It is so diverse, the type of stuff people are doing, the type of people. There'll be like a complete variety of age. Some people just doing envisioning themselves in scenes that seem like motivation oriented. People just memeing with their friends, people cameoing some of like the public figures on the platform that have done cameos. So I think the diversity has surprised me.

• 24:17Speaker 03

  I was kind of expecting this sort of like, you know, the Twitter AI crowd to like heavily dominate the feed. They definitely dominate like the press cycles, at least the ones that, you know, we're most exposed to. But in terms of people actually using this, it's quite a wide variety. And last thing I'll say is a bigger departure from like the sort of niche AI film crowd that existed before, which is great, early adopters, but now you kind of get these. I thought it would start there and it felt like it started with just a way wider range of people. I think getting to the top of the app store helps with that and just get people who are like browsing and see this thing.

• 24:53Speaker 01

  My mother keeps cameoing Thomas.

• 24:56Speaker 02

  Is that right?

• 24:59Speaker 01

  So weird. We got a lot of strange cameos.

• 25:02Speaker 03

  There are real stories like that.

• 25:03Speaker 01

  He's at 11,000. She's done 10,000 of them.

• 25:08Speaker 04

  Thomas, you wrote the original algorithm, if I'm right, for the Instagram ranking algo. There was a lot in the Sora 2 blog post about how you guys are clearly being very intentional about how you want to do ranking in the algo. Can you, can you talk a little bit about lessons learned from Instagram and how you're approaching it over at Sora?

• 25:26Speaker 02

  Yeah, I mean, there's, there's a lot to cover in that. I think that the first thing to think about when we think about these platforms or think about Sora specifically is it is the thing I was mentioning before about creation. So Sora enables basically everybody to be a creator on this platform. And that is a very, very different environment than something like Instagram, where you have this like extreme power law of the people that are creating. And the power law just naturally gets more. Narrow, what's the right word there? But more head heavy, yes. So sometimes I feel like I have to defend myself on the Instagram algorithm side. We actually did it for, I mean, we did it for a reason. It was to solve a problem.

• 26:08Speaker 02

  It wasn't just kind of like a random decision to, you know, optimize for ads or something like that. And the reason we did that was that we noticed that like what was happening on Instagram over time was because it was chronologically ordered, every single person that posted was guaranteed to have the top slot of all their followers. And so if you think about that for a second, the incentive for somebody in that environment is actually to create constantly because they are guaranteed distribution when they create. And over time, because of this like power law becoming heavier and heavier,

• 26:42Speaker 02

  or like more head heavy, those type of people, which are great, they provide a lot of value to the ecosystem, but they start to crowd out people that you really care about. And so maybe you follow National Geographic or something, not to dunk on National Geographic, I love them. But, you know, if they're posting 20 times a day, your friend's not, they don't have the same like optimization objective, they're probably just a picture of their coffee or something. And so you'd have 20 Nat Geo posts and then one picture that you actually really cared about, that you never really scrolled to. And there's not too many solutions to that problem if you have a guaranteed ordering. One of them is that you have to unfollow all these accounts that you maybe care about, but care about not as much as the person that posts once a day.

• 27:26Speaker 02

  And the other is that you have to permute the permute the feed. And so we went with that path. We tried it, we tested it out internally, it was very kind of controversial to do.

• 27:37Speaker 02

  And, but I think that you can actually kind of like math this out. It's like a proof that basically over time, you're going to have to take control over distribution on the platform in order to prevent these kinds of issues and show people what they actually care about. So that's why we did it. And it actually showed a lot of value. I remember the early tests, I won't get into the numbers on them, but they were pretty unambiguous actually about this was showing more people that you cared about. It was improving your experience with the platform, it actually moved creation, which is unusual. It made people create more because they were seeing more content that was like accessible to them. But I also think that these things can go astray over time. And I won't say like the Instagram algorithm unequivocally bad or unequivocally good. But when we started to open up to more unconnected content and

• 28:22Speaker 02

  ad pressure was very strong, there's also a natural company incentive to optimize for just blind consumption. Because that's how you make money. So maybe cheaper content, or maybe just like get people to scroll more and more and more and more. And that also can encourage people to create less, because it's just like a more mindless scrolling mode.

• 28:41Speaker 03

  You guys have very concretely committed to doing things to prevent that kind of behavior.

• 28:46Speaker 02

  We have, yeah. We have, we have a lot of mitigations there in place, but I, I think what it really comes down to me is just like, what are we trying to do as a platform? And I think the magic of this technology is that everybody is a creator. And so we want this feed to be optimized for you to create, to inspire you to create. And that can be like, sometimes when you think of inspiration, you think of like, oh, it's this beautiful, crazy scene that's so elegant. When I think about that, I think about like a meme culture or something really funny or like, oh, that's cool. I've got a riff on that. And I think that's a very different brain mode when you're browsing the feed. And of course, we have lots of other things in place. So like, I think it starts with incentives. That's our incentive right here is to encourage more creation in the ecosystem.

• 29:31Speaker 02

  But there are certainly use cases we want to prevent. We're not going to get them right all the time. It's very challenging. It's a very living system. It's also very hard to write a recommender system when you have no data and you don't know what to recommend. You don't know how the platform is going to evolve. But that's like basically how I kind of think about the incentives of feed. And then Rohan, we have a lot of mitigations in place that I think you've been kind of like thinking about and maybe even more deeply than I have. About like preventing maybe the, the extreme cases. And so I don't know if you, you want to talk a little bit?

• 30:02Speaker 03

  Yeah, happy to. But one thing before you, I mean, just one thing to add is that the stated intent of like optimizing for creation is working really well.

• 30:10Speaker 02

  Yep.

• 30:10Speaker 03

  It's almost a hundred percent of people who like get past the invite code and all that on the app end up creating on day one. When they come back, it's like 70% of the time they come back, they're creating. And 30% of people are actually even posting to the feed. So not just like generating for themselves, they're actually like posting into the ecosystem, which is incredible testament to the model, how fun it is. And to like how, what we're optimizing for is actually working pretty, pretty well right now. But yeah, beyond that, I mean, like, one of the top of mind things is I think we don't want this just to be like a mindless scroll and beyond just optimizing for creation in the ranking algorithm, there are things we can do. Like trying to just get you out of this sort of flow state of just like consumption and push you into like creative mode. I think there's a great article on this called like the curvilinear

• 30:55Speaker 03

  nature of casinos where they design it so you never have to make any decisions. It's just like you walk in a circle. There's no windows, all that kind of stuff. We can be very intentional about not doing that and like, you know, whether it's an in-feed unit that's like, hey, you just kind of viewed a couple of videos in this domain. Why don't you try creating something? Or other ways to just kind of like push you out of that. We actually have things like that in the product. Yeah, those are some of the things that come to mind. Yeah.

• 31:21Speaker 04

  I really commend you guys for what you've done to, you know, make sure that there's a version of the world where video model as world simulator could have just ended up with us, you know, each retreating into our own computer screens and just becoming addicted and just retreating into ourselves. And I think the amounts to which you're, you know, prioritizing the the human element and the social element, I think that the care you've put into that really shows.

• 31:45Speaker 03

  I don't think we would have launched like a feed of just like AI content that wasn't, that didn't have a human feel like just, I don't think that excited us. And as soon as we we like had the product, we had cameo and we had that feeling internally. We were like, okay, this is actually a little different than.

• 32:00Speaker 02

  Yeah, I don't think it was totally obvious. Again, it was like a pretty crazy sprint to go through this. It wasn't like super obvious to us what would emerge, but I think that the idea, it makes sense in retrospect, but it was a completely not obvious product decision. The cameos would be the thing.

• 32:17Speaker 03

  Yeah.

• 32:18Speaker 02

  Where it's like, of course, you just want to see your friends doing cool things. So it's like, that makes sense. But I was never actually that afraid of competitive pressure in that. Crazy product phase because I was like, we sort of had these, all these non-trivial decisions that are obvious in retrospect, but were not obvious at the time that we're sort of building on top of each other. It's like, okay, cameos. Well, there's also a version of cameo where you have a crazy flow that's just for you and it's a one-player mode cameo and you like go through this onboarding flow and do your stuff. But we were already seeing these interesting dynamics where it's like, oh, I could tag Ron in my video. That's crazy. Like, and then we can have like an argument or like, I'm gonna have anime fight, doesn't matter.

• 32:56Speaker 02

  And I was like, okay, so that's, that's actually the human element. That's the, that's the magic of this. It's actually strangely more social than a lot of social networks, even though it's all AI-generated content. It's very unintuitive.

• 33:07Speaker 04

  Totally. Is it a separate, is it fine-tuned version of Sora 2 or is it like, is it a separate model from what's available over the API or is it the same?

• 33:15Speaker 00

  Between the app and products, we're currently exposing like the models in the same state across API and, and the app.

• 33:22Speaker 04

  Okay, really interesting. What are you saying people do on the API side and is it different from the types of things people are doing on the, on the consumer app?

• 33:30Speaker 03

  The motivation behind even launching an API is just like support of these long-tail use cases. Like, we have this vision of enabling, you know, chat GPT scale level consumer audience with this tech, but there's tons of very niche things out there. You can imagine people who are much, you know, with Sora 1, we went out and talked to a lot of these studios. What we heard from them is like, they want to integrate this in this specific part of their stack in this specific way. And we'd love to support all these long-tail use cases, but we don't want to build a thousand different kind of interfaces for this stuff. So that's the kind of stuff we're excited to see with the API. So far, it's been, you know, it's been kind, those kind of like, a little bit more of a niche company, not trying to build like a first-party social app, but maybe,

• 34:12Speaker 03

  you know, has some either filmmaking kind of audience or kind of people they're supporting, or even just like, we've definitely, we've seen some like people trying to, you know, I think there was like a, some company making, they were doing something with CAD where they were like using Sora. Yeah, yeah, yeah. So there's, there's, there's cool use cases out there. I think we're still getting a sense of what they are.

• 34:34Speaker 02

  Yeah, I, I think there's a lot that can be done with these things. I think about gaming all the time, just based on my background and I'm, you know, AI in gaming is always a very controversial subject, but it's very clear that there's a, there's a place and there's a role, maybe doesn't. Have to interrupt the creative process, can enhance it. And I'm pretty excited to see some of those use cases emerge.

• 34:53Speaker 04

  Do you think the video models are good enough now to, for people to be able to build video games on top of the API? Or do you think we're still another rev or two away?

• 35:01Speaker 02

  I have my own take on this.

• 35:02Speaker 03

  I was gonna say, never bet against the ways people can be creative with technology to build. Like, someone will be able to build a game and maybe has built a game already. Will it feel, look and feel like a, you know, obviously there's latency with this model. So you'd have to do all sorts of crazy stuff to get around that, but.

• 35:17Speaker 02

  Like, I think that your mind immediately goes to like, kind of the obvious sort of things that you would do in gaming. And we've seen some of that sort of stuff, certainly in research blogs and that kind of thing. My mind often goes to like, okay, this is like a creative tool that's a little bit different. And the types of games that really excite me there, I'll just go off on one, which is this, like, there's a game called Infinite Craft, which is the world's simplest game. It's a web game where you just take elements. It's like fire, water, earth. You have like four elements to start and you just drag them and it. I love this game. It combines into something new. And the thing it combines with is like, it's LLM based. So it's like fire and, and, and earth might be a volcano.

• 35:57Speaker 02

  And then volcano plus water might be an underwater volcano or Godzilla or something like that. You always end up in Godzilla for some reason. But that's a game that like, it's like, oh, it kind of makes sense where it's like, yeah, you don't really need a crafting tree. The LLM can derive this crafting tree and it's a process of discovery. And so I think there's a lot of untapped stuff in that space where, again, I like the idea of a process of discovery. In fact, my philosophical, philosophical view on LLMs and video models to some extent is that it is a process of discovery. These are all in the weights. You're just unlocking it with like a secret code, which is your prompt. And I, I love that. That is, that is very magical. That was always in gaming.

• 36:40Speaker 02

  That was the thing that like excited me the most was discovering something new. Especially if it was a true discovery, it wasn't put there by somebody else. Maybe they just enabled the mechanics around it. So I think there's a huge opportunity in that space of of gaming. When you think about games and just a different thing and like embrace this technology in a very different way.

• 36:58Speaker 04

  It reminds me of how some of the earliest use cases for GPT three were the kind of these text games. So it's different from how you think of a, you know, video, like a playable video game, but actually a lot of these mechanics are very game like.

• 37:10Speaker 02

  Exactly, yeah. I think there's a there's still constraints and I think that's going to be like the mechanism design. That's still very human. Like a lot of the early games with G P three, they're kind of like, yeah, it was fun for a minute and then it kind of went off the rails. And you're like, I don't really know what I'm doing anymore. But again, like this is sort of in some ways, so sort of feels like a little bit of that where it's got a little bit of gaming. D N A inside of it where it feels very fun and different and exploratory. So I like things like that. And I think there's going to be more use cases that we can't even think of. Too creative.

• 37:40Speaker 04

  What are you guys seeing on the creative filmmaking side? Like, is that an important target market? Do you want to do an empower the long tail or do you want to empower the the head, so to speak, of the creative market?

• 37:50Speaker 00

  It's a really good question. You know, we've benefited a lot from creatives who are really willing to like go all in on, you know, even like the early technology like Dolly one, Dolly two, and really like help steer us along the path. And like, I think it's important that we continue to, you know, build things for for those folks. And we are working on some things that are like more targeted towards like creative power users long term. At the same time, you know, I do think like A I is a very democratizing tool right at its best. And so what's kind of beautiful about the Sora platform in general, right? Is whenever someone kind of strikes gold, right? You see one of these like beautiful anime prompts that like goes to like the very top of the feed for everyone. Like anybody can go and remix that, right?

• 38:34Speaker 00

  Everyone has the power to like build on top of that and like learn from all of these people who come in with this like incredible knowledge about how to like really get the most out of these tools. And so I am really excited just to see the net creativity of humanity just increase as a result of this. But I think a big part of that, right, is continuing to empower people who are always at the frontier, which are these like. More pro oriented like creator type folks and so we want to keep investing in them as well.

• 39:00Speaker 01

  We've nerded out for a while, like almost a couple of years now about that vision of feature film.

• 39:07Speaker 00

  Yeah.

• 39:07Speaker 01

  Length content. Like, yes, you have these amazing cameos and shorter content, but at some point, the individual creator. It's been something that you've been excited about for a very long time.

• 39:17Speaker 00

  Yeah.

• 39:18Speaker 01

  When do we get there? You know, is there a point where we have a feature film that is created on Sora 2?

• 39:23Speaker 00

  Yeah.

• 39:24Speaker 01

  And how do we consume it? Is it in the Sora app? Is it posted somewhere else online? Do you go to a movie theater and watch it?

• 39:31Speaker 00

  Yeah, it's a great question. I mean, I think this will happen in stages to some extent. So like, if you guys watch the, right, the launch video, I mean, that was made by like Daniel Fraden, who's on the Sora team, and he already with these tools, right, is able to pump out these like incredibly compelling short stories within like days at most. I mean, he literally made that like all by himself in almost no time. And he's been like continuing to like put new ones out there on like the OpenAI Twitter since. Uh, so like clearly this is like massively compressing the latency. Uh, that's associated with like filmmaking. I think to get to the point where like really anybody can do this, right? Like any kid in their home can just like fire up

• 40:11Speaker 00

  the app or sora.com or something and go and make this. You know, it's really like a, an economics problem of like the video models. Video is the most intensive, compute intensive modality to work with. It's extremely expensive. And, you know, we're making good progress on the research team, like really continuing to figure out ways to make this affordable for everyone long term. Like right now, for example, you know, the Sora app is like totally free. Um, in the future, there will probably be ways where people can pay money to get more access to the models, um, just because that's the only way we can really scale this further. But, uh, you know, I think we are not far off from this world where anybody can really like have the tools to make amazing content. You know, I think there's gonna be like a lot of bad movies that get created by this, but like likewise, you know, there's probably

• 40:50Speaker 00

  the next great film director who is just kind of like sitting, uh, you know, in their parents' house, like still in high school or something and just like has not had the investment or the tools to be able to like really see their vision come to life. And we're gonna find like absolutely like amazing things from like giving this technology to like the whole world.

• 41:06Speaker 04

  I'm looking forward to the feature film length, Constantine's Greek Odyssey.

• 41:09Speaker 00

  Yeah, me too.

• 41:10Speaker 04

  Coming to theaters near you.

• 41:12Speaker 01

  We're all in it together, actually. Different characters. I play the Cyclops. It's a, it's a good one.

• 41:18Speaker 02

  I think, um, just to touch on that one more thing, the, something I've learned from recommender systems over and over again is that like, Oftentimes, so the tools, getting more people more creative is going to be a huge unlock for just, you know, making people more more creative in general. And because you don't need this access to this like filmmaking equipment, all that sort of stuff. But we do consistently see that things, content is like also a social phenomenon in a way. And like movies and all that, all, all, everything you see out there is kind of a bit of a social phenomenon in addition to the actual content itself. And so I think we're going to enter a very interesting world where, you know, there's so many people creating and so much content out there that even the idea that people aren't paying attention to and watching it is going to become more and more important.

• 42:04Speaker 02

  And I think that's actually going to make the quality of content just to kind of elevate. Because there's this, anybody can create. And actually, it's going to be the consumption that's going to be quite limited, which is very different than the world we live in today.

• 42:16Speaker 04

  You guys are very thoughtful and intentional about how you treat IP holders. Can you say a word on that?

• 42:21Speaker 00

  You know, we've been in close partnership with like a bunch of folks across the industry and like really trying to like both show them kind of this like new technology, right? That is actually like a huge value proposition for rights holders across the board, right? And like, we're hearing so much excitement from the folks we're talking with. Like, they really see this as being like, you know, a new frontier for, again, like, you know, every kid in the world having the ability to like go and like use like some of this beloved IP and like really like bring it into their lives in a way that feels much more personal and custom than what's been possible before.

• 42:55Speaker 04

  Yep.

• 42:55Speaker 00

  At the same time, you know, we really want to make sure that we're doing this like in the right way. So we've been like really trying to take feedback and like really steer our roadmap in a way where we know that, you know, both users are going to have an awesome experience getting to use this IP, but also the rights holders are going to get, you know, properly monetized and rewarded in a way that, you know, everyone wins, basically. So we're right now actively working on trying to scope out the exact details about like how we're going to, you know, for example, make it so if you want to cameo your favorite character from some like beloved film or something, you can do that in a way where you have access to it, but like monetization will flow back to the rights holder, right? So really trying to figure out this kind of like new economy for creators. We kind of just have to create this from scratch right now.

• 43:38Speaker 00

  There's a lot of deep questions about how to do this the right way. And, you know, as with everything with this app, we come into it with like an open mind and we hear feedback and we iterate quickly. You know, we're not sure where this is going to totally converge, but we're working closely with people to figure it out.

• 43:52Speaker 04

  Really cool. What's ahead?

• 43:56Speaker 00

  Pets.

• 43:57Speaker 03

  Yeah, I think, I mean, one, one.

• 43:59Speaker 04

  Sorry, what?

• 44:03Speaker 04

  Is that one of the most demanded features?

• 44:04Speaker 01

  Break, break.

• 44:06Speaker 02

  Bill's demanding.

• 44:08Speaker 01

  I will remind us, we were just talking about curing diseases and world models. And now we're to the future.

• 44:14Speaker 02

  Uh, this is something, um, no, it's actually, so that's definitely true. We've committed to that. It's coming. But we, I promise, the, uh, we actually had Bill's dog as like, when we were playing around with this.

• 44:26Speaker 00

  Rocket, the goodest boy.

• 44:27Speaker 02

  Yeah, uh, and actually it was very, very cool, um, to actually feature a pet. You can imagine where that goes. It doesn't have to necessarily be a pet. Could be anything, uh, clock or whatever you have.

• 44:39Speaker 03

  Clock?

• 44:40Speaker 02

  Well, yeah, yeah, yeah.

• 44:42Speaker 03

  Do you have a special clock?

• 44:43Speaker 00

  Actually, it's really compelling. I didn't think it could be so compelling until Thomas showed me this clock. It's like a sentient clock. But it's, it's like based on like a, a real clock.

• 44:51Speaker 02

  Yeah, I, I had a clock. My father, it was, my father was a technology person for a while. He, it's the company Veritas gave him a clock for his like, whatever anniversary. Anyway, so I have it on my, my, uh, my, uh, like table somewhere. And, uh, there's this old Simpsons episode where they talk about a walking clock. And for some reason that's just been an earworm in my head for the last 30 years. And so I always, it's like, you know, they're, they're telling some joke and it's like, is it a walking clock? It's a walking clock. It's like, walking clock. And then it's like, no, man, it's my dog. And so it connected in my brain where I was like, okay, Rocket, walking clock. And then, so I tried it.

• 45:27Speaker 03

  Thomas is the deep floor.

• 45:32Speaker 02

  Yeah. So, uh, it connected my brain and we've been playing around with this, um, just to see if we can get it to work and whether there's something special there. Which is part of the fun of being on the sword team is you get to play with this emergent crazy technology and like, maybe it does something. You wouldn't even have expected. So I recorded a two second video of my clock. And then I gave it some cameo instructions and I said, you're just a walking clock. You're a walk clock. You talk like you talk, your character. And then I generated my first video and it was insane. It was crazy. It was a walk clock. And then I had one where it was talking to Bill. And Bill was like, I didn't think it would ever land the pet cameo feature. And the walking clock's like, here I am, you know, I just landed. So it's coming.

• 46:17Speaker 00

  It's all internal means.

• 46:18Speaker 04

  Talk about emergent IP.

• 46:19Speaker 00

  Yeah.

• 46:20Speaker 04

  Who needs Pokemon when you can have a walking clock?

• 46:23Speaker 02

  What's the greatest IP?

• 46:24Speaker 03

  One thing to add in terms of the future, I think on the feature film question, sometimes something I think about all the time is like, what? You know, what will that actually look like? I think my, I mean, caveat, Bill's the only one who's who's good at predicting the future here.

• 46:37Speaker 03

  But my sense is that the, you know, as we get to longer forms, what our equivalent of a feature film will look and feel very, very different from what a feature film is today. You know, I don't know exactly what that looks like, but I think on the subject of creators and what what's coming in the world. I think a new medium and a new class of creators, new class could include a lot of existing creators and and support existing sort of mediums and stuff like that. But I think we're just in the early innings of of what I imagine will be the next film industry, rather than thinking about this being a feature film. But I think there'll be something new. There's some anecdote. I hope this is true because I say it all the time. But apparently when the recording camera, like, you know, hit the world, the first thing people did was record plays.

• 47:23Speaker 03

  This is like the least interesting thing you could do with a recording camera. It's like, what's the big idea? Oh, we people don't have to travel around acting. We can just film them and distribute it. And then someone was like, wait a minute. We can make a film and film in all these different areas. And I feel like we haven't, we're in like the first inning of so many different sort of things that people will do with this technology, especially as the constraints change with latency and length and all that kind of stuff.

• 47:46Speaker 01

  So cool and fun film history nerd fact is one of the original videos, and we should check this as well, but I think the original video was made just down the peninsula to settle a bet on if a horse, when it galloped all four legs, it left the ground. And I could see a world where you have new, that is an example of new scientific discovery. People didn't actually have an answer to that. Now that you have a new simulation format, what are we going to be able to discover in that?

• 48:18Speaker 00

  It will be crazy. And I think one, one broader point here is, you know, this app right now feels very familiar in a lot of ways, right? It's like a social media network at its core. But fundamentally, like the way that like we really view it internally, right? Is with Cameo, we've kind of introduced the lowest bandwidth way to give information to Sora about yourself, right? Aspects about your appearance, about your voice, et cetera. You can imagine over time that like that bandwidth will greatly increase, right? So the model deeply understands your relationships with other people. It understands, you know, more than just how you look on any given day. It's, you know, seeing your full, like how you've grown up, all of these details about yourself. And we'll really be able to almost function as like a digital clone, right?

• 49:03Speaker 00

  It's like, there's really a world where the Sora app almost becomes this like mini alternate reality that's running on your phone. You have versions of yourself that can go off and interact with other people's digital clones. You can do knowledge work. It's not just for entertainment, right? And it really involves more into a platform, which is really aligned with kind of where these like world simulation capabilities are headed long term. And I think when that happens, the kind of immersion things we will see are crazy. And, you know, for OpenAI across the board, it's really important that we kind of like iteratively deploy technology in a way where we're not just like dropping bombshells on the world. When there's like some big research breakthrough, we want to co-evolve society with the technology. And so that's why we really thought it was important to like do this now and like do in a way where, you know, we've hit this, again, this kind of like GPT 3.5 moment for video.

• 49:46Speaker 00

  Let's make sure the world is kind of aware of what's possible now. And also, you know, start to get society comfortable and like figuring out the rules of the road for this kind of like longer term vision. For where, again, there are just copies of yourself running around in Sora and the ether, like just doing tasks and like reporting back in the physical world. Because that is where we are headed long term.

• 50:06Speaker 01

  So cool.

• 50:07Speaker 04

  So you're building the multiverse.

• 50:09Speaker 00

  Actually, kind of, yeah.

• 50:10Speaker 04

  Okay, well, can, can Simi and me go and find my soulmate somewhere in there?

• 50:14Speaker 00

  I mean, anything is possible in the multiverse.

• 50:18Speaker 00

  That's a call for action, everyone.

• 50:20Speaker 04

  It is kind of crazy though, because now I'm going to sound totally cuckoo, but if we're in a computed, you know, environment, you're building the perfect simulator. That kind of is the way you ultimately understand and break out of the computer environment, right? Like, are we getting closer to the heart of the matrix?

• 50:36Speaker 00

  There's some very

• 50:36Speaker 04

  We're in the matrix?

• 50:37Speaker 00

  Some very deep existential questions. Yeah, yeah. What's your guys' P of we're simulated? Like, this is all

• 50:44Speaker 04

  Oh, rising.

• 50:46Speaker 00

  Yeah, me too.

• 50:47Speaker 01

  Oh, I'm low. I'm, yeah.

• 50:50Speaker 00

  Oh, man.

• 50:50Speaker 01

  But, yeah, it's okay.

• 50:51Speaker 00

  You're really, uh, okay. I respect that.

• 50:53Speaker 01

  I'm just like, you know what? This has got to be real.

• 50:56Speaker 00

  Yeah. I feel like I'm at, like, solid 60%. I don't know. Like, more likely than not at this point.

• 51:02Speaker 04

  I'm there, too.

• 51:03Speaker 01

  Whoa. Yeah. Zero.

• 51:06Speaker 04

  Should we make a calcio on it?

• 51:08Speaker 01

  Yeah, trivially small. How do you settle that? What's the oracle? Sora 10 will answer.

• 51:15Speaker 00

  Yeah, Sora 10, yeah.

• 51:17Speaker 01

  Yeah.

• 51:18Speaker 04

  What do you think are the theoretical limits to Sora?

• 51:21Speaker 00

  Yeah, it's actually a great question. I thought a little bit about this. Like, I think there's like a question, can you eventually, like, simulate, like, a GPU cluster rate in Sora or something? And I assume there are some very well-defined limits on, like, the amount of computation you can run within one of these systems, like, given the amount of compute you're actually running it on. I've not, like, thought deeply enough about this, but I think there are some, like, there's some, like, existential questions there that need to get resolved. Yeah, yeah.

• 51:53Speaker 04

  See, that's why his P in him is so high.

• 51:55Speaker 01

  Fascinating.

• 51:57Speaker 04

  Wow.

• 51:57Speaker 01

  Got a few lightning round questions for the team that we just kind of generated on the fly here.

• 52:03Speaker 01

  And take your time. Jump in whenever you have an answer. Your favorite cameo on Sora to date? And what happened? That is so tough.

• 52:13Speaker 02

  I have a hot one.

• 52:14Speaker 01

  Go, go, go.

• 52:17Speaker 04

  Shit.

• 52:17Speaker 02

  Yeah, okay, so the, there was this TikTok trend of I, and I got obsessed with them. I don't know why, but these Chinese factory tours, where they're like, hello, I'm the chili, this is the chili factory. They get, like, one like, and it's me. And it's like, they're showing their chili factory, and they're like, it's the chili factory. Like, this is amazing. And, or like, there's an industrial, uh, chemical one, is it?

• 52:42Speaker 02

  Yeah, I've lost the name, but there's an industrial chemical factory. And the first day, I had my cameo options open just because I was like, I just want to see what happens. And the first day, late at night, I opened my cameos and I was starting to get tagged in factory tour cameos that were all in Chinese. And I was like, I'm in the Chili Factory. And I was so excited. I got zero likes. I liked it. It was just me. But I was like, I'm the Chili Factory guy now. I'm like doing the ribbon cutting at the Chili Factory. Amazing. That's too deep of a cut, though.

• 53:23Speaker 01

  Congratulations. Fun fact, I actually have done Chinese factory tours.

• 53:27Speaker 02

  Amazing.

• 53:27Speaker 01

  In real life, and they are truly epic.

• 53:29Speaker 02

  Yeah.

• 53:30Speaker 03

  There's this one, just I saw Mark Cuban and jorts dancing around that.

• 53:34Speaker 00

  That was pretty good.

• 53:35Speaker 03

  That got me. But I mean, my more back to the like, just I scrolling the latest feed and just seeing like the wholesome content of people like doing things with their friends. Actually, I think what brings me the most joy of they're not like super liked, but it's like people just like getting a lot of, you know, value, obviously, from just like making videos with their friends. So.

• 53:55Speaker 00

  Sam has so many bangers. I like the one of him doing like this K-pop dance routine about like GPUs or something. It's very good. Actually, I would put it on my Spotify if like we had the full song.

• 54:06Speaker 03

  Wow.

• 54:07Speaker 00

  It's very good. It's like generated by Sora. It's like, like very compelling. Yeah.

• 54:11Speaker 01

  All right. Well, that leads to the next one, because you mentioned Spotify. What does an AI, fully generated AI, win first? Oscar, Grammy, Emmy?

• 54:23Speaker 03

  I think the like logical answer is like a short winning an Oscar. Yeah.

• 54:28Speaker 02

  I think that's probably right.

• 54:29Speaker 03

  Yeah.

• 54:30Speaker 00

  What would we win it for? Like for like.

• 54:32Speaker 01

  The jorts.

• 54:33Speaker 00

  Yeah. It's gonna be the jorts.

• 54:35Speaker 03

  The jorts trilogy.

• 54:36Speaker 00

  Yeah.

• 54:36Speaker 03

  Yeah.

• 54:37Speaker 00

  We need new content. Yeah.

• 54:39Speaker 02

  I do think if people stitch things together in interesting ways.

• 54:41Speaker 03

  Yeah.

• 54:42Speaker 02

  Yeah, I think there's a you can actually start to make some very compelling storytelling in that. And I don't think it's like, it doesn't really feel like AI anymore. The content I'm seeing like that was actually something I noticed with Sora as well, just like throwing. It wasn't even noticing it was AI. It was just kind of interesting content.

• 54:58Speaker 03

  That's a more interesting question. What will we know?

• 55:00Speaker 02

  Oh, yeah.

• 55:02Speaker 03

  Maybe it's already happened.

• 55:03Speaker 02

  Maybe it's already happened.

• 55:07Speaker 01

  I feel like for Oscars, one of the cool things that'll be unlocked is this long tale of epic stories in history, stories of heroism and struggle and all of these things that have been locked up because of the cost of creating. You know, as a, as a history enthusiast, I cannot wait for AI to unlock all of those stories.

• 55:30Speaker 04

  Have you seen the Bible video app?

• 55:32Speaker 01

  No, I haven't.

• 55:32Speaker 04

  Oh, it's really good. I'll show it to you after.

• 55:34Speaker 01

  Like, like, perfect example.

• 55:35Speaker 04

  Yeah.

• 55:36Speaker 01

  Or there's this movie, The Last Duel, a few years ago about this, this really terrible crime that was committed in medieval France that was historically relevant and, you know, basically says a lot about humanity. And it just got picked up because eventually Hollywood picked up this important story about humanity. But how many more are there in human history? That's gonna be really cool. Favorite character from any film or TV show?

• 56:02Speaker 03

  I have a really random one.

• 56:04Speaker 01

  Go for it.

• 56:06Speaker 03

  You guys seen Madagascar? King Julien?

• 56:09Speaker 01

  Oh.

• 56:09Speaker 03

  Played by Sacha, Sacha Baron Cohen. He's a lemur. He's a lemur kid.

• 56:13Speaker 01

  Absolutely.

• 56:14Speaker 03

  It's just like.

• 56:14Speaker 01

  It's a banger.

• 56:15Speaker 03

  It's, his humor meets kid-friendly storytelling. It's just, it's perfect.

• 56:20Speaker 02

  I play a lot of video games, so I mean, your classic answer is gonna be like Mario or something like that. Although, I'll do the deeper cut of, we were always joking about Parappa the Rappa.

• 56:28Speaker 03

  Yeah.

• 56:29Speaker 02

  Yeah, Parappa the Rappa, like old, old PlayStation game, one of the original rhythm games. And it's got a great artistic style and it's got a great IP of just this little dog.

• 56:37Speaker 03

  What, what is he? A dog?

• 56:38Speaker 02

  He's a dog, yeah, yeah.

• 56:40Speaker 00

  That's a good pick. When I was a kid, I played the like Pokemon trading card game competitively for a while. So I was like really in like the Pokemon rabbit hole. So like, I don't know.

• 56:51Speaker 03

  Snow White.

• 56:52Speaker 00

  Pikachu.

• 56:54Speaker 03

  Pikachu.

• 56:55Speaker 00

  Mudkip.

• 56:57Speaker 01

  Super non-consensus.

• 57:00Speaker 01

  A fringe deep cut.

• 57:03Speaker 01

  Okay, first world model scientific discovery. Most specific possible. Obviously, you're not gonna save the discovery.

• 57:11Speaker 00

  I suspect it will be something related to like classical physics, like a better theory of turbulence or something. That would be my guess.

• 57:19Speaker 02

  I was guessing that it was gonna be something like that. I was like, that'd be a Stokes. I don't know. Yeah, some fluid dynamics thing that's maybe hard to understand. Now, there's a lot of like unsolved kind of problems. I think sometimes they call it like continuum mechanics where it's like in between and we don't have good models of them.

• 57:35Speaker 03

  Something that lends itself to simulation, just like the amount of iterations you can do of a simulation unlocking something, which I don't, yeah, something in that realm.

• 57:44Speaker 01

  The last thing we'll be able to accurately simulate.

• 57:49Speaker 00

  I do think there's like a set of physical phenomenon for which video data is like a poor choice of representation, right? So like, for example, is it really efficient to learn about, you know, like high-speed particle collisions or something from like video footage? Maybe. I really think video is at its best when, you know, the phenomenon that you're trying to learn about is just natively represented in the physical world. And so when you, when you need to do like, you know, like quantum mechanics or some other discipline where, you know, it's more theoretical, we don't have like video footage beyond.

• 58:28Speaker 01

  Can't see it.

• 58:28Speaker 00

  Yeah, things that we've like manually rendered for like educational purposes. It feels like a weaker medium for understanding those things. So I suspect those would come last.

• 58:37Speaker 01

  I guess it's the things we don't have sensors for.

• 58:39Speaker 00

  Right, right.

• 58:40Speaker 03

  Yeah, maybe the last things we care to simulate is another way of thinking about the answer. I don't know. I mean, people aren't doing much with smell right now.

• 58:48Speaker 00

  True. Greenfields.

• 58:50Speaker 01

  I've been meaning to tell you about that. It's kind of awkward.

• 58:54Speaker 00

  We're still trying to figure out how to simulate Thomas with bad hair.

• 58:57Speaker 02

  Oh, yeah.

• 58:57Speaker 00

  It remains an unsolved problem. Not even Sora can do it.

• 59:00Speaker 01

  Thomas's hair flow, just general.

• 59:02Speaker 00

  Guzzling catch-up, yes.

• 59:03Speaker 02

  There was a, there's a good round of people being bald. We were all doing bald.

• 59:07Speaker 03

  Oh, yeah. Bald gems were good.

• 59:09Speaker 02

  Actually, kind of cool. That's, that's a use case that doesn't, I don't really talk about very much, but it's like.

• 59:14Speaker 03

  Visualization.

• 59:15Speaker 02

  When you're bald, yeah, everybody wants to be bald. No, it's just like, you just see yourself in some different context. I think that can be quite powerful, even like therapeutic in some ways where you just like see yourself in some context that you either want or don't want yourself to kind of be in and just see, see yourself.

• 59:29Speaker 03

  It's a real use case.

• 59:29Speaker 02

  Yeah.

• 59:30Speaker 00

  Yeah.

• 59:31Speaker 01

  Guys, thank you so much for coming. From spacetime tokens to object permanence, world models that will enable scientific discovery, the democratization of creation, all the way to walking clocks. You guys have covered it all. Thank you so much, and the future is being created by you. Thanks, Konstantin. Thank you.

• 59:53Speaker 03

  Thank you.