The Next Frontier of AI Video Is Control

a16z General Partner Jennifer Li sits down with fal co-founder Gorkem Yurtseven and Head of Engineering Batuhan Taskaya to discuss what changes when generative video becomes fast enough to run in real time. They unpack the technical work behind H3 Max, fal’s post-trained version of MiniMax’s open-weight video model, and how combining model post-training with systems and hardware optimization significantly reduced generation time while maintaining quality. That speed has enabled experiments with continuous video, including streams that can remember previous scenes and respond to new directions while they’re running. They also discuss why the next challenge may be less about speed and more about control, from camera movement and lighting to characters, motion, and lip sync. And they explore what those capabilities could mean for professional creative workflows, where artists and studios need predictable tools rather than simply generating a video from a prompt.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Generative media is along with the coding agent market, what we call is token market fit. Everyone is waiting for a large consumer moment in AI. I believe H3 Max makes it possible.

• 0:14Jennifer Li?

  Were you surprised by the speed up and the gain you could get from post-training this model?

• 0:20Speaker 02

  We have a version called H3 Max Turbo that's public, that can generate like a five-second video in like 1.5 seconds. From a cost standpoint, it's also like 2X less.

• 0:28Speaker 00

  People starting creating these beautiful scenes using an LLM model, GPT-Astra in Blender, and all of a sudden it unlocked the whole new workflow for Hollywood and professional people.

• 0:42Speaker 02

  We have been very, very focused towards speed, performance, quality, and now we have a really good base model. The next month or two is gonna be fully focused on.

• 0:50Erik Torenberg?

  What happens when AI video becomes fast enough to generate in real time? A16Z general partner Jennifer Lee sits down with FAL co-founder Gorka Mirzehven and head of engineering Bhutan Tashkaya to discuss H3 Max and the rapidly changing generative video stack. They unpack how post-training and systems optimization made video generation significantly faster, opening up new experiences where video can run continuously, remember previous scenes, and respond to direction as it plays. But speed is only part of the story. They also discuss the push toward greater control over camera angles, lighting, characters, and motion, and why those tools could make generative video more useful for professional creative workflows.

• 1:35Jennifer Li?

  Welcome, Gorka, Bhutan, to our podcast again. We did the last one last year. This is long overdue, and we have such an exciting model to talk about, which is FAL's H3 Max. The day when it came out, I was calling it, "It's really in the league of its own." Like, it's so funny to see the benchmarks where you have the dot of this model on the far left or far right, and then everything else is on the other half.

• 1:57Speaker 00

  And that graph is actually log scale. It's actually further, but we had to fit it in. We had to do log scale.

• 2:04Jennifer Li?

  That is hilarious.

• 2:05Speaker 00

  The time portion, the quality is not, yeah.

• 2:08Jennifer Li?

  For sure. The internet noticed, for sure. There are so many viral tweets about it, like people really played around with this model. Maybe just give us the backstory of what inspired you to post-train this open-weight model from Minimax? And how did you get the quality and speed to where it is?

• 2:25Speaker 00

  First of all, the Minimax H3 model is the first truly open-source, very capable, like latest generation video model out there. So even though we work with some of the other model labs to run inference for them, we never had this capability, like had the right to add this capability on top of it. So when Minimax came up with their very capable open-source model that is truly last generation, can take references, like very familiar architecture to any other video model, we thought this is a great opportunity to go all in and see what we can do. And again, we did like many different things that we are gonna talk about that combined gave the results that you show on the graphs.

• 3:10Speaker 00

  But the biggest reason why everything came together for this particular moment was because H3 was the first truly next-generation video model that's open source.

• 3:21Jennifer Li?

  What is the idea, given like FAL has been known to be like a generative media inference-serving platform, like what is the idea to get into post-training open-weight model? Like, you talked quite a bit about it in the blog of like combining the system work with the model itself. Maybe talk more about the work behind that.

• 3:37Speaker 00

  Like, generative media is, I would say, along with the coding agent market, what we call is token market fit. And the way we define it is as can a single person productively spend a lot of tokens? And the amount is like 10K a month, something like that. So there is incredible amount of demand in the market to generate video, to generate many things at the same time. And a person who is doing this for their daily job, they spend in front of a computer and do this all day long, and they spend thousands of dollars, lots of tokens. And since around April, the whole industry and FAL itself, we've been compute constrained.

• 4:21Speaker 00

  We are growing as much as we are adding compute. Like, there are things we do here and there, but the whole industry has been compute constrained. And we've always been looking for efficiencies where we can relieve that a little bit so people can use this more. So that has been the idea behind everything we've been doing since April. And this just came at the right time because this makes everything maybe an order of magnitude more efficient. So it gives more compute for other models or even like more tokens can be generated using H3 Max. I think Bhutan would agree on that.

• 4:59Speaker 02

  Yeah, like just from a system-wide optimizations, which is what we have been doing for the past three, four years, you can maybe make the model 2X, 3X faster while producing the same quality, right? Because it's at the end of the day, same model, same architecture, you have the same constraints, you're just trying to optimize what you can get out of the chip itself. And there is a roof line there. And we have been approaching that roof line more and more, especially like lately because our entire team has been focusing on how do we get out more video pixels from a single chip as much as possible. And this new set of like post-training related optimizations with system slash model co-design enables us to go beyond that roof line by an order of magnitude. And like, we just felt the pressure.

• 5:44Speaker 02

  We have been working on it on top of open-source image models before the video models. We did one version with Ideogram, we did one version with Flux. So we have been like experimenting with how can we build post-training infrastructure to take an existing model, build kernels and systems design around that to run it very, very fast for a specialized version that can beat anything else that we would get just by running the model itself. And you know, combination of that plus just like getting a frontier video model on our hands and all this expertise, we were able to go by like an order of magnitude in terms of speed.

• 6:19Jennifer Li?

  Incredible. Let's dig into that. I may get some of the numbers wrong, but...

• 6:22Speaker 00

  There's like efficiency numbers, there's cost numbers, there's speed up numbers.

• 6:26Jennifer Li?

  Right.

• 6:27Speaker 00

  Not everything means efficiency, but it all adds up to be very efficient, yeah.

• 6:31Jennifer Li?

  Yeah, I guess what is stunning to me is there is like a magnitude of lower cost and also much faster, I think it was like 35X speed up.

• 6:40Speaker 02

  Yeah, compared to the original Minimax H3 endpoint.

• 6:42Jennifer Li?

  Well, at the Elo score, you didn't really sacrifice quality. So yeah, just maybe reveal a little more of the secret sauce behind of, is this more of the type of system work you have done? Like, did you have to do like model architecture change? Is it system work that really brought down the cost and latency? And how about like the next generation of chips, like GB200 fits into the whole story?

• 7:05Speaker 02

  It's just a compounding effect of like multiple different optimization variables that we have been targeting. The first one is obviously, okay, you go from like the base model to a model that's like post-trained to be like more efficient. For diffusion models, this is just essentially, how do you go from running 50 steps to running something like 20 steps, right? Like, you're just like trying to optimize that pipeline. But as soon as you go from 50 steps to 20 steps, you lose quality. So you need to target in the optimization scene, "Okay, I wanna improve the quality and then I wanna apply the optimization." So we have like checkpoints of this that are significantly higher quality, but obviously slower. So what we initially did was, okay, let's run our post-training and our L pipelines so that we can improve the model's quality and then apply the optimization stack on top of it so that the end result gets you to the same quality

• 7:50Speaker 02

  or like even like higher quality than the original model, but at the same time, you're like an order of magnitude faster. So most of the gains come from post-training this model to be like compatible, that you can run this on like less amount of steps. But on top of that, you add like all the kernels and systems engineering work that you do that brings your like hardware utilization from 30, 40%, which is like standard in like many inference workloads, to like 70, 80%. And 70, 80% on theoretical MFU, which is like impossible to reach. So you're essentially at the roof line of what you can get out. And then these models are not just like a single, oh, you just give a prompt and you get a video back. They're actually pipelines underneath. You need to take a prompt, you need to like run an LLM, like a very large LLM

• 8:32Speaker 02

  to go expand that prompt to a format that the model was initially trained at, generate the video in the latent space, and then decode those latents back into pixels. And then depending on the workload, there might be an upscaling component involved. So there's like multiple components, and every single component by default is unoptimized. There's still like lots to be gained there. And like, we just looked at it from a perspective of we are gonna get the maximum out of every single component. This made us go around LLMs at super high speeds, right? There is that component, but for a different workload. This is not like something like an agent decoding LLM workload where you have very high cache rates, where you have higher sessions. It's a single shot, give a prompt, you get a prompt back, and there's no caching. You're operating at low batch sizes. So there's a completely different set of optimizations on the prompt expansion side, completely different set of optimizations on the diffusion model,

• 9:17Speaker 02

  completely different set of optimizations on the VAE that you take from latent to pixels. And you just combined all of these to have an effect that compounds. From a hardware standpoint, going from something like Hoppers to Black Valves, you see something like 2 to 3X improvement by itself. But from a cost standpoint, it's like pretty comparable because the cost is also like in that league. So I would say it only reduces your like wall clock time, but not just the efficiency itself. But it obviously helps if you wanna go significantly beyond real time. If you wanna generate five seconds of video in less than three, two seconds, then you need like some of these like latest generation hardware today to unlock that possibility.

• 9:55Jennifer Li?

  Maybe this is a detail of a question, like is the model being served on single GPU or like...

• 10:00Speaker 02

  It is, like majority of the video models today run in a single-node configuration, which is eight GPUs, because once you start scaling beyond eight GPUs, the efficiency gets less and less because of the communication overhead. And existing, both like the existing Minimax H3 endpoints as well as like other video models are probably getting served that like, you know, single-node configuration, same with this, it's like running on in parallel across eight GPUs.

• 10:23Jennifer Li?

  And do you think there will be more efficiency gains in there that you can either, you know, optimize more of the steps in between by sacrificing maybe some of the, like narrowed down the user experiences, let's say, like the different type of inputs and outputs, or like as you're thinking of parallelism, like is there more juice to squeeze? Maybe that's the question.

• 10:43Speaker 02

  We released a turbo version of H3 Max. So initial idea was calling this H3 Turbo. And we were like, we don't want to call this turbo because the quality is like better than the original one, right? Like that's, like this needs to signify how good of an achievement it is. So we released H3 Max, but like a week later, we had like, you know, like our team was like, we can run this 2X faster at like 97th percentile of quality. Like we run evals, they're like almost the same, right? Like there's still like, there's like a noticeable, like there's a small noticeable loss in quality, but we have a version called H3 Max Turbo that's public that can generate like a five second video in like 1.5 seconds, which is like insane. And that's also like 2X, the 2X like from a cost standpoint, it's also like 2X less.

• 11:24Speaker 02

  So there like, depends on like how, how okay you are with like losing quality, you know, you can go down. And like today, these models are so cheap and so fast that I don't think people need any faster or like any cheaper. Like it's already like at a point where the, from a cost standpoint, compared to the frontier itself, it's an order of magnitude cheaper compared from like a speed perspective. It's more than an order of magnitude faster and like you just like enable all the experiences. I think we would need to see, like, I think we would need to see what else lowers that people would need. But I, my bet today is we just need to improve quality more than like the speed at these speeds, right? Like let's fix the speed and let's try to push for quality and controllability of these models,

• 12:09Speaker 02

  which is like, you know, what we have been pushing in the past two, three weeks.

• 12:13Speaker 00

  I think controllability is, is key. Like when we first did it, we did text to video and then image to video. And then references came later, which, which adds a ton of controllability. And it's basically the default mode, how people use these models these days, references. And then we are now adding different loras, fine tunes of the, of the base model as well. We are working on like a lip syncing version. We are working on a different, different camera angle lora, different style loras. So again, open source adds a whole ecosystem around the model and it really, really helps.

• 12:50Jennifer Li?

  Were you surprised by the speed up and the gain you could get from post-training this model? Like, like I, I, I saw it as a little bit of a surprise that like one day, I think it was a Saturday, you'll launch the model and the Sunday people put it on Twitch, it become a real-time model. Like what, that's the interesting part of like when you reduce.

• 13:07Speaker 00

  We, we did evals, like we spent a ton of money doing evals on our own. I don't know, like tens of thousands of dollars even. And like the results were unbelievable. And, and then like the plan was to just release the model without doing external evals. And then, okay, we decided, let's, let's hold off. Let's not, let's not tell people that this is like so much faster and so much better before we have some external validation. So, so we waited like three, four days to all these other like eval platforms to actually run the evals. So we, we matched the results that we have externally as well. And that's, that's how we launched it because as you said, the results were a little too good to be true.

• 13:52Speaker 00

  And it was.

• 13:54Jennifer Li?

  I guess, were you taken by surprise that the real-time use case that came out of it? Or what are some examples that you think this model has like unlocked of the experiences that prior models couldn't?

• 14:05Speaker 00

  This, this happens at FAL once in every couple of months where like the whole company gets, gets hold of something and, and the creativity just explodes and everyone is just working on a, a, a new little app or, or a, a, a different optimization lora, whatever it might, like the whole company gathered around this, this model. And like some, some front-end engineers started working on like interesting applications. And we can talk about our like world model accelerator team, which is, which is brand new. They, they started working on the, the live experience.

• 14:45Jennifer Li?

  RTC.

• 14:46Speaker 00

  The web RTC live experience. So like there were five, six different parallel little projects within the company. And like, I, I, I think we broke a record on Slack that day, how many messages were, were sent in the company. Because like, and, and, and like we have a distributed team. We have, we have people all around the world, like mostly in San Francisco. But it's, it's like incredible when you see like the 24-hour development, like when people like work 16, 17 hours and then someone else wakes up and picks up that like, and that went on for like three, four days. And that's, that's when we released all these projects.

• 15:26Jennifer Li?

  Take me into that, it's so interesting. Cause like you imagine like a model or product launch being like planned out, like having again, like having all these like eval vendors being ready, lined up and, you know, ship something out. And then like you let the, the world or the external users take it and then experiment and like build experiences, put online. Yeah, it seems like, you know, people internally who are very creative, just like took this, drop everything they were doing, like launched an experience that got really popular on Twitter. Do you want to tell us about that one?

• 15:58Speaker 00

  Yeah, of course. One of our engineers, Rehan, just, just by himself completely started streaming a live stream of continuous generations of H3 Max from his laptop, like he was.

• 16:11Jennifer Li?

  His computer.

• 16:12Speaker 00

  His computer, exactly. He, he was doing some like prompt tricks, trying to keep like a coherent story. And then like he started live streaming that on, on Twitch. In parallel, LevelZIO, famous Twitter influencer at this point, had a, had a similar idea. And he reached out to us that he has a website ready already. He wants to like post the streaming himself and, and have, have a website that does infinite streaming. Internally also, we had another team who was working on a continuous version of H3 Max. So H3 Max is like the, the Rehan's version and, and LevelZIO version were independent clips.

• 16:53Speaker 00

  It's still very fast, but the clip starts, it ends, and then you take the last frame of the, of the clip, try to put it in the next one and try to create a continuous.

• 17:04Jennifer Li?

  You need to put some work into like the last frame.

• 17:06Speaker 00

  There's no memory, like the, the second clip doesn't really remember anything from the first clip other than the last frame. But internally, we, the, the ML team was working on a version where it's, the, the transition is more seamless. There is like two minutes of memory. So like you're in a scene and when, when you direct the, the model or someone else enters the room, it actually like everyone looks at that person entering and the scene is continuous. So internally, we were working on that. And then another team was working on an experience we, we called FAL Live for the continuous version. So we had three parallel efforts going on that were all independently going viral on Twitter.

• 17:48Jennifer Li?

  And these were all like spontaneous, like you didn't.

• 17:51Speaker 00

  You didn't plan for it at all.

• 17:52Jennifer Li?

  Plan for any of them.

• 17:53Speaker 00

  Exactly.

• 17:53Jennifer Li?

  And they just became products and experiences in the next, in the following days.

• 17:58Speaker 00

  But Tom, let's talk about the, how, how we made the model more continuous. That was very surprising to me because I've, I've never seen that actually work on, on a video model before.

• 18:11Speaker 02

  Yeah, so going back, we have been like very, very focused towards role models and essentially like action controlled or like, you know, action-driven, real-time, continuous streams of video. And the problem till like, you know, something like H3 Max was quality was not good enough at all. It was just like, you know, you, it degraded a lot. It didn't remember the past before, but we built the infrastructure. We built the infrastructure that we can like go stream video, have people control it in real time, being able to like multiplex it to multiple people, very low latency. And at the same time, our ML team was essentially trying to take every single video model and try to apply this set of like optimizations and tricks to, okay, how can we make this generate instead of a five-second video? 15 seconds, 30 second video.

• 18:56Speaker 02

  But you were always like below the real-time factor, where you know, you, you were always like, you know, you like, you, you never could generate like five seconds under five seconds. Once H3 Max unlocked it, the ML team was like, this is insane. We shared like separate teams internally. We have a research team, we have an inference team, we have an ML team. They're like, they, they saw this and like, this is insane. We can apply all these like set of learnings that we had in previous models where we attempted to do this. Where instead of trying to generate the five-second chunk, that's try to generate, you know, like a 50, like 10-second video. And then the five seconds from previous one is still attended, we still remember it. And like as the video goes up, we can like extend that memory up to two minutes. And you need to do extremely clever optimizations because attending to a two-minute video

• 19:40Speaker 02

  is just extremely, extremely compute intensive. It just like, it goes up exponentially because, uh, uh, from, from like a compute standpoint. So like we, we, we did like lots of optimizations there, but that none of the state we were able to, okay, we can remember back to two minutes, which is like generally good enough from a memory perspective. And then obviously with like prompt tricks, you can still like continuously remember more finer, like the grain details above the two-minute mark. And you can essentially stream infinitely, we captured an hour from that perspective. And then that team just like released that model under H3 Max Director, which is public for people to use. And I think it's the only model that can generate like, you know, up to 60 minutes continuous videos that is action-controlled.

• 20:21Speaker 02

  You can like, you know, start with a prompt, say, uh, like there's like an office setting and someone is like, you know, working. And then like 30 seconds later, it just imagines by itself. 30 seconds later, you can like say, a woman walks in through the door. Like it, it can take the prompt and reflect it immediately, which is the most fun part.

• 20:36Speaker 00

  And the office is still the same office. The camera can pan back to the original person and the original person is still there in the same state. Yeah.

• 20:45Speaker 02

  So, you know, we, we, we released that and it, it got like, we, we, we did this like FAL Live website to just like demonstrate it because it's like, people need to see how cool this is, right? This is a new technology. I, I don't think people are like really aware. And it, it got also like very viral immediately because we also let people vote on what the next action is. It was like, you know, uh, like a form of like a.

• 21:05Speaker 00

  Crowdsource.

• 21:06Speaker 02

  Crowdsource, like the chat was controlling whatever was happening, which, which is fun. But obviously, you know, we, we limited on like the options and then they could pick, oh, like a banana entered the office instead of a moon. And it's like more fun and like people start like, you know, having this. And we started adding more channels and like every channel had a concept. There's like a channel where it's like full chaos. There's a channel where it's like cartoons from like eighties. And like the model is like extremely capable and it just like remembers like so many different concepts. And there's like, it has like a big, big memory, uh, from, from like, uh, styles and like, you know, concept perspective. So it, it just became like a very fun experience underneath.

• 21:41Jennifer Li?

  Again, like, there are so many really incredible experiences coming out of this. Like, H3 Max Director was just another huge surprise to me. It's like, I, I, I found it interesting in the, in the gen media market that you, it's not like, you know, like, language model have like this linear graph of like just continuously of compounding on like, you know, intelligence capability and so on, like, feels like in the field you're operating in, it's always like a few months of like, sort of quiet time, but like, a lot of things are bubbling. But like, in a very short period of time, like, everything bursts. Like, all these things come in, in combination, come together of like, the, the, um, uh,

• 22:23Jennifer Li?

  base model being good enough, like, you can get the latency down to the point where you can like-

• 22:28Speaker 00

  References, yeah.

• 22:29Jennifer Li?

  Yeah, um, like, get the real-time experience, but also like, apply controllability on top of that real-time experience. Like, this just opens so many, you know, opportunities of like, live experiences where like, end user can control what's happening on the screen, which is incredible. Like, uh, we have imagined a lot of these experiences, but never been able to like really play around with it. Maybe just like, tell us more about what you're seeing from the market of like, how are people using, like, um, uh, the director capability? Like, what are you, uh, seeing creators are creating that you haven't seen before? Uh, and what do you think that unlocks as far as, you know, what people can do with the, this, this medium?

• 23:10Speaker 00

  Yeah, it's been like almost three weeks since we released H3 Max. And already, it is the most popular video model on the platform, on, on the FAL platform, by like double, almost, like, more than double, um, in terms of like volume. Um, so, in, in a lot of other platforms, it's also becoming, uh, the default model that people interact with, because it's so fast, so cheap, it just makes sense. If, if you, if you come to a platform, this is the experience that, that you wanna see. Um, so in terms of like popularity and volume, it's, it's taking over, at least from our vantage point.

• 23:51Speaker 00

  And for, for Max Director, again, there has been, I don't know, tens of different versions of these live streams. Some of them are, uh, still going on and like, becoming more and more popular. We are trying to work with some like AI IP holders, people who have like, AI shows on Instagram and TikTok and, and do, train a Laura on, on their style and do a live version of their show. So, uh, we have couple lined up already, so that's gonna be very exciting. Um, and like, the way people, like, i- if you talk to, uh, a creative technologist, prompting with voice has already become, uh, something that, like, they use all the time.

• 24:36Speaker 00

  It's like using Whisper Flow or the ChatGPT voice mode.

• 24:40Jennifer Li?

  Yep.

• 24:40Speaker 00

  And, and now, like, you can keep talking to the model and it, it like, almost as if it's a real director in a real movie set, uh, directing like the camera, directing people where to go, you, you can do that. And like, our creative engineers started using these models like that. So, we'll see, like, a lot of interesting experiences are built as we speak.

• 25:03Jennifer Li?

  Very interesting, as in like, the video is playing.

• 25:05Speaker 00

  Video is playing, you are, you are like talking to the video and, uh, what, what's, what's being displayed changes accordingly, yeah.

• 25:12Jennifer Li?

  That's incredible. Um, and talking about like, how the, the memory piece holds now, like, are, again, this may be a technical detail, like,

• 25:24Jennifer Li?

  the capability of remembering what happened in the last scene or in the last couple minutes of scene? Like, are you remembering that through like the, the frames, the images, or is it like through text?

• 25:33Speaker 02

  It, it essentially, it, no, it's, it's essentially the, like, it remembers the, the video obviously very, very compressed, because you can't attend the full video, but it essentially knows like, eh, most of the, like, the details happened in the past two minutes from its own generations. And above the two-minute mark, it has like, think of it as like, it has this evolving system prompt, on top of the two-minute mark from two to sixty minutes, where it knows like the overall structure, overall detail. So it, it remembers like the last few scenes. If you think a scene is like fifteen, thirty seconds, then it remembers like the last four to eight scenes. And then on top of that, there's like a continuously evolving, gradually evolving system prompt that like, keeps remembering the, like, the overall, uh, coherence of the, of the world.

• 26:16Speaker 00

  Like, e- everyone's waiting for a, a large consumer moment in AI. Now, it's like, good enough and, and, and cheap enough that like, a truly novel, social AI experience can be built on top of it.

• 26:32Jennifer Li?

  Maybe let's talk more about the economic side of this, like, what is the, um, uh, I guess, one, just like talking about serving cost for like same, uh, minutes of video, uh, with, it's, it's three Max.

• 26:48Jennifer Li?

  And how has it changed your thinking around, like, your footprint of like, inventory of chips? Like, how do you wanna have, um, like, different, uh, steps of-

• 26:57Speaker 00

  Yeah.

• 26:58Jennifer Li?

  -experiences serving to the end user?

• 26:59Speaker 00

  Yeah, but I mentioned this a little bit, like, everyone talks about how, uh, complex the next generation LLMs are, but video models are actually very complex as well, because the, the pipeline has different components and sometimes they require different hardware configuration for efficiency, things like that. So, the, the, i- if you were to do this even more efficient, let's, let's call it maybe even cheaper, uh, you would probably run different parts of the pipeline in, in different types of hardware. Um, another interesting thing would be to, um, ma- run it on consumer hardware, for people to run it in their, in their own machines at home.

• 27:40Speaker 00

  Like, optimizations don't translate hundred percent, but translate somewhat, somewhat close to that. And then we can do extra work to translate more of it. Um, so, doing these optimizations in different types of hardware and combining the pipeline in a way that it's even more efficient, uh, I think that's what we are gonna do in the, in the next coming weeks.

• 28:04Jennifer Li?

  Amazing, so, you will have, uh, people like Rohan that can stream partially of the experience from, from his computer, but also having like, the director and the, the control plane more living on the-

• 28:16Speaker 00

  Exactly, yeah.

• 28:17Jennifer Li?

  On the cloud, makes sense. So we talked about all the consumer experiences, um, this model could unlock. Um, and it seems like Bhutan is happy with all the efficiency, uh, like squeeze out of the GPUs. Now we're talking more about how do we like improve quality and controllability of these models? So that like, you know, the high-end of the market, the Hollywood creators, uh, directors, can, can take this to the next level. Um, I saw some demos, uh, coincidentally, like, you know, this model came out of, uh, came out the same week or week prior to Astra. People were combining the Blender experience with, uh, H3, uh, Max from, from FAL. Like, um, talk about how it's gonna impact the, the, the Hollywood world.

• 28:59Speaker 00

  You, using Blender, uh, with one, one of these AI models together is a extremely popular workflow for, for professional work. Basically, you, you, you render a low resolution of your scene, what you wanna do using, using Blender, previous like non-AI technology. And then, once you add that video as a reference to an AI model, you basically get close to hundred percent controllability. Um, and this, this is an in- incredibly popular, um, workflow for VFX artists, people who are doing this professionally, because they want exactly, they, they, they wanna get exactly what they put in into, into the model.

• 29:40Speaker 00

  And a- as, as you mentioned, uh, a week after we, we launched, uh, H3 Max, uh, people starting creating, generating, uh, these beautiful scenes using an LLM model, GPT-Astra in, in Blender. And all of a sudden, it unlocked a, a whole new pipeline using LLM to create a Blender scene and then passing that to the H3 Max model or, or any video model. But it, it works very well with H3 Max, because it's extremely fast and you can like, try many things all at once in parallel. Um, and that, that unlocked a whole new workflow for Hollywood and professional people. And it gets you to like close to hundred percent controllability.

• 30:23Speaker 02

  As, as I said, we have been very, very focused towards speed, performance, quality, and now we have a really good base model. I think- The next month or two is gonna be fully focused on, okay, how much controllability we can add to these models? So that professionals at studios, professionals who want to actually produce, produce content that fits their use cases perfectly can leverage these models. The team has been working on an amazing, you know, like a lip synchronization model where, you know, you can just supply the audio, you can supply your, uh, you can supply like a video or an image reference. And then it can like synchronize the lips perfectly. Same with like motion controls. Uh, you can just take a motion of someone dancing and apply it to like your

• 31:07Speaker 02

  AI generated character and it fits perfectly. And this like, you can like get these results with like basic prompting and you're gonna get like eighty percent, ninety percent reliability. What we are targeting is like nine, nine point nine percent reliability in the outputs. So that you can actually trust the model did every single aspect of this generation perfectly. And that's like what we've been pushing. Uh, one big launch that we had last week was the camera controls, uh, which is essentially you can direct where the camera is going within the video perfectly to the, to, to the degree.

• 31:39Jennifer Li?

  And this is by like describing in the prompt or like generating the?

• 31:44Speaker 02

  We just essentially like underneath you give a JSON of like, I want camera at like zero, zero, zero at T zero. I want camera at like ninety degrees angle at T one. Like you essentially supply a structured, uh, structured description of where your camera needs to be at any point in time. And then the model is like perfectly conditioned to, to regard it as like the only source of truth and it doesn't like hallucinate, uh, um, like where the camera should go. And it just like, you can essentially reconstruct three D scenes from a single input, like because the model itself is a very good video model. But at the same time, you know, it's like perfectly adheres to, to the camera itself.

• 32:21Jennifer Li?

  And this is because, uh, the base model itself already has the understanding of the camera angle that you can?

• 32:27Speaker 02

  It doesn't respect it. It just under like, you need to tune them all. You need to tune the model to a certain degree. And this is like what enables like at large scale, you know, post-training infrastructure. We now have the infrastructure to take H three max, add any capability to it. Same applies for any new model, right? There's a new video model. We, we, we, we essentially spend most of the time building it as an infrastructure than just like one-off training runs. So that we can, we can build like services around this for not just like, you know, open source models, but for like frontier closed source models as well. Because we see in the market, this is like the biggest gap. It's just how controllable these models are. Even like first we start with text to video where you put a prompt, you got a video back. It was good, but like you never could describe the perfect character for you.

• 33:10Speaker 02

  And we had image to video where, you know, you use an image editing model and then generated like, you know, the first scene and then the model was like obviously much more fitting. But you still couldn't like say, oh, I want this new character appear at like second three. You need to put it to your first frame or like you, you can't like, you, you can prompt it, but this model's never perfect. And then we added reference to video where, you know, uh, you can provide like an initial starting frame. And you can also provide, I want these characters with these voices. Like, you know, that's also like a big unlock where you can essentially say, this is the, this is the voice for this character. And now like, you know, we are adding, oh, within the scene, I want camera to look at this degree at like T zero. I want camera to look at this degree at like T three. And then we are adding lighting controls where you essentially say where the light is coming from. These are all, all compounding on top of each other.

• 33:53Speaker 02

  And like, we just have the unified infrastructure to just apply this to any model at this point.

• 33:57Jennifer Li?

  That's incredible.

• 33:58Speaker 00

  Hollywood is our fastest growing segment and um, there, there's a lot of noise about how AI might disrupt Hollywood, but Hollywood usage was non-existent a year ago. And in, in the past year, it grew and now it's, it's the fastest growing segment. Like, um, Amazon MGM studios in, in their conference, they, they released their NARA, uh, tool. It's, it's mostly backed by, uh, file infrastructure behind the scenes. And we are seeing incredible, incredible pool coming from Hollywood. And it's exactly what they need, these like small point solutions rather than generating everything from scratch. They wanna be able to extend the video a little bit. They wanna be able to change the, the camera controls.

• 34:43Speaker 00

  They wanna change the lighting and someone has to build these solutions for them. What, what Hollywood needs and what the creators actually need and what the research labs are working on, there's a little bit of a disconnect there. And we believe we can come in and do these little post-training projects to, to close that gap. Because we work with, uh, all the Hollywood studios and we hear from them what, what they need. And these are exactly the things they need. These small point solutions that actually make them more efficient, push out more, more video and, uh, AI can actually close that gap very nicely.

• 35:22Jennifer Li?

  Maybe say in a little bit different way, like we have been staring at this problem for the last three years as well. Like we see like companies trying to like build a, you know, uh, movie director, uh, like video model like by, uh, either pre-train or post-train on, on the video side. But what I'm hearing is like different people expressing the way they want the output to come out very differently. Consumers talk about it and then like write the prompt and generate the results very differently from a Hollywood director, which is obvious, right? Like professionals wanna talk about like, you know, these camera angles. They wanna talk about the lighting, like you sort of have built a library or like a, a collection

• 36:04Speaker 00

  Toolkit, yeah, yeah.

• 36:05Jennifer Li?

  of, um, post-training, like, I'd call it data and toolkits that can apply these, uh, any model that you can like grab the weights on so that they are adapted to like a different audience where they can express their creativity in a bit different, uh, fashion to control the model.

• 36:26Speaker 00

  Hundred percent.

• 36:26Jennifer Li?

  When it unlocks a lot of the capability underneath.

• 36:29Speaker 00

  And half the problem was capabilities of the, of these models. We are solving that. The other half of the problem was legal and data residency, things like that. We, we, we made a ton of progress there as well. Uh, we now have a system of, uh, people can apply with, with their own IP and we unlock their own IP in, in the models. Uh, we are gonna grow that and that's gonna be, uh, a, a very powerful thing we do with Hollywood studios. Also, we, we now have Sea Dance US hosted as well. We already had previously other Chinese models. Sea Dance was the missing part. Every Hollywood studio wanted us to have it US hosted. Now that's available.

• 37:12Speaker 00

  And so there are no, no obstacles in front of these Hollywood studios now. Everything is ready and we, we believe they are gonna ten X, hundred X their AI usage in the coming months.

• 37:25Jennifer Li?

  It's such an exciting world for, uh, movie lovers, consumers, people who consume a lot of video and, and creative content.

• 37:32Speaker 00

  And, uh, we have our conference, generative media conference next, next week. Uh, this is our second time we are doing it. Last year it was mostly consumer AI. There were like maybe a couple Hollywood executives here and there, just curious about it. And now it's dominated by studios. New AI studios who are like offshoots of the bigger studios trying to do like only AI, AI shows. But also like the, the biggest of the Hollywood studios are also there because now they have big plans integrating AI into their workflows, into their, uh, existing systems. So you can see the, the change in the attendance of the conference as well.

• 38:14Jennifer Li?

  That's awesome. Well, for the audience, check out, uh, the content coming out of the, the Gem Media Conference. Uh, it's gonna be very, very exciting and thank you so much for coming and Botone coming on to our show. It's a very exciting time for Gem Media.

• 38:27Speaker 00

  Thank you.

• 38:30Erik Torenberg?

  Thanks for listening to this episode of the A16Z Podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts and Spotify. Follow us on X at A16Z and subscribe to our Substack at A16Z dot Substack dot com. Thanks again for listening and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. Should not be taken as legal, business, tax or investment advice or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast.

• 39:14Erik Torenberg?

  For more details, including a link to our investments, please see A16Z dot com forward slash disclosures.