Luma Labs' Diffusion Revolution: from Dream Machine to Multimodal Worldsim - Amit Jain, Jiaming Song

In this episode of the Cognitive Revolution podcast, the host Nathan Labenz welcomes Amit Jain, CEO and Jiaming Song, Chief Scientist at Luma Labs, alongside co-host Stephen Parker. The conversation delves into the latest advancements and products from Luma Labs, makers of the Dream Machine, including cutting-edge models and features like camera motion and creative video generation tools. They explore technical aspects like pre-training for diffusion models and the development of concepts to improve AI capabilities. The discussion also covers the philosophical and practical implications of AI interpretability and multimodality, along with a deep dive into the intellectual history and recent innovations in diffusion models. Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker https://www.imagineai.live/ https://adapta.org/adapta-summit https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ SPONSORS: ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U. S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive Shopify: Shopify powers millions of businesses worldwide, handling 10% of U. S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Nathan Labenz?

  Hello and welcome back to The Cognitive Revolution. Today, I'm speaking with Amit John and Jamming Song, CEO and Chief Scientist at Luma Labs, makers of the Dream Machine and the new Ray2 video generation model. I'm also joined for this episode by my friend Stephen Parker, Creative Director at Waymark and one of the few creators that has logged a proper 10,000 hours with video and image generation models dating back to the original DALL-E over the last few years. Our conversation begins with a discussion of how the Luma team trains models to create fantastical and other fundamentally out-of-distribution visuals for which there is little to no relevant training data available. But considering the force of intellect that both Amit and Jamming display, their belief

• 0:43Nathan Labenz?

  that video models are on the critical path to AGI, their ambition to create multimodal AGI at Luma Labs, and the range of novel and occasionally hot takes they share, I think this episode should be of interest to anyone, regardless of whether or not you're particularly interested in video generation models specifically. Keys to Luma's model development success, as you'll hear Amit explain, include a relentless focus on dataset curation, frontier advances in efficient learning algorithms, and a strong drive to understand what their models are actually learning as they go through the training process. These fundamentals create base models that can learn new concepts, including the Bolt Cam and many other camera motion concepts they've recently introduced, in a highly sample-efficient way.

• 1:28Nathan Labenz?

  Meanwhile, for things that existing models can't learn so quickly, we also discuss Luma's outer loop of product development, which consists of building scaffolding and other behind-the-scenes systems that unlock new model capabilities and also validate customer demand. With that done, they then seek ways to internalize those capabilities in the next generation of the model, and then they repeat this process for each generation as customers continue to apply new and better models to harder and more valuable challenges.

• 1:56Nathan Labenz?

  For me, the most interesting part of this conversation was the discussion of model interpretability. Emphasizing that we should not expect AIs to process, represent, or understand information like we humans do, or even to do so in a way that's generally human-grackable, Ahmet Lakin's current interpretability techniques to archaeology, in the sense that they're fundamentally limited to piecing together what models have already learned in the past. More interesting from his perspective is the study of training dynamics and engineering of data sets that are needed to teach models what they most need to know. In the last 15 minutes or so, Jiaming offers an intellectual history of diffusion models. This gets pretty technical, and for most people, myself included, it will require some additional study to fully understand.

• 2:40Nathan Labenz?

  But I would summarize it by saying that the generative AI era really began with the realization that with the right problem formulation, unsupervised learning can work on web-scale data sets. For text, this was simple next-token prediction, and for images, it was gradually adding noise to real images and then training models to remove that noise one step at a time. Since then, there've been a mix of practical tricks, theoretical insights, and model-enabled data set improvements that have unlocked far more precise steering of outputs and breathtaking efficiency gains. From distillation techniques, which amount to training a model to perform multiple denoising steps in a single pass, to consistency models, which try to ensure that a model will generate

• 3:23Nathan Labenz?

  the same output regardless of where it begins on its denoising path, to flow matching models, which use theoretical connections to differential equations to take a more direct path through the latent space, to Jiaming's latest inductive moment matching technique, which optimizes the model in distribution space and performs generations in a small number of optimized steps, all of this should at minimum give you a sense of how inference prices have fallen so precipitously, even as quality has dramatically improved.

• 3:52Nathan Labenz?

  While we didn't have time to go as deep into the philosophical underpinnings of Luma's multimodal strategy as I might have wished, I left this conversation with the sense that Luma Labs is definitely a company to watch. It won't be easy for any model development startup to compete with the big tech hyperscalers in the scaling laws era, but Luma's mix of product market fit, vision and ambition, and research prowess gives them as good a chance as any I've seen. And I absolutely look forward to having them back again in the future. As always, if you're finding value in the show, we'd appreciate it if you take a moment to share it with friends or write us a review on Apple Podcasts or Spotify. And of course, with the stakes of AI development continuing to rise, I welcome any feedback you might have for how I can do a better job of elevating the discourse

• 4:35Nathan Labenz?

  and help steering the future away from catastrophic risks and toward the dream of AI abundance. You can reach us via our website, cognitiverevolution.ai, or by DMing me on any social platform. Finally, a quick reminder that I'll be speaking at Imagine AI Live, May 28 through 30 in Las Vegas, the Adaptive Summit, August 12th and 13th in Sao Paulo, Brazil, and the Enterprise Tech Leadership Summit, September 23rd through 25th, again in Las Vegas. Tickets are on sale for each of these events now, and if you'll be there, please do reach out and let me know so we can meet up in person. With that, I hope you enjoyed this insightful and thought-provoking conversation about the development and philosophy of frontier video generation models with Amit Jan and Jamming Song of Luma Labs.

• 5:21Nathan Labenz?

  Amit Jain and Jamming Song, CEO and Chief Scientist at Luma Labs, makers of the Dream Machine and Ray2. Welcome to the Cognitive Revolution.

• 5:32Speaker 02

  Thanks for having us. It's very exciting to be here. Yeah, thanks for having us.

• 5:36Nathan Labenz?

  I'm excited for the conversation as well, and also excited to have my good friend and longtime teammate, Stephen Parker, here as well. Stephen occasionally co-hosts when we do an episode on creative models, especially in the image and video domain, because he is the Creative Director at Waymark, and he has logged more hours than anyone I know with these kinds of products. He really has an excellent handle on the exploding array of options in the market today. So I thought- To broadly structure this conversation, we might start with discussing like the latest and greatest stuff that you guys have launched in the products, the latest models, the camera motion, all that new cool stuff and how it fits into the broader picture and what you're seeing in terms of usage. And then I want to get into some of the more technical stuff

• 6:18Nathan Labenz?

  because you've also put out a very mathematical paper recently on an advance in pre-training for diffusion models and a really interesting position paper on how you think we should be thinking about pre-training going forward in general. So I'm excited to get into all that as well. And then maybe at the end, if there's time, we can get a little bit more speculative and talk about like world models and the future of multimodality and what superintelligence looks like and all that great big picture stuff as well. So we've got a lot of ground to cover and I'm excited for it. Steven.

• 6:48Speaker 02

  Awesome.

• 6:49Nathan Labenz?

  Kick us off with maybe a few reflections on your use of Ray2 recently and what that has you thinking about as we begin today.

• 6:58Stephen Parker?

  Yeah, thank you. Thanks for having me. Amit and John Ming, it's an honor and a pleasure to speak to both of you. So thank you for taking the time. I have been playing a lot with Luma Labs recently, really enjoying it. It's one of many video gen tools that I love to use in my arsenal of possibility when I'm working on various projects. And for my own money, tend to find tremendous value in several different models being used at the same time as they all have different strengths, different weaknesses, really just a whole blend of capabilities. And Luma is right up there at the tippy top of especially the cinematic end of models that I like to use. So just really want to give a shout

• 7:42Stephen Parker?

  to you guys there. And my first question there is like, I think more and more of what people want from an image to video scenario starts with AI images. That's just the hypothesis on my part, but is that correct?

• 7:54Speaker 02

  I think a lot of workflows do start on Amit's side because images are just much easier to iterate through. Iteration cycles are fast and generation times are low. So like you can generate 100 images and find exactly what sort of things that you're thinking about. So yeah, I think a lot of people actually lean in on image workflows as a significant part of your work today.

• 8:16Stephen Parker?

  Okay, so that makes sense with my own workflow, especially what I see out there on social media. And so I think what I'm driving at is like, a lot of those images to me seem like they can be strange or new. I'm thinking of like avocado chair type combinations, weird characters, all of these sorts of things. And what I really want to know is like, what has it been like to push these models toward a greater understanding of what I presume is like more novel subject?

• 8:42Speaker 02

  Yeah, that's really interesting. And currently we are designing a feature. And that is one of the most more important problems because see that there's no data. Right? Because by necessity, these are out of distribution things and stuff that you would just not find in regular, regular use cases. So I think the singular answer there is like, you need to really work on a very strong base model that has strong capabilities of like, understanding what is happening and being able to like then deal with these very unrealistic outer distribution scenarios. So let's say like a pickle sitting on an avocado chair like, and you start with the pickle standing up and you want it to come and sit down. There's no scenario, you've ever seen that, right?

• 9:25Speaker 02

  But you've seen chairs, you've seen people, and you've seen anthropomorphic things, and you've seen them doing things like sitting down. Right? So this again comes down to the idea that like, these models are not memorizing behaviors. These are not memorizing like how this is done, how this is done, how this is done. They're like, they're generally distilling out base ideas and capabilities, like the core things. What does it mean to be anthropomorphic? What does it mean to sit? What does it mean to be a chair? That kind of things, right? And then when you combine them together, the better your model is at these foundational understanding, the better it's going to be able to do when it's presented with these kind of like entirely out of distribution, funny, uncharacteristic scenarios.

• 10:06Stephen Parker?

  If I'm understanding there correctly, like improving the base model is just giving you better and better performance with things that are out of distribution. But another thing I seem to notice is that, especially with image to video, the kind of inherent characteristic qualities of that initial image also seem to be carried through better, more effectively with longer and longer Gens. Do you think that's aligned with the proven of the base model as well? Or are you guys doing more stuff behind the scenes to check against that image more regularly as the gen is occurring? What's going on behind the scenes there?

• 10:40Speaker 02

  There is a lot that going on, goes on behind the scenes of like trying to understand what is in there. These things are not just models, they are systems. And so we definitely take care of many things in the back like if to try to break some of these down. But generally, they are crutches for the model to be able to understand in the next iteration of the model. We make it so that we don't need that system. So like initial model like which was called Dream Machine. And now it's like, or now retroactively renamed to Ray One. It had many of these crutches to make it feel and work much better, right? Like, you know, to understand motion, to understand different vocabularies, to like translate it into language. It understands all this kind of stuff.

• 11:22Speaker 02

  Ray two doesn't need even like 10% of that. But now, because people are now pushing it to do more things, we have to like build some of these systems again. Like, oh, well, okay, I see. That's what someone is trying to do. But how do we understand this part? For instance, like characters, right? All the character understanding in Ray two is very much like an external to the model. Right? But then as we design Ray three, it's all going to be internal to the model. So this is a trend we have seen in language models as well. That then people push the current model to doing things it was not designed to do. Like labs like us, we build systems around it to at least address. We gather the data and then we just bring it to the model. And this also applies to like people, what we call application layer, that kind of stuff.

• 12:05Speaker 02

  Like, application layers come up, they build like these specific use cases, but then we see that. And we are able to like then just build it out in the next one. And the general rule of thumb in this industry is that, or at least in the technical side of it, is that anything that can be done in the model is going to be better than what is done external to the model. And we can talk a lot about that, but that's generally the situation.

• 12:26Stephen Parker?

  That's great. I'm happy to double click there as much as you want. I know we're early in, but I do think this is a super fascinating topic that it's kind of skewed towards secret sauce, company proprietary, is that sort of thing? And so there isn't frankly, a lot of conversation about these kind of systems built behind the models that then you try to reincorporate into the model moving forward. Is anything else you could say maybe more specifically about beyond data as a loose term? Like making sense of these systems and how you teach a model from there to appreciate what was inherent about that system that was helping you. I think that would be fascinating to know.

• 13:04Speaker 02

  That's a big conversation in Jaming, you should also chime in anytime, but the two X. Go a little bit deeper into like in inside the model versus outside the model, right? Like intra-model versus extra-model. That's a really important thing, because whatever you can do in the latent space, all the thinking, whatever you can do about, so for instance, let's talk about video for a second. If you're able to reason about like the actions that are being performed, the characters that are there, what would that character do, the lighting of the character, all these kind of things, inside the model, rather than like a system that actually like generates the right like lighting, that tries to piece together the narrative art, that tries to piece together all these things.

• 13:46Speaker 02

  Of course, that system can work, that's not the problem. But of course, like people tried to do that, like if you remember, what was the name of the company? It was a script-writing company or like a copywriting company, Jasper, right? And they tried really great ideas, to be honest with you, right? But the thing is, when you can do it inside the model, inside the latent space, you're just able to work with a lot more information. You're able to do the kind of edits that like outside the model, you just don't have the control mechanisms for the model to be able to do. It's like this thing, but there's a blood-brain barrier between like a generative model and the outside world, right? And when people say, hey, we want controllability, they want to penetrate the blood-brain barrier better, right?

• 14:31Speaker 02

  And be able to tell the latent space what they exactly want. And it will get there, there's no question about it. But inside, there's a level of intelligence that is just really, really great. Outside, there's a level of intelligence that's really, really great, which is you or I who is using it, and then there's a barrier. So things where you can actually communicate from outside in a higher level, and then let the brain inside do as much as possible internally, collate information, think about sequencing of events in the video, think about what happens to a character, their arc, their timeline, their causality, all these kind of things. The more you can do inside the model, the richer the outputs you're gonna get. This is what chain of thought also looks like in language models, right?

• 15:15Speaker 02

  Instead of forcing the model to like structure its thinking into like, oh, like do this, do this. No, no. Chain of thought is, if you have read the recent Anthropic paper, where they claim chain of thought is actually facetious. And it's just a way to seed the latent space of the model, right? Like to direct it, make the model be able to direct it. And whatever it's outputting is not that realistic of what it's actually thinking about, right? That is actually an indication of that. We have seen that now, like instead of like trying to force the model to output JSON through external coercion, let's just make the model very good at structured output. And then like suddenly you have agents, right? So you're going to see that in multimodal especially, because in multimodality,

• 15:57Speaker 02

  if you're able to simultaneously think in audio, video, language, image altogether, right? And combine reasoning from language, appearance from video, and like all the aesthetics you have seen in images, and then audio that you're able to like hear in all the videos. These concepts, if you're able to combine them, you're going to have much better outputs than through external systems. But yes, we do build these systems, then we obviate them in the next model training. Jamming, any particular thoughts there?

• 16:26Speaker 04

  Uh, yeah, of course. So I tend to agree with, I mean, uh, here. So, uh, like instead of trying to like program this into complicated workflows, just imagine like how you would like communicate with another human. You don't actually like need to program their brains for them to understand that you are, you want to achieve this level of workflow. Of course, there might be some like back and forth, but the process itself is pretty much natural. And I think this can only be done if you build these capabilities into the model, in the sense that the models need to be more intelligent. Whereas I guess right now, even the current like generation of models, like especially like video models feels.

• 17:06Speaker 04

  Less intelligent in the sense that you have to tell them or like program them to achieve like some particular type of task like image to video, for instance. Of course, other, you know, practical reasons that this like is done, but I think eventually be intelligent enough to tell, like basically taking what other task, what in the multi-model sense, you you tell them, for example, I want to do image to video, keyframes, like camera motion and all these different tasks, and the model should be able to like do it without say even thinking about it as a different type of task.

• 17:40Nathan Labenz?

  So could you guys maybe give a

• 17:45Nathan Labenz?

  couple of examples of like, presumably wouldn't be too sensitive to say, things that were essentially scaffolding in the Ray one generation that are now handled internally by the model. And then if you're willing and open to it, things that are currently scaffolding on the Ray two that you hope to be inbuilt into the model when we get to Ray three?

• 18:08Speaker 04

  Like, I guess one example would be like maybe camera motion, like in like Ray one point six, we had camera motion features shift, but that was like obviously like harder to actually control and like lower quality than what we recently released. So in in this iteration, you can see that there is a lot more like different types of like flexible camera controls, such as like both camps, which without the user having to prompt for exactly the extreme six and the extreme six of the camera, it it just works. So of course, you can also give it more explicit controls. That's what some users would want. But I guess for other users, maybe what they really want is just like I convert this scene into like, like give me this effect into the scene without like me having to specify the very detailed controls.

• 18:54Speaker 04

  So I think that's one of the like features that we have it in the model. I think what would be like really interesting is to like build into the model like, like now is one level ahead of like being able to represent the scene, like the book camp. Like, again, you want slightly more control than just generating a book camp, but not as much control. As you have to specify the exact camera coordinates. So maybe somewhere in between, you can have something like, oh, I given this camera trajectory scene, I want this bolt cam to be like moving faster, moving slower, having like more angle changes or like focusing on some other like subject matter. So this kind of like more interactive,

• 19:38Speaker 04

  like editability of the scene is something that we are like thinking about like maybe having in the model in the next generation.

• 19:46Speaker 05

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 19:52Stephen Parker?

  Cool. That was actually my next question. So maybe just to restate a little bit here for our audience who might not be super familiar with bolt cams. That's bringing us to Apple TV plus Severance season two, right? Which made a huge splash this year with their opening shot, which is a bolt cam robo arm shot. I think it was super impressive, famously took them months and months to create and was just a big wow moment for audiences. And now, you know, just a little bit later this year, we have it as one of your key like camera motion concepts already available for people like me to use when generating. It's one of a range of motions that you have added to the model capability. Super useful. And I imagine like very highly in demand from your editor type of user

• 20:36Stephen Parker?

  approaching the model as well as novices and everybody else, but gotta be high on the professional list. What has it been like to develop that feature specifically with an outlook on the professional user or maybe customer requests? Coupled with modern trends like Severance season two and the bulk cap.

• 20:56Speaker 02

  So basically like there are models, right? Like the things you teach it during pre-training. And then there are many capabilities that people want to teach them on. And language models have this really special ability, which is called in-context learning. You give you showed some examples and like it becomes that. And I think that's the truly emergent intelligent capability. Visual models aren't there yet, but they'll get there and soon enough, actually. But they're not there just yet. But that doesn't remove the need for teaching these models. Specific things you want in the moment. So we have been working on this idea. So we published, we published a post about this white paper, whatever you know, call it. We call them concepts.

• 21:45Speaker 02

  We realize that like in visual, especially creative use cases, there are many things people want to teach, like many things that they come up with, like a specific motion or particular kind of color grading or a particular human pose, right, which is just really suited for the story you're trying to tell or the ad you're trying to make, whatever it is. But there's no way to get it out of the model because it's a new thing you just came up with. There is no data for it on the internet. There is no like way to like generate like large samples of it to even able to fine tune the model. So we designed this idea of concepts to make it so that models can learn from one to just a few examples.

• 22:26Speaker 02

  Their capabilities don't degrade like they do when you're fine tuning or creating Loras. And they can be composed together. So closely can it represent a capability that the model had at pre-training, but users actually teach it. So we call them concepts that the tool that we're using to build them, they're called concept school. Right? Like you go to school, you take the model to school and you're the teacher and you're going to teach it concepts or lessons, whatever you want to call it. So the camera motions you're talking about, the bolt cam and like dolly and the reverse dolly, all these kind of things that we taught, actually we were able to teach it to the model very quickly, or relatively speaking, quickly with very few examples. And you're going to see like the next batch of it come out this week and the next batch come out next week,

• 23:11Speaker 02

  so on and so forth. And our goal is to basically teach our models everything about filmmaking this way. And eventually also give this ability to people to be able to teach them. Right now like it's a little bit finicky as new technologies tend to be. So we haven't made it open source or sorry, open access yet, but we build in the future.

• 23:33Speaker 02

  So these camera things, right? Like, you know, the way we are building them to answer your question on feedback and these kind of things.

• 23:40Speaker 02

  There are like some of these which are basic storytelling tools. Being able to track a shot, being able to move left, being able to move right, like, you know, basic things, and they're done by people a thousand times a day. Whenever you're like shooting something, it's like, oh, the camera moves left, like, up, uh, right? Uh, or you're tracking of a, an actor who's like doing these kind of things. So you want that. And then you want to also balance it with some things which are absurd and which are funny or which people just can't do in real life very easily. Like, for instance, Bolt Cam. If you want to do Bolt Cam really well, with that smooth tracking, you actually need a robot.

• 24:15Stephen Parker?

  Yeah.

• 24:16Speaker 02

  So MKBAC has one. I'm sure James Cameron has a few, but like most of us don't. So can we just have it in the model? So like, we try to balance like this great utility with some things which are just absurd and fun. But ultimately, like, we're building this for professionals. We're building this for people who want to tell stories, who are already telling stories, or who want to become professionals in that world. Then we can talk about like the changes that are happening in the AI industry very quickly. Not AI industry, but movie making industry very quickly. But yeah, we're designing for people who want to tell stories, and these are all storytelling tools.

• 24:53Stephen Parker?

  So two things there that just brings to mind for me. One of them selfishly is talking about those repeat actions that the pro user takes. One of the selfishly, one of the pro actions I take all the time is leveraging your audio generation feature, which is great. That's for people who don't know, after you generate a video, you can just press the audio button and it will give you another prompt opportunity and give you the ability to generate audio for that clip. However, one thing I do all the time is reverse the playback direction of my video when I'm editing. And so just a tiny little plug here. I would love to have the ability to quick flip the playback direction before generating that audio

• 25:35Stephen Parker?

  so that I'm not dealing with reverse audio when I take that clip out. But I'll just leave that as a footnote. I think really what this is getting to is like. You're about storytelling. You have all kinds of users telling all kinds of stories and that skews towards fantasy, Hollywood cinema, anime, that's everything, right?

• 25:54Speaker 02

  Yeah.

• 25:55Stephen Parker?

  How do you imagine that translating to multimodal understanding? Is it just like it's an attempt to understand everything everywhere all at once? Or is there like particularly unique insight that you feel like you gain in the pursuit of art first that helps with the overall mission for multimodality?

• 26:15Speaker 02

  See, what we are trying to do is, so the mission of Luma is to build a multimodal intelligence, right? Multimodal general intelligence. If you were to personify, not personify, but if you were to materialize it in front of you and you were to ask like, what does that look like? The intelligence that LLMs embody, that looks very much like abstract intelligence that humans have, that language and things like that. When you think about multimodal intelligence, of course it has that abstract part, but what's that actually look like? It starts to look very much like world in a globe. Like you have this world in front of you and it has all the physical properties and all the physical phenomena that happen day to day in our physical world.

• 26:57Speaker 02

  It also has those intelligent beings inside it that do things that like increase the entropy of the universe, that like interact with each other, that do all these kind of things. So a term a lot of people use is world simulator, right? But I think simulation is a weaker term here than it should be used. It's basically just like it's a world model. It's a physical manifestation of the universe that you have outside. Of course, it's a weak facsimile, it's an approximation, but yeah, it's a model of the world, of the processes that we are having. Now, coming to the creative side of it and why that it's important for this mission is like storytelling. Like if you think about it like language models, right? And we were like, oh, this is only good for JSON and then we're going to produce this JSONs, right? That's not very good.

• 27:40Speaker 02

  When you, when you force intelligent systems to play games, to come up with like abstract new things, to come up with like, like instruction following, like, oh no, I want this, then I want this, when you ask them to dream.

• 27:56Speaker 02

  That's when like you get intelligent systems versus just like procedural systems that are like following a set of rules that you have created. When you force them to deviate from that, making movies, telling stories is a very critical part of it. It's a part of like human existence too, right? Like, how good someone is at storytelling is generally a very good bearer of IQ, right? Like, can they actually think beyond just the most like physical thing that is in front of them, right? What is the consequence of A? What is the consequence of that? What is the consequences of that? What does it lead to things? That's what stories are, right? Event A then happened, then B happens, then C happens, then C, D happens. Video models are on the critical path to that general intelligence.

• 28:37Speaker 02

  Once we're able to combine video, audio, language all together, they will become really, really good at storytelling, right? Being a partner for us, like who can actually sit next to us and like help us dream, help us imagine, help us like think through those kind of things. So creative pursuits, it's not a surprise, by the way, that people thought the first things to be automated will be like the mechanical things. It's not a surprise that the first things to be automated are, or not automated, but the first things where AI is able to help are actually creative pursuits. Because that's where we require intelligence, that's where like these things come into play. So, yeah, I think art has always had a very significant role to play in general thinking, general intelligence.

• 29:18Speaker 02

  Art is also going to have a very significant role to play in building artificial general intelligence.

• 29:24Speaker 05

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 29:30Nathan Labenz?

  Can I ask a couple kind of world model questions?

• 29:33Speaker 02

  Yeah.

• 29:34Nathan Labenz?

  I think this is super interesting. One big thing that jumps out at me, though, is if I was trying to create a generally useful AGI to go out and do stuff in the world, right? To maybe like control robots and come and walk into my house and make me a coffee, to take a famous example, and to vacuum up my living room after my kids have thrown toys all over it, whatever the case may be, I would want this sort of world model. And I, you know, I'm a certainly a big, I wouldn't say believer at this point is the right term, because I think it's like pretty well demonstrated that the large foundation models are learning these higher order concepts. And that's like, I think, become pretty much indisputable at this point.

• 30:16Nathan Labenz?

  So, you know, well convinced by the evidence in the literature that this is happening. But I wonder about the sort of fictional side of it, the magical side, the like, there are no dragons in the real world, but your models have learned to also represent dragons, right? So in a way, you have a real world model plus and sort of goes beyond what is real and into the imaginative, the fictional, et cetera. And I wonder, like, obviously, that's good for storytelling, because people want to tell stories that are not bounded by reality. But does that have downsides for the sort of practical utility of making multimodal

• 31:00Nathan Labenz?

  intelligence that I just want to like, come into the world and do useful work for me?

• 31:07Speaker 04

  Yeah, so I think like, there is a short term, like answer for the short term, and there's the answer for the long term. I guess the answer for the short term is that you, if you want to use this type of model for your daily activity task, like right now, then yes, it might be better to angle towards the more like physical, like applicable, like vertical, so to speak, for this type of models. However, your, your model, there are many cases where the, you ask the model to do some tasks. So for example, say, pick up this like clothes with a dragon on it and push it, like, put it in the like, like washing machine. And if the model doesn't really know about the concept of a dragon, you can actually pick up the dragon,

• 31:51Speaker 04

  like, like pattern, like clothes into the washing machine. So even this kind of like knowledge that is like fictional can still be very useful for like real world, like physical tasks. Because again, even the concept of a dragon by itself is related to like other physical concepts that like people see in the world. For example, it looks like a lizard, it has like scales that are exhibited in like reptiles, it has wings, which are, you see them in like bats and birds. So. The like even these concepts are this, if you think about it, are what like humans interpretation to like what they see in the real world. So it's like the dragon is generated by humans looking at the real world as all part of their training data. And they generate something that is like out of ordinary.

• 32:36Speaker 04

  But back to the original question, yes, if for the short term, yes, I do think maybe like focusing on more like physical, like like realist, realistic things can be better for these kind of tasks. So the model doesn't like hallucinate as much. But in the long term, I believe that the model should be able to tell itself apart from like what kind of world it is like acting in. Because there's also other like types of like a world that needs to be able to like act in. And it's not just like in the physical world. For example, in a virtual world where like things people are currently doing with agents are also a perfect example where like the model doesn't, they need like a world knowledge, but it is a world that is different from the physical world that to be ever like acting in.

• 33:18Speaker 04

  So having a unified AI that can be doing like both tasks could be useful. For example, eventually your robot could be like, okay, put this like dragon T-shirt into my dishwasher. And then go to the computer and type an email like responding to like my friend or something like this. This requires the AI to be able to like reason between like physical and like non-physical or imaginative worlds or virtual worlds, just like humans can do. So I think eventually this will be a capabilities that does not have to be specialized into any sort of models.

• 33:51Nathan Labenz?

  I don't know if you're doing interpretability work on your models internally, or if you have any partnerships, academic or otherwise, that allow you to do that kind of stuff. Maybe you know the answer because you maybe have done this work, or maybe you could speculate as to whether you would expect to find features in the model that would represent like this is a realistic physics simulation versus this is a magical realism type of scenario versus this is a Minecraft environment that we're in right now, or we're playing Pokemon or whatever the case may be like. It seems like that probably would happen at sufficient scale, but I don't know if we are there yet, or if anybody's really had the opportunity to look.

• 34:34Speaker 04

  Up front, I'm not an expert on the topic of like deep neural network interpretability, or I have like delved too much into this space. My very shallow understanding about this is that it might be good to like seek like more alignments, or like try to get the models to align with like what you want and like try to reason from there. So for example, one feature that we have discussed heavily in the first virtual machine is this like implicit knowledge about 3D. So for example, you can ask it to generate things that reason about like depths. And even in like a fantasy, like our realist, it has in art, it has a reasonable interpretation of like depth.

• 35:17Speaker 04

  And also other types of like physical features such as like clothing simulation, such as waves and such as like air, these kind of stuff. So it's in this case, and the question will be asking that you're asking is, is there a kind of neuron or like a substructure I can find within the model that like shows that? And yes, the honest answer will be it's very difficult to like find that within the model. But it's probably might be easier if we prompt the model and ask it to do the task for us. So one example is like, for example, people have shown that these models are very great priors for things like three division tasks, such as like depth estimation.

• 35:56Speaker 04

  So in this case, like even the almost like a lot of the state of the art, like depth estimation methods are like actually based on these like pre-trained models, which actually have a very good understanding about the world. So I do believe that some degree of interpretability or like the model has a great internal knowledge about the world. But it's also up to humans how like interpretability exists. So currently, it seems like the API between like the human neurons and the like model neurons are not that well welded just yet. So probably we'll still have to use like the language that both sides understand to communicate this interpretability thing. Yeah, there's definitely some fraught challenges there. Yeah.

• 36:36Stephen Parker?

  This is an interesting philosophical question that Nathan and I go back and forth on all the time. I mean, we tend to hear from people a lot that these models have physics knowledge inherent in the training. They have a great, rich 3D understanding of the world that we see it in this or that. I personally am not entirely convinced that they aren't just seeing more and more kind of movement across a 2D space and have like greater and greater pixel understanding. I can understand somebody arguing against that from maybe a systems-based training regime where we're training first on 3D scaffolding or something like that. But as I my own naive understanding is that sort of thing isn't happening in the training. I just think it's a really interesting

• 37:18Stephen Parker?

  question. Like, are we actually seeing the physical appreciation for a 3D space? Or are we just seeing more of an expert interpretation of a 2D pixel space?

• 37:27Speaker 04

  So I would say it's, I would rather this like ability to maybe predict or like reason about physical scenes in this representation of the 2D pixel space to like a Turing test. So in the Turing test, your goal is to like not tell whether the model knows like grammar or not, like whether this internally represents this scene with the grammar that humans define. It's like not really part of the question that we are asking. What we are asking is that whether talking to the model, it feels the same to talking to a human. In this case, it's like similar. So on the one hand, it's like being able to like render do

• 38:09Speaker 04

  the next scene or like prediction even perfectly over the next like few seconds or like minutes or so, doesn't actually mean that you like inside it is the same physical knowledge that we currently like define as humans. But on the other hand, to a utility standpoint, it might constitute as passing the Turing test or like visual generation.

• 38:33Speaker 02

  Yeah. And I have had a lot of time to think about this problem. And the question is basically philosophical to what Zhuangming is saying, that it borders on the definition of understanding, right? What does it mean to understand? And people make the argument that humans have something very special or deep where we understand at some level. But it's very hard to actually argue about that as well, because how would you know? Right? You can say like, oh, yeah, but I know this is 3D. That's why I understand it. Like, it clicks in my brain that this is 3D and not 2D. But that's all like the the brain is this really interesting organ that is simultaneously thinking and telling you whatever you is or itself that it is thinking.

• 39:18Speaker 02

  And that's so weird, right? Think about it like it's self aware. The self awareness part is the most interesting part of it. So coming back to like say 3D understanding for just a second like what does it mean for humans to have 3D understanding? Actually like humans don't really, I'll take a slightly different stance than most computer vision researchers take, which is that like, oh, humans actually perceive 3D and things like that. We don't really perceive 3D. Yes, we have two eyes, we have stereo vision, but like stereo dies out at like about 20 centimeters. Outside of that, the disparity is almost nothing, right? Like and then like case in point, if you like hurt one eye and you have an eye patch, you can still drive. You have some initial trouble trying to grab a piece of glass or like something close up, but you really drive.

• 40:02Speaker 02

  Of course, you have hands in your proprioception, so the brain gets other signals, right? That teaches it like depth and that teaches it some of these things. But it's hard to really argue that the like the brain actually maintains some sort of 3D representation in the brain, right? It might not, we just don't know. We have no way of understanding it. So a generative model, why is it any more special that it has a explicit 3D representation inside it like a mesh or whatever have you? Then just understanding these concepts more implicitly or more in terms of 2D space and time. And as long as it, to John's point, as long as it's able to generate

• 40:44Speaker 02

  something that looks really consistent, why do you care how it actually did it? Like planes fly, but they don't flap their wings. The phenomena that birds use and the phenomena that planes use is actually very similar, right? Birds generate lift by by like moving the wings itself. Planes do that by pushing themselves through the air and like doing that. So while the phenomena is the same, the mechanisms are very different. Right, humanoids right now, like we are trying to build them. They move, but their movements seem very different from how we do it. Right, it's gonna happen more and more, like all machines that do things humans do, like dishwashing. We wash dish very differently than dishwasher. But is it, are the dishes any less washed? Because the dishwasher actually didn't have hands? I don't think so.

• 41:28Speaker 02

  So philosophically, coming back to it. I don't think machines have any prerogative to think exactly how we do. Have representations inside them like how we do. It doesn't make them any less intelligent. It doesn't make them any less capable. In fact, they should do it differently because the substrate they're on is very different. Human brain is a twenty watt piece of organic tissue. On the other hand, here we are running things on gigawatt scale clusters. Why should they think the same way? So that's my answer to that problem, actually. Like, I think people who are focused on making machines think exactly as humans do, making them exactly as interpretable as humans have, are misguided.

• 42:13Speaker 02

  They are not only reducing the capabilities of machines in that process, but also wasting their own time. They should use that time to scale attention better. They should use that time to like design regimes that can like do more efficient learning, like all these kind of things, right? Yeah, I think it's wasted.

• 42:31Nathan Labenz?

  All right, I agreed with like the first eighty-five percent of that, but the last fifteen percent I want to challenge.

• 42:37Speaker 02

  Go for it.

• 42:38Nathan Labenz?

  When it comes to interpretability, in terms of like, why would you care? And I am, the eighty-five percent does include the idea that like, I don't expect the AIs to be representing things in the same way that we are, or thinking broadly in the same way that we are. And I don't think that should be used to discount them. I have a funny series of tweets where it's like, it's only a concept if it comes from the concept region of the human brain. Otherwise, it's just sparkling notions or whatever. So I'm with you in terms of viewing that as a straw man. But I do think when you look at something like Golden Gate Claude, for example, where they were able to say, okay, we were able to go in, isolate what very strongly appears to be the Golden Gate Bridge concept. And now, when we artificially turn that up, we get Golden Gate Claude.

• 43:23Nathan Labenz?

  And then, yes, that's just a curiosity and invoke demo. But I actually thought when you were talking about the concepts feature that you were building, that maybe you were doing it that way, where I could imagine that you might say, if you had taken it, and I'd be interested to hear how you are doing it, if it's because it seems like it's not this. But what I had imagined you might be doing there is taking a similar approach, trying to identify directions in the latent space and then injecting them or turning them up in order to enable these different concepts. And it's like, you probably could do that and be like, turn on anime or turn on Toy Story style or whatever. Steven has a much better vocabulary for the different styles than I do. But I assumed that your concept work would be similar to Golden Gate Claude.

• 44:08Nathan Labenz?

  And I guess the question there would be like, doesn't that seem like it would be a very useful thing to study and potentially be able to marshal? And if you're not doing that, maybe can you tell us how you are doing it?

• 44:20Speaker 02

  So there's a difference between treating something as a black box and being able to do empirical experiments on it. But we are in the philosophical land again, because like we are not really, think about physics, right? Lot of people think our knowledge of the world, and it's completely interpretable. And because we have an equation, we understand how the systems work. That's not how like any of the laws of physics actually are, right? Science is not the act of interviewing God, if that existed, right? I'm not saying one way or the other. Science is basically, we have an observation, can be derive a pattern out of it. That's about it.

• 45:05Speaker 02

  Science is empirical. Theoretical physics is also, again, very much about building a mathematical model, an approximation, a representation. Machine learning is very much the same. The entire universe is very statistical. Right, like the best theory that we have in of how the universe functions right now, quantum mechanics is entirely uninterpretable. Today, we don't understand why things are the way they are at all. The measurements get verified up to like six sigma, eight sigma, sometimes ten sigma. As accurate as it gets, there are some measurements that are up to twenty three sigma, right? But we don't understand why the world is that way, right? That's what you say when you're asking for interpretation, right?

• 45:50Speaker 02

  I'm saying, ML models, because of their sheer scale, are just not grocable by the human brain. You're not gonna come up with this coherent, consistent model of how the ML model functions.

• 46:05Speaker 02

  Because it's just not that kind of system. We can do empirical experiments on it after the fact, like the Golden Gate experiment that you're talking about. Like, oh, we we found a cluster of this, we found this. It's like archaeology. We can do archaeology, all right? But here's something really powerful that we can't do in the physical world. In the physical world, we can only do archaeology. Here, we are actually involved in the formation of the planet and all the eons it went through. That's called training. So we know what the model is going to do. So like, instead of people spending their time on archaeology, I think a much better use of time is understanding data and training processes. You actually learn so much during training.

• 46:45Speaker 02

  At the risk of Xiaming giving something away here, like when we train our models, like we are working on on actually designing these models with great degree of efficiency. And in that process, we do so much information theoretic work on our models to try to understand what sort of information flow architecture is in there. Like, what is happening to high-frequency details? What is happening to low-frequency details? What like, what is it learning in the earlier stages, later stages? When should we actually change the curriculum of like what it is actually learning? This is all basically like interpretability, what people talk about. This actually has real consequences. This changes what the model learns, and some of the things we have learned are like really interesting, which is like the information theoretic pathways are set about in the first twenty, thirty thousand iterations.

• 47:28Speaker 02

  Like, the model starts out as this.

• 47:33Speaker 02

  Scalar field, really. Like, what is a good example? Think about like a field of like a maize or a field of corn like that is grown. It's just plants everywhere. They look like just uniform, all these kind of things. As you train, pathways keep forming in between them. Like someone walking and like putting the plants down, right? And then they're like crop circles. And when you look from the top, you're like, oh, there's a pattern. There's a circle and there's things like this. These happen very early. And the distribution of data you have in that time and like the kind of learning rate regimes you have used decide a lot of what is going to happen in the later stages. But in the later stages, a lot of different things happen, right? In the latest stages, like these pathways are emphasized or de-emphasized. Someone goes and undoes the crop circles. Someone goes and undoes like, well, we're not going to take that path as much.

• 48:18Speaker 02

  This is the kind of interpretability allows you to design the models that give you the output you want. So the post fact interpretability work, I think it's pretty interesting. No, don't get me wrong. Intellectually, this is like empirical science, right? Like, it's very good. And if you have an alien model in front of you or something someone else made, this is very useful to understand what's going on. But if you want to actually control the models, you want to control the data and the training process. And I mean, the actual hyperparameters of the training process. There's so much that goes into that. So that's where like actual interpretability comes into play. And that's where we do a lot of work.

• 48:54Nathan Labenz?

  Yeah, that's super interesting. So, I mean, you could expand on any number of dimensions of that. One that I've been thinking about quite a bit recently is, you might call it like batch strategy. It sounds like you are probably getting pretty intentional in those early phases of training about making sure that you are feeding batch by batch, like the right mix of data. So that, because I can imagine if you had, I've always looked at these like loss curves and you see these occasional spikes and you're like, what's happening there? And my one sort of interpretation is like, maybe that's a bad batch, some sort of cluster of data that the model wasn't prepared for. But then also, if my interpretation is right,

• 49:38Nathan Labenz?

  then that batch is probably sending a bad signal to the model as well. That's like not actually constructive for general learning purposes. So are you actually doing this sort of like batch by batch construction of data to try to get the mix right at that very granular level? Or am I over or misinterpreting what you're saying?

• 50:00Speaker 02

  I think Xiaming can answer this a bit better, but I can clarify my own statement. I don't mean like handpicking it that much, but I think we come at it from the other direction, which is a lot of duration and filtering and removing like garbage examples. The technique to training really great models is not to throw billion samples of garbage data at it. And it will figure out out of the garbage, right? The technique is show it what you want it to learn. And show it good examples from there, right? At large scale. So yeah, you do a lot of this preprocessing, filtering, that kind of stuff. But Xiaming, I don't know if you have a different take on that.

• 50:36Speaker 04

  Yeah, I think it's a very interesting question to be seen. Basically, it's like, what is the right data set, like essentially? Interestingly enough, the kind of the statistical methods that we're using like for like training these machinery algorithms are actually quite different from what the real world actually is. So if you run these algorithm, whether it is on like language models or a diffusion model, you're mostly making like ID assumptions. Basically, you have a data point, you assume that these were drawn independently and identically from the world distribution. But the real question is like, is there a world distribution or and whether like the like,

• 51:18Speaker 04

  how you even define the world distribution is a real question. So I think regarding the question of like, what is the best data set? It's actually very hard to like, like have this this right answer. So I think people probably take a second, like, you know, less statistically driven, but more say like objective driven approach. So it's like, okay, how do I control the data set such that I get a better quality or like better outcome? And that's been a deeply studied also in the realm of like, let's say like language model, for example, like how much like English do you want to have in your entire corpus? To be honest, it is very hard to reason about these in a like a theoretical level,

• 52:02Speaker 04

  because you wouldn't want to say that. Okay, we just based on how much English to be put into the data set is on like number of populations who speak English. So that's probably not the right way of like doing it. And instead, like having a more empirical focused approach is probably the right way of finding it. But the exact solution is probably very messy and there is not really a lot of like key, like concise principles behind it, despite how easy it is to explain maybe the idea of like a machine learning tool, like general purpose obvious.

• 52:36Nathan Labenz?

  Maybe, Jamming, would it be too much to ask for you to give us a short masterclass in the intellectual history of diffusion models? Because I think you're probably the best person in the world, maybe, to do that?

• 52:50Speaker 04

  Yeah, of course.

• 52:50Nathan Labenz?

  So I sketched it out. And the mantra that I always say to myself about the original diffusion models is the big unlock was, there's all these images out there. It's really easy to just programmatically add noise to them step by step until you get to pure noise. And then the insight is if you just train the model to do reverse and learn a de-noising step, then if you run a bunch of those steps in a row, you can start from pure noise and eventually get to some image. The original versions of that were like totally undirected and just generated an image out of nowhere. Then we got classifier guidance, and that sort of was enough to steer us in the direction of particular images. And then I want you to take over and tell us the major updates in the field that have brought us to where we are in terms of efficiency and control.

• 53:35Speaker 04

  Yeah, of course. So I guess, like, the first known method of diffusion models was actually reported in this 2015 ICML paper. I, it's from like Raphael Soldixstein. So he described the actual like algorithm for like how to train or how to formulate the forward and reverse process of a diffusion model. However, that never really caught up because back then the experiments were done on like very small, like MNIST data sets, and it has like still the same problems that like plagued diffusion models like very early on back then. So it was like. To researchers of that field, it seems like, okay, I have like GANs that work in one step, and it also works much better on the large scale.

• 54:21Speaker 04

  Why am I bothering myself on this method that seems like very hard to grasp and like very slow to like generate samples and so on? So the first kind of like real breakthrough on this field came then on 2020 by, which was on the paper called like Denoising Diffusion Probabilistic Models by J. Roth and H. Adol. And the basic idea is that, first of all, it validates the like the validity of this idea and made it ask like efficient or not efficient, but made it as performant as GANs on certain like tasks. So it was still very inefficient. So in the sense that you need like many of all the steps to like converge to the right image.

• 55:02Speaker 04

  But it is one of the first algorithms that does not have the problem of like the unstable training that GAN gives. So for like practitioners like us, like the algorithms being trained like stable to train is very important. Because it's a huge difference between like you lost the job, you can't go to sleep at night and waiting like until the next few like hours, the result gets better versus in the GAN case, like things just become unstable like out of nowhere. And you have to do a lot like digging in this case. So the training algorithm being stable is a very huge deal here. And this also, this is actually why people got interested in the diffusion models, even though like they were like very slow like for at their time.

• 55:46Speaker 04

  I was like part of the people who work on these like general directions. I worked on other type of zero two models, including GANs before. So like to me, like diffusion model seems a very like fresh perspective because before the existence of that paper, there was really not any like method that has the trains stably and high sample, like high quality like sample generation. So then I was trying to think more into this case of now we have this model generates high quality samples, but back in that time for SIFAR. This model is very stable to train. What is the problem of the model? So the problem of the model is that it generates very slowly. It takes like a thousand neural number iterations to get to a high quality sample. So I was more looking into like solving like that particular angle of the problem.

• 56:29Speaker 04

  And this is how I led to my work on like denoising diffusion implicit models or something that is able to accelerate the model something time to like 20 to 50x back in the days. Of course, like simultaneously, Yang Song was also from our like lab at Stanford, was working on the more generalized idea of diffusion models. So his idea was more like instead of having this like a fixed discrete number of time steps, we can actually make the time step continuous matter and wrap it more into the mass that came from like 1980s from this person called like Anderson. So it was all in this like stochastic diffusion equation framework and score matching. And Yang was like very early kind of advocate on like score matching and denoising score matching.

• 57:13Speaker 04

  So it naturally fit into his framework. So then he found this very interesting connection between differential models and denoising score matching. So that was the like kind of the first two breakthroughs, one on the theoretical level and the other is on a practical level. And the rest you have described from like OpenAI did the work on training these models on like ImageNet, which is again a big breakthrough at the time. It shows that it is competitive with GANs in these cases as well as like using classifier guidance that people are using until now. And then people start to use like classifier free guidance in the sense that they realize we don't actually need to train an additional classifier. You can add, say, this unconditional signals to the model such that you can replace the goal of a classifier.

• 57:56Speaker 04

  And this is also from like Jonathan Ho and Tim Salinas. After then, that there's a few papers around 2022, which tries to even scale it up beyond ImageNet. In early 2022, there was this paper called Glide from OpenAI, which basically tries to try to feed the models on text to image. Like, a few months afterwards, the ImageNet paper from Google came out, and a few months afterwards, the stability diffusion basically got released. So basically, this is how we got to stability diffusion. But of course, like after stability diffusion, there has been an explosion of these techniques and similarly like open source models. And then the next thing people care about is like, one, how to get higher quality models, how to make it work on videos, and how to get it to be more efficient. I will talk about more on the more efficient side of things.

• 58:41Speaker 04

  So there has been a lot of efforts, like initially to do a distillation, and there still is. But previously, the way people do distillation was kind of hacky. So the idea is that I have this, like, I've followed maybe many minutes times, like, process. I tried to use a model to represent like two contiguous steps, so that I'd reduce the number of steps by two. And then I tried to train another model to simulate that two time steps, so that overall, like, have four access and repeat this again. So this is called like progressive distillation, will also come from like Jonathan Ho. But this is like pretty tricky to train, because the implementation gets a bit hairy.

• 59:18Speaker 04

  So early 2023, there's this other paper called like consistency models, which at the time was aiming to be a replacement to diffusion models in the sense that it is both efficient and we can train it in a single stage. Of course, he also described a method called consistent distillation, which is a distillation-based method to this, like, for consistency models. Basically, you use a existing diffusion model as the base, and you try to distill it with like consistency models ideas. What actually became more popular in the field was consistent distillation, because actually when people tried consistency models training on other use cases, it turns out to be like not that as easy as it seems to train it in like very stability.

• 59:59Speaker 04

  So people came up with a new methods to stabilize the training process for the consistency models, which usually is still based on initializing the model from a diffusion model, so to speak. So then comes with this type of like distillation techniques. And there's another set of techniques that are based more on like GANs, so to speak. So the stuff between consistency models and the GAN-based methods is that the consistent distillation techniques are, in some sense, Like more stable to train, but in the GAN-based like methods are like less stable to train, but they have like maybe higher quality if you run the amount fewer steps. So again, this is this like interesting trade-off between like how stable it is to train the model and how like easy it is to get to like high-quality like samples.

• 1:00:44Speaker 04

  Back to the original story between like GAN and diffusion models again. So, you know, that kind of is the current sales call on diffusion models, the history of diffusion models, how people have scaling it up, and what are the like key problems in how to make diffusion models even faster via this path of distillation? Yeah.

• 1:01:03Nathan Labenz?

  Could we do just a little bit more on classifier-free guidance? And maybe also on what the intuition is with the consistency model? Like, when we're doing guidance, what exactly are we doing to guide the model? I get it when the classifier sense, or at least I have an intuition where I would be like, okay, feed it to the classifier and take its feedback. But when we're classifier-free, I think it's a little less intuitive. And the consistency model thing, I think, is also a little less intuitive than the simple distillation.

• 1:01:37Speaker 04

  Yeah, sure. So before we talk about classifier-free guidance, we can talk a little bit more about like classifier-based guidance. So the idea is that diffusion models, it tries to represent a score or the gradient of local probability of this distribution called P of X. You try to guide it with some like conditional signal, let's call it Y. So basically, instead of actually trying to sample from P of X, you want to sample from P of X given Y. However, this, you can train on this, of course, but another way to like treat P of X given Y is to apply the Bayes rule. So basically, the P of X given Y is equal to the joint of P of X and Y divided by like P of X.

• 1:02:19Speaker 04

  So I guess that's, basically, you can think about like P of X as the score of the diffusion model without any like condition. And the P of X and Y part is the score of the model with condition. Because like, the other reason is that the P of Y, which is the condition part, you like don't actually care about it during sampling, so you can just drop it. So basically, that is why in classifier-free guidance, we have this like unconditional model, which represents the denominator and the conditional model, which represents the numerator. But all in all, it's basically application of Bayes. A consistent model is basically something like this. Maybe it's easier to explain consistent dissipation first.

• 1:03:01Speaker 04

  The idea is that I want to have a one step model to generate the like right solution. But I want to find a way to bootstrap it from a regular like like diffusion model. So the idea is like this. Suppose I have a perfect like one step model. And it follows the trajectory of the diffusion model. So basically, the idea here is that we want to learn a model to distill the process in which a diffusion model would have normally go through with many steps. We're just like dissolving this function. So a consistent model would do in principle in the dissipation case is that you can compute the one step prediction at different time steps. And I guess time step is a concept that is correlated with how much noise you add.

• 1:03:45Speaker 04

  So you know, the more noise you add represents like a different time step or maybe a higher time step in this other case. So in a consistent model, the idea is to build a connection between like two qualities. The first quality is basically at a given particular time step. What is the prediction that you are going to make in one step? The second quality is that if you suppose you are using two steps and one step is the regular diffusion model step. So you run one regular step of diffusion model to a time step that is closer to like your original state. And then you run this like same consistency model to like reach the final step. So basically, the way how consistent dissipation is trained

• 1:04:29Speaker 04

  is that it tries to basically minimize this like loss function. And because your consistent model at the earlier times, like at the time step that is closer to the like clean signal, has an easier time predicting what a real number is. Then it allows the model to like build this connection and like train this function. But essentially, what consistent model tries to do is that it tries to use a model to distill the otherwise like hard to compute process that is run by like this diffusion models.

• 1:04:58Nathan Labenz?

  So there's so many directions we could go here. Let's do your latest contribution, the inductive moment matching, which I basically take as being a best of all of these prior approaches combined into one. There's sort of echoes of the consistency model idea and echoes of distillation with a sort of an interesting, what jumped out to me maybe most about it was the idea that the model is being optimized in distribution space and more of a, I mean, I guess it's always batch level, right? But it's at a higher level than just example by example?

• 1:05:36Speaker 04

  Like I mentioned, we want to achieve these in the generative modeling algorithms. One, it is that it has like high sample quality. And two is that it is like stable to train. And three is that it is like relatively efficient in when you are trying to draw sample from it. So most of the, like, not most, but all of the existing methods like suffer from one of the two like drawbacks. So for example, like a generative as for several numbers like GUESS are not that stable to train. Same kind of goes to some degree of consistency models as well. Diffusion models are stable to train, has high sample quality, but it can't generate like high quality samples in very few steps. So the inference cost is higher. So what we want is actually, we want to find an algorithm that satisfies all three of them.

• 1:06:19Speaker 04

  So basically, high quality generation, like fast to sample and also a stable training process. In this case, we try to actually like reason about a generalization of like what consistency models are kind of doing. So in the sense that instead of trying to match like this point wise sample. So basically I have this function and I want to exactly match the samples. What we can end up with is we only need to match the distribution wise. Because we actually don't care about exactly this function has to have match. For example, if you try to have a bunch of samples and you're trying to push them to another set of samples. There are many different solutions you can go with, but in consistency models, you are forced to follow one type of solutions.

• 1:07:02Speaker 04

  And that is actually a little bit more restricting to the model. And maybe your model needs more capacity to like achieve what it is being asked for. And this is possibly one of the reasons why, you know, these GAN based methods are like having advantage. Because the GAN based methods are actually comparing samples at a distribution level. So instead, we are thinking about, okay, instead of trying to match the samples pointwise, we can just match the samples at a distribution level. What is another algorithm that can match distributions based on samples that is not GUESS? So it turns out, actually, this idea has been discussed in the statistics community in like at least fifteen years ago. And this idea is called like a maximum mean discrepancy. So the idea is can be like heard.

• 1:07:46Speaker 04

  It is maybe a bit more scary to hear about this this concept. But what you can think of is it has a very interesting relationship with GUESS. So in GUESS, you have the discriminator that is trying to maximize the distance between like the prediction of a real sample and the prediction of a generated sample. But in maximum mean discrepancy, you do the same thing, except for that the discriminator is no longer a like neural network. It is a function defined on the space called R K H S or reproducible kernel Huber space. So this you can just think about it as a like feature representation of the function, a simple function based on like complicated like infinite dimensional features. It is a more closed form function.

• 1:08:30Speaker 04

  And what is interesting about like MMD or maximum mean discrepancy or the R K H S this choice in general is that you don't actually have to optimize the discriminator. So once you define a particular like type of space to optimize for, the optimal solution can already be like represented. So meaning that in this case, you skip the like inner loop of optimizing the discriminator, which makes this whole process more like unstable. And you're just ending up with a like very stable like optimization process that still minimizes the like distance between like distributions as you're trying to learn with a general model. Of course, the downside of that is that you chose this like a space of functions a priori. So you don't have the space to maybe optimize it to the best like case scenario.

• 1:09:15Speaker 04

  But in in our experiments, we didn't find us to be a huge problem. So essentially, the easier way to interpret how like our approach on like inductive moment matching versus what consistency model is doing is that it is a generalization of consistency models in the sense. It does like distribution level matching where I've seen consistent models. The distribution matching is based on like a single point. And of course, you can easily represent a distribution with a single point. So it becomes a more degenerate case. So this kind of explains why, like, in certain cases, consistent models are unstable to train. And we actually did operation studies controlling the amount of like samples we use to control the compare distributions. And in one sample is like the consistent model is unstable.

• 1:09:57Speaker 04

  With two samples, it is also a bit unstable, but it's becomes unstable later. But with like four samples or more, the training process actually becomes more stable. So that's how we get into this like one stage process that has high cost generation.

• 1:10:11Nathan Labenz?

  Maybe just last question, because I know we're at time. Just looking forward, you guys are obviously deeply invested in multimodality. I would love to understand like how you think about multimodality broadly and where you think it's going.

• 1:10:27Speaker 02

  We probably don't have time to explore this answer, to be honest with you. This is like the entire foundation of the company. But obviously, this which is that like currently, what people think of multimodality is not really it. These are language models that have been fine tuned to like work with image, work with audio, work with video. And it shows some capabilities which are actually really beneficial in the context of like whatever the thing the language model is doing. That's very good, but as you're seeing, i- individual capabilities like understanding of optics, like object recognition or segmentation. Or like just being able to understand what's going on, following long, long threads of like things, people doing causality, all these kind of things.

• 1:11:07Speaker 02

  These understandings in multimodal models is still worse than dedicated computer vision models. And that tells you that this might not be the approach, right? That we're not quite there, like it would be like if we built all these language models, but they were still worse than R N Ns. At like interpretation of language or natural language, right? Then that's not the case, language models are fantastic at that. What we're seeing basically is that like, yeah, this is a very promising direction. And obviously, the bag of tokens approach is extremely helpful. But we need to look beyond just language backbones and trying to like retrofit everything to it.

• 1:11:46Speaker 02

  So Luma's approach is very different and we are coming at it from the direction of like this unified singular like latent space. Where we can think and reason about all these different pieces of information as if it's one. And technically, it is one, right? Nature doesn't make a distinction between video and image and audio and things like that. And like, these are just signals. They just happen. They're all part of the same simulation. They're all part of like the same environment that we are in. And then an action produces these signals in all of these different modalities, exposing different facets of its existence and occurrence. So we need to think about it in that same way. And currently, we see that most of the industry is extremely short sighted

• 1:12:30Speaker 02

  when it comes to like thinking about multimodality. Or, well, for good reason, by the way, right? There's so much to be done in language and people should continue to do that work. But there's a new approach that is necessary to actually solve multimodality, right? And that's what we're taking. It doesn't answer all your questions, but we don't have time. We'll come again next time.

• 1:12:48Nathan Labenz?

  Okay, cool. I'm looking forward to part two already. So for now, I'm John, a Juman Song from Luma Labs. Thank you for being part of the cognitive revolution.

• 1:12:57Nathan Labenz?

  It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please, don't hesitate to reach out via email at [email protected] or you can DM me on the social media platform of your choice.