What does AI-powered content creation look like? with Runway ML’s Cristobal Valenzuela

For a long time, AI-generated images and video felt like a fun toy. Cool, but not something that would bring value to professional content creators. But now we are at the exciting moment where machine learning tools have the power to unlock more creative ideas. This week on the podcast, Sarah Guo and Elad Gil talk to Cristobal Valenzuela, a technologist, artist and software developer. He’s also the CEO and co-founder of Runway, a web-based tool that allows creatives to use machine learning to generate and edit video. You've probably already seen Runway's work in action on the Late Show with Stephen Colbert and in the feature film Everything Everywhere All at Once. Show Links: Watch Cris Valenzuela’s 2018 thesis presentation at New York University’s ITP program. Read how Runway is used on the Late Show and in Everything Everywhere All at Once on the Runway Blog. Sign up for new podcasts every week. Email feedback to [email protected] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @c_valenzuelab Show Notes: [1:50] - Cris’s background and how he doesn’t see barriers between art and machine learning [6:46] - How Runway works as a tool [8:36] - The origins and early iterations of Runway [12:22] - Product sequencing and roadmapping in a fast growing space [15:43] - Runway as an applied research company [19:10] - Common pitfalls for founders to avoid [22:35] - How Runway structures teams for effective collaboration [24:22] - Learnings from how Runway built Greenscreen product [28:01] - Building a long-term and sustainable business [32:34] - Finding Product Market Fit [36:34] - The influence of AI tools in art as an artistic movement

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:07Speaker 00

  I was speaking with this director who was like working on a film, was using Runway. And he came up with this idea of like, when he was chatting with his editor, he was like, we should just Runway that.

• 0:18Speaker 00

  I'm not constrained by the time and the cost, I'm constrained by whatever idea I think works the best. And that's just phenomenal, right? And so our goal, and I think our goal still is, is not to like build this like kind of like autonomous like systems that don't engage in any sort of like relationship with humans or with creatives. On the contrary, it's like you have humans coming up with great ideas and they want to express those ideas. How do you build systems that will help them get there really quick?

• 0:49Sarah Guo?

  This is the No Priors podcast. I'm Sarah Guo.

• 0:52Speaker 02

  I'm Alad Gheil.

• 0:53Sarah Guo?

  We invest, advise, and help start technology companies.

• 0:56Speaker 02

  In this podcast, we're talking with the leading founders and researchers in AI about the biggest questions.

• 1:07Sarah Guo?

  We're thrilled to have Cristobal Valenzuela on today's episode of No Priors. He's the CEO and co-founder of Runway ML. Runway is a web-based tool that allows creatives to use machine learning to generate and edit video. You've probably seen Runway's work in action. Visual effects editors have used Runway to create visuals on The Late Show with Stephen Colbert and the movie Everything Everywhere at Once. Chris, welcome to the podcast.

• 1:29Speaker 00

  Thank you for having me here. Super excited to chat with you.

• 1:32Sarah Guo?

  So can we start all the way back? I think you are the only person I know with degrees in economics, business, design, and then also went to art school. How did that happen? And then how did you stick an interest in ML in there that became very real at some point?

• 1:45Speaker 00

  Yeah, that's an interesting question. I've always been very curious about just things in journalism. And so we're trying to like find ways of channeling that curiosity. And I'm originally from Chile and I studied in Chile, a combination of like business and econ, and then went into design. And it was a very particular design kind of like program. I spent a lot of time with physical computing, which is like Working with hardware with like electronics mostly applied to design and like art. And while I was doing that, I was also consulting. So it's kind of like I had I for a moment I thought I had like two lives. I was like doing art on the one end with like Arduinos and electronics. And on the other side, I was like consulting for these banks, which is very like

• 2:29Speaker 00

  different, but I love it. I think it's perspectives and worldviews that are very opposite. At the same time, you gain from being at both. Long story short, I kind of like fall in love or was experimenting with early computer vision models in twenty sixteen, fifteen. And then went into rabbit hole, apply and got a scholarship at NYU and then spent like two years in in art school. ITP, that's the name of the program. It's a very unique program for me, though. It's very fundamental, kind of like piece in in my career of like understanding how to bridge business, design, art and and and technology in a cohesive way.

• 3:04Sarah Guo?

  Amazing. And now you have one life that combines those. But in the art side, how should I picture Arduino electronic art?

• 3:12Speaker 00

  Um, media arts probably is the best way of describing it. I think for me, media arts is a way of like expressing a worldview using technology like any other form of art, like you just kind of like experimenting and reflecting and and and expressing a worldview using a piece of like a tool. And in this case, like it happens to be that we like to express it via like computers and software and writing software is a form of art and writing, making hardware is also a form of art. One thing I I remember early on in my career, when I was doubling between like art and business. I met this very famous Chilean artist that sees a photographer and he was just like mentoring me and like we were chatting and

• 3:56Speaker 00

  and he was speaking to me, he was like, Chris, this is the same world. We all live in the same world, right? It's the same, right? We just build like silos and like arbitrary definitions of what is what. I think he just said it. I really like stuck with that. And and I think that's that's how I like to look at the world. Like it's it's just the same world. You're gonna apply different points of views and perspective on on that. We build arbitrary like definitions of like this is this is art and that that's that's like design and that's econ and that's business. But I think true creativity and curiosity comes from just like looking at it as a whole and taking things that weren't supposed to be part of one thing and then adapting them. And sometimes it's hard because you need to learn things that you've never done before and it's uncomfortable and it's perhaps you feel like an impostor, like you haven't,

• 4:40Speaker 00

  you shouldn't be doing this. I've learned not to care, to be honest. I just like, you just drive by curiosity, like you'll figure something out and I really like that.

• 4:48Speaker 02

  That's super cool. Yeah, it seems like a lot of the history of Silicon Valley actually ties in really closely with art and the art scene. So if you go back to like the Stewart brand world of the seventies, or some of the early things that were being done on the Mac, or even look at some of the people in technology where the art side of them is understated. You know, like Paul Graham obviously wrote a whole book on this, Hackers and Painters, and is a painter himself, but there's people like Seth Kombar, who started a company that Google bought and has done a lot of crypto related things. He was a co-founder of Cello and he's exhibited digital art at the MoMA as well. And so it just kind of feels like it's almost under discussed now in terms of this overlap between technology, art and the two scenes, except for, you know, occasionally when people go to Burning Man or something, they bring it up. Other than that, it seems like it's very under focused on.

• 5:27Speaker 00

  Yeah, I agree. And for me, it has always, to be honest, a bit new. Like I've been in New York like six years and a runway now is going to turn four years. And I was also new to like just the tech world and SF, like I've never been to SF like three years ago, right? So I'm relatively new to the space and but I think what what I just, how we approach it was with that same level of curiosity of like, I'm going to figure it out, I'm going to learn about it. And I think that there's two like sides of that. The one is that it takes time for you to adapt to that because it's just new. Like everything else, it's just you need to understand it and it doesn't show the patterns of that, that subject, that domain, that area, right? But at the same time, I'm looking at it with fresh eyes, with things that I've perhaps the ecosystem itself has considered like norms.

• 6:10Speaker 00

  I don't consider them norms. Like I've just like want to try new things, right? And I think that opens the door again to do new things and experiment with new things. And that has, I think, Being a consistent path in both my career, but also in Runway as a whole, but we look at things, we try to look at things with very fresh eyes and like pretty much with like a first principles kind of like mentality to it is like, okay, why are we doing this? Like, but really, why? And to go to the basic aspects of it and then innovate. I think that's a lot of innovation comes basically from that way of looking at the world.

• 6:42Sarah Guo?

  Runway is, I think, a very creative shape of product. It's not the kind of product you can come up with if you're just like casting around for a good idea. It obviously comes from creativity and discovery. And maybe what you could do for our listeners, actually, just can you explain how Runway works as a tool? And what people do with it to set context?

• 6:59Speaker 00

  Yeah, totally. Also happy to set a bit of context of the company itself. So I think that better helps contextualize the product itself. The best way of describing Runway, I would say, is to think about it as an AI research company. We do core fundamental research on neural networks for both content creation and video automation and generative models. We then transfer those models into an infrastructure or a system to deploy those algorithms and systems in safe ways and in ways that will make us build products that are useful for people, right? And those products can take different shapes and forms. We have around 35 different, what we call AI-powered tools or magic tools. And those tools help serve a wide spectrum of creative tasks.

• 7:44Speaker 00

  From traditional like editing, editing videos or just audio or images has been a very expensive, time-consuming and sophisticated process. And so we build systems that help you do that. So we have tools like Green Screen, for example, which a lot of broadcasting companies and film studios and post-production companies use to reduce the time of rotoscoping, which is if you ever speak with a filmmaker, that's the one thing no one's doing, just no one's doing, but you have to do it. And so we basically just help you reduce that time. And we also have tools that help you ideate and design and craft. And we have a set of suites for generative image editing, for generative video editing.

• 8:25Speaker 00

  So it's the best way perhaps to think about it is it's a creative collection of tools and systems that just help you augment your creativity in any way you want.

• 8:33Sarah Guo?

  From an origins perspective, like you had this thesis project, which were all of these creative tools. And it's really, I remember like watching the presentation, it was around accessibility of, you know, the increasing number of algorithms that could help people in this sort of creation and editing. Process for different modalities. And when we met in 2019, you framed it quite differently as this kind of desktop app store for ML models. Can you talk about the iterations from that collection of algorithms you were experimenting with to like the app store idea to where Runway is today?

• 9:04Speaker 00

  Yeah, totally. A lot has happened, I would say, over the last decade or so. When I started building Runway, it was perhaps like the AlexNet, ImageNet kind of like moment. There was image classification was the kind of like the big thing and the breakthrough. And a lot of interesting applications were coming out of that time. But still very early, like TensorFlow was just perhaps a year old. PyTorch might not even have been released at the time. I think PyTorch was 2016. GANs were just like very early, early, early, like inception time. But what I kept seeing was like, there's, there's this neural like aesthetic, this neural like capabilities that are impacting not just like the visual world or like the perhaps industries and markets like self-driving cars that are using a lot of this like technologies and hardware.

• 9:47Speaker 00

  But the outputs are very interesting from a visual perspective, right? There, it seems to be a correlation and a approximation towards the, the visual domain. And so I started just experimenting with, what, what does that actually mean, right? What do you mean by, how do you experiment with this sophisticated algorithms that were very early? They had all this like obscure CUDA dependencies and C++ like libraries that were just very research centric because they were basically research, right? Like core research. But I was just fascinated by the outputs of the research elements. And at the same time, I mean, everything I would say that we consider like, uh, like a baseline today wasn't really like there yet at the time. Things have progressed radically.

• 10:28Speaker 00

  The space has been growing exponentially, but systems and like software and obstructions to tap into that potential wasn't really there. So our first intuition and our first kind of like product experimentation was let's build like thin layer, right? Basically, let's take this, this research set of models and the amount of models that are coming up. It's just like so interesting. Let's add a thin layer of accessibility to those models, specifically target and aim at creatives, right? And so if you're a designer, a filmmaker, an art director, a copywriter, you might want to tap into some of these things. So you want to experiment with them, but they're just very hard to get started with. So we built what at the time was a model directory. It's an app store of models, right?

• 11:11Speaker 00

  We had around at some point like 400 different models. It was one of the first, like, I would say, model hubs. I think there are a few out there now that you can tap into and use them. This was very, very early. And we built a whole system around it. We built an SDK. We built systems for deploying those models into real-time applications. So we built a RESTful API systems where you can use a model, train a model, and then deploy that model. And so people were building web apps and like interactive GPT-1. Someone was training a model and like fine tuning a GPT-2 modeled on a specific corpus of data and then creating an API to build a text generation app. And we had all these very interesting layers of applications that, to be honest, for us was just a way of learning,

• 11:54Speaker 00

  learning a lot about the space and a lot about what was visible, what was possible, who was kind of like interesting in like building more of this. And from there on, we kind of like continuously iterating. We've learned a lot from that model registry or like model hub. We still use a lot of those in our infrastructure kind of like parts on the app. But also we gathered a lot of insights on how to build these kind of systems in scalable ways.

• 12:18Speaker 02

  How did your technology stack or the approaches that you took transition over time? Because I think when I look at the evolution of the area, to your point, you know, a lot of people are doing like CNN and RNN-based things and GANs and all the sort of early things in neural networks. And then, you know, the analogy may be, I know a lot of people who started companies right before AWS launched and their whole like infrastructure stack got stuck on the past set of approaches. And then later a subset of them transitioned on AWS and a subset just continued with their own private clouds. And I'm just sort of curious how you thought about it as, you know, obviously diffusion models, I think, were invented around 2015, transformers 2017, but it took a couple of years for all this stuff to catch on. And so when did you start transitioning architectures? Or have you or how have you thought about this sort of whole evolution of the field relative to the tools that you provide

• 13:03Speaker 02

  and reinventing them over time and everything else?

• 13:06Speaker 00

  No, that's a great question. Something actually we think a lot about when you think about product sequencing and roadmap, which is just, I would say, one of the most important aspects of product building. It's like, how do you sequence everything you have to do? And specifically in infrastructure, like what makes the most sense and how do you spend time? Every single day like means a lot in a startup. I think for us was a few realizations, to be honest. One is that the moment something gets released, like, let's say transformers or a particular piece of technology that you think would be interesting, would be worth experimenting with. I think it takes a collective. Set of months, like 12, 24 months sometimes to understand the implications of that. Right. And we've seen this with like language models. GPT three has been around for some time, but it took like a collective 24 months of like

• 13:49Speaker 00

  just tinkering and experimenting to truly understand like, okay, where can you go? And what can you build and what's possible? So I think that we embedded that and we always keep that in mind. The second thing I would say is things are changing really fast. Right. And so if you're thinking about building a long-term business and a long-term product, which which we are, you always kept the desire of like, okay, what are long-term bets versus short-term bets? And I think a lot of building and software engineering and developing products is just saying no to a lot of things. There's customers might wanna ask you for to build something and could sound good. It could bring you actually revenue and some growth, but it actually might move you away from like a more consisting long-term plan. I think for us was was a recession of those kind of like things.

• 14:33Speaker 00

  And then the third one I would say is the third component of how we think about that stack is really understanding our users, right? Who are we building for? And so early on, it was more a technical product. So you had to know CUDA and Docker containers and managing your Docker Nvidia GPU cards. And like you have all this like sophistication that I think it's in some part natural when you things are so early, because it's the only way of making sense. And also you have to build more things. But for us, we've always been thinking about artists and filmmakers and creatives are hard and really those things don't really matter that much. It what matters is like your idea and how you execute that idea. And so from the stack perspective, we've iterated a lot on the the kind of like backend side of things.

• 15:16Speaker 00

  But from a user perspective, we did it even more on how to present those things and what abstractions and metaphors you need to build to really aim to solve the things that you wanna solve. But yeah, it's a fast growing space. So there are a lot of things that are changing.

• 15:29Sarah Guo?

  In an area where the research, like nobody can keep up with the papers, right? The progress is mind blowing and has been. You referred to runway as I think an applied research lab. Is that the right term?

• 15:40Speaker 00

  Yeah.

• 15:40Sarah Guo?

  Like, where do you decide? Given the progress in the community, like when you need to do in-house research and push the state of the art versus exploit what's out there?

• 15:49Speaker 00

  Yeah, I guess going back to that set of learnings early on, I think one thing that we realized is models on their own are not products, right? A model is a research component and taking a model and productionizing that model, it's a different problem that actually building one single model, right? Or one single task or problem or improving a metric in a specific kind of direction. There's a lot of nuances of how that model will get deployed, it will get built, how users will interact with it. The unit economics of running this kind of like systems as well is very important, right? So they have all these complexities. And as we started leveraging perhaps open source solutions at a time or trying to build our own, we quickly realized that having control is like key.

• 16:31Speaker 00

  Like you need to be sure that you can understand your stack and you can understand and know how to fix your stack, right? Because if things are changing really fast and you think about going in one particular direction, but it then happens to be the case that there's a break to somewhere else, you need to react really fast, right? And you need to be able to incorporate that. And if you're just relying on third parties or some other solutions, then it might be very hard, right? And so for us, it was a survival realization that if we really want to make and move the standard of creative tools in the ways and vision that we had, we had to own our stack. And so we started building this research team, right? And this research team has very deep like

• 17:13Speaker 00

  understandings and knowledges and perspectives on how to build models. And we've done this when we've collaborated and contributed to like breakthrough moments in like the creative AI space. But most importantly, we have these researchers working really closely with creatives. Half of our team have arts backgrounds, right? Which is, which is very unique. And we've put a lot of emphasis on finding those, those very unique, like they're very hard to find folks that can speak both worlds, right? I just went back to the world analogy. And so in one single table, you can have a PhD scientist that's been contributing to like fundamental research on the space, working really closely with someone who's working on video for 20 years, right, who's been editing and post-producing films or content, right?

• 17:58Speaker 00

  And the things they learn from each other is just so, it's so unique. It's so radically different and it helps inform how we build products, right? And so we don't treat research as a standalone kind of like department that comes every six months with here's a paper and just like do something with it. We see it as an applied thing. It's like, it's at the core of who we are and like how we drive the product forward. And it helps just drive the product in a different way. I think that the only thing I've learned is that building that muscle takes time, right? It's not that something you can just like, I'm going to hire a bunch of creatives and a bunch of researchers and just put them in a room and like you figure something out.

• 18:35Speaker 00

  It's, it's a lot of learning and a lot of processes and like frameworks of how you make decisions, how you understand what's really possible versus what's feasible. And there's a lot of just nuances of how to do that.

• 18:48Speaker 02

  It seems like there's a lot of founders now who come from the research community in the AI and ML world. And you know, you've navigated that extremely well in terms of saying, okay, let's be very product centric and yet still capture the best of what new technology has to offer and new research has to offer. What do you think are common pitfalls that research centric founders should avoid or things that they should think about more as they sort of start their own companies?

• 19:09Speaker 00

  Yeah, I think it's just phenomenal to see like that progression of more researchers that being perhaps in academia for too long, progressing or moving into just the operational world, like building products. I think it's a great realization of you're working on something for six, eight months a year. And but you see something else in the world of someone using something very similar to what you just built and impacting the world in very meaningful ways. I think that's great to see people transitioning more. I think we need more of that. I still think that there's a lot to be learned around the difference between a model and a product. And again, there's a lot of back and forth of how you embed models into usable products.

• 19:49Speaker 00

  And so coming up with training a model or improving some sort of quality or benchmark in some particular way, even if you have a very cool demo, it's a long way to go to like actually build a business and a reliable system that will continuously iterate over that. And so I think having that more product perspective is always just a good. And Releasing and working with real people as fast as you can. I think that's just key. I think a lot of researchers just assume how people work and how creatives work. And so we'll just do that. But the realities might be very different. And so having tools being used by people is, I think, the best way of learning how to develop products.

• 20:25Speaker 02

  Yeah. Are there specific areas of research that you're especially excited about when it comes to video or images right now?

• 20:30Speaker 00

  Yeah, for sure. I think, I mean, everything we've seen on the explosion of diffusion has been just so, so exciting to see. I think I'm particularly excited about multimodalities and like combining different input or like outputs in ways that they are yet to be explored. I think we're moving away from like very siloed domains. So like someone who could be an NLP researcher and computer vision researcher, right? I think we're like starting to see them gradually converge and mix. And so building a diverse team that can understand like those multi domains is really interesting. And I'm excited to see how that's going to play out in video and, and in images. And I like to think also of how you translate, again, and go back to product, I'm a product obsessed. But how you translate that into products that are, that are useful, right?

• 21:12Speaker 00

  I think a common natural evolution of just the creative stack or the creative software solutions out there, they tend to be very specific to domains of, of content. So you have a tool that specializes on like image editing. And then you have a tool that specializes on vector graphics. And you have a tool that specializes on motion graphics, which is different from video editing, which is different from like compositing, which, and you have all this like very sophisticated software stacks. And I think the, the very interesting aspect of what I would like to see and what we'll probably see more with multimodal systems is that you're able to merge all of those. And what I really find interesting about that is that's how we humans think, right? You don't go to a movie and watch the movie, the video first, and then you stop and you hear the audio and you just stop and you read the subtitles.

• 21:58Speaker 00

  It's a combination of all of those things, right? And our art director thinks in, all of those things at the short time as well, right? So having systems that can translate ideas and text descriptions into videos and then having a precision with what's the input of those videos into like audio. And then I think that's the, the kind of like creativity instead of tools that I'm really excited to discover and build.

• 22:17Speaker 02

  Cool. And I guess, how do you organize your product efforts? Because I think to your point, you have a really unique approach in terms of, you know, effectively turning research into products or being product centric in terms of what you're asking from the research organization. Is there a specific structure, you know? For example, at one of the companies I started, Color, we basically would embed somebody with a very deep bioinformatics background with a systems team. So that they basically inform that team around the needs of what they had and then the rest of the team would build it. And it sounds like in your case, you have people who are in both worlds. Is there a specific structure where you're like, I always put three, you know, full stack engineers with a researcher, with a product person, or the researcher is the product person? Or how do you approach all that?

• 22:53Speaker 00

  Yeah, we're a small team. We've been consistently historically small team. And until like two weeks ago, we didn't have a product person. Product was led by a combination of research, design, and engineering. And I think that drives a lot of fundamentals of truly understanding the things that need to be explored. We've iterated a lot on building squads, or being like teams, or having more autonomy. I think it really depends. I think you have a, you tend to have a different company every like four, five, six months. If you've successfully built stuff, it's a continuous like process. And the thing that worked when we were like five people sitting at a table, it's not going to really work when you're like 20 and you have new technologies and system things available. And so I don't think there's one answer in particular. I think we're pretty much with

• 23:38Speaker 00

  how we think about product, we like to iterate a lot. Right now, we're working a lot with squads. And so we've come to a kind of like a place in time where the organization can have a bit more of like domain expertise. And like, instead of having like very generalized engineers, we tend to like more specialize a little more. So you can still jump and be and collaborate, but you tend to have a bit of a focus of area. And we're iterating with that and seeing how that works.

• 23:59Sarah Guo?

  Maybe we can talk about an example of like what that iteration looks like. So you mentioned rotoscoping and like green screening as like a, like one of the magic tools that Runway creates. When we were building that feature, like what was hard? What were the iteration processes like?

• 24:16Speaker 00

  Yeah. I think that green screen is a great example of how to build and how to deploy useful AI products at scale. When we were building that model directory and we're just like early stages of understanding limitations and capacities and directions, we quickly realized that the type of like user that was coming for segmentation models, right? And at the time, we didn't have a green screen tool. It was just like a image segmentation model. And those folks were coming from a specific domain, and they're actually applying a model that was image based into a video task. And so they were like exporting themselves with FFmpeg, creating these like sequences of images to then render them back in video. And it was like, why are you doing that? Why are you, what's going on? And the thing is like image models don't really work really well with like video.

• 25:01Speaker 00

  And so we started interviewing them and we got to a point where it seems like this could be something we could like improve. And we're bringing our research team, so we started like iterating more on that. But no one ever asked for one click solution for green screen, right? If you ask people what they wanted, they wanted a better alternative that was faster to create mask from their current stack, right? And they're probably using something like rotor brush too, right? So whoa, what I would really like will be like a better brush to just brush over my frames, right? And I think customers and people are really good at telling you like what their problems are. They're really hard at verbalizing like solutions. And so you aggregate that amount of data, you see what's possible for research, you see you chat more with people and you start productizing a lot.

• 25:42Speaker 00

  And then we came to like the realization that we could build and we have the expertise to build a system that will help you automate that, right? And most literature around video object segmentation, which is that in filmmaking is basically known as rotoscoping or green screen, right? Was around like fully automated systems, right? You fit in a video and the video automatically like understands like subjects and then rotoscopes or segments, right? One specific central object or two, let's say. But just a few minutes of chatting with a professional filmmaker, you'll probably discover that that's rarely the case. Because the shots, the scenes and the compositions and the camera angles really depend. If you have a shot of 10 people, you might want to rotoscope the one on the left. Depending on your idea, right? It's a creative tool, right?

• 26:27Speaker 00

  So it should be general. And so what we did was we've instead of like relying on fully automatic systems, we embedded a human in the loop, kind of like component in it, right? And we thought it would be great if before you start doing that, you can guide the model. Like you can tell what kind of like selections or areas of the video, then you can zoom in and define you want, right? And that also really helped us train the model because we train a model on, we build a probabilistic model of like human simulated, like human clicks on a mask. And the model was trained on that knowledge, right? From the very bare bones. And that helped the product itself because people were using that model in that particular way. And that realization was, I would say, like a combination of different things.

• 27:12Speaker 00

  There was some research knowledge and understanding of what was feasible. Can you build that segmentation model, what data sets and what do you need to do it? Who would be using it for? How are we going to test if it works? And the first version of green screen was working at like four frames per second, right? It was like incredibly slow. It was like not as good as the one we have now, which is incredible. But it didn't matter. It was significantly better than anything else that was at the time, right? And people were like scrambling to use it just because it proved to be a percentage of amount better than anything out there, right? And people were hacking things and we're trying to like incorporate it's like great. That means that like that you hit something and then we started iterating a lot.

• 27:53Speaker 00

  And so we keep iterating a lot on it, but the fundamental piece of how we build product is still pretty much similar to that.

• 27:59Sarah Guo?

  Very cool. Let's zoom out and talk about runway as a business. So you, as you said, now you're very intent on building like a long-term durable business. Who uses and pays for runway today?

• 28:09Speaker 00

  We're devoted to like storytelling and like creative exploration and ideation. And that's a wide spectrum of people who are, you can consider work in the storytelling business, right? On the one end, you have professional, really professional people that have been doing this for years, right? Folks working in post-production agencies, VFX agencies, broadcasting companies that are creating video as their main business. Like this is basically what you do, right? It's entertainment, it's sometimes sports.

• 28:37Sarah Guo?

  You know, that's kind of counterintuitive because like one of the sort of beliefs of many people who will look at the research, which is fast progressing is like you can't get the quality level for like the sort of highest production value type assets with today's research. So it's really interesting that like, you know, you're talking about VFX studios and, and sort of that type of content.

• 28:57Speaker 00

  Yeah, I think, I think the realization for us is like what the goal is, right? If you're trying to automate the entire process of like the whole end-to-end system of making a movie. Yeah, like we're not there, right? We're very far from that. There's a lot of things to be, that have to be developed, that have to be the like research and kind of like understood and tested. But going back to the green screen, if, if you look into the processes and the nuances of how video is created, and you look at the inefficiencies of how people are doing it right now, and you offer these people like a hundred, even like 10% or 20% or like whatever percentage of like speed and cost reduction, it's just so radically better, right?

• 29:37Speaker 00

  And it's radically better for two reasons. Of course it has helped reduce the cost of like, you can do things faster, so it's just easier. At the same time, you can explore creatively more, right? And this happens a lot. I was speaking with this director who was working on a film and was using Runway. And he came up with this idea of like, when he was chatting with his editor, he was like, we should just Runway that, right? Just Runway the thing that you wanna like do. And before Runwaying something, they had to marry themselves or like just lock one specific idea, right? Because if we try to do two other things, it's gonna take us too much time and we just don't kind of afford that. Like every creative is always on a deadline.

• 30:16Sarah Guo?

  It's very waterfall era, right? You must choose a direction and do the whole thing.

• 30:19Speaker 00

  Exactly. And now, now he was telling me like, now I can do the three, right? I can just see the three and pick the one that I like the most, right? I'm not constrained by the time and the cost. I'm constrained by whatever idea I think works the best. And that's just phenomenal, right? And so our goal, and I think our goal still is, is not to like build this kind of like autonomous systems that don't engage in any sort of like relationship with humans or with creatives. On the contrary, it's like you have humans coming up with great ideas and they wanna express those ideas. How do you build systems that will help them get there really quick, right? And sometimes what you need is to get 80% there, 90% there. And in research, going from 80% to 100% is really hard. I think that you'll be seeing that in like autonomous vehicles where like it's always like two years ahead and always 80%.

• 31:04Speaker 00

  But like that 20, 10% is just really hard. But in, it's really hard in that domain because if there's a 1% failure, someone might die, right? In creative domains, it's not the case. Like even if you're 80% there, the 20%, like sure, I mean, you could worry about it. I can improve it. I can like find ways of work with that. But you've, you've made incredible progress, right? From that perspective.

• 31:25Sarah Guo?

  I think that's actually an interesting like filter for what domains are interesting for applied research today. Like areas where there's built-in tolerance for, you know, lower levels of accuracy is one way to look at it.

• 31:38Speaker 00

  Yeah. And you always integrate with, uh, there's ways of like combining existing tools, right? So for Rotoscope, for example, you can get Eighty percent there. And then if you're a professional filmmaker working on Nuke or Flame, you can do the 20% in that stack, right? But you still save yourself like days of work, right? So it's still better than anything you were using before, right? So this depends a lot. And I think over time, more models will get to like higher numbers and will have higher outputs. But there's a lot yet to be developed, and I think we're still scratching the surface of what's coming.

• 32:10Speaker 02

  What was the moment? You know, you mentioned there's an evolution, both in terms of the number of tools you provided, as well as their relative quality in terms of, you know, 80% versus more or less and things like that. Was there a specific moment where you really felt that you had product market fit? Or where you felt that, okay, this is something a lot of people want and they want to use? Is that, was it immediate? Was it after a specific tool came out? Like, when, when was that moment for you?

• 32:32Speaker 00

  Yeah, I like to think of product market fit as a spectrum of like, you have either really strong product market fit or weak product market fit. And as you build new products and new research, you're always seeking to be very on the strong side of things, of course. I think for us, there were a few factors that we've kind of like realized that what we were building was beyond just like a niche. Because I think we started with a very niche like audience and everyone like dismissed a little bit of what we're doing like as toys. She's just sort of like art student building like some toys. And I think you shouldn't dismiss toys. Toys are like very interesting to learn a lot. And I've learned that over time. But it's by the time you're building those, of course, it's just like you're focusing on the output and they're glitchy and they're like abstract and it's just weird. I can't make sense of it.

• 33:13Sarah Guo?

  And it's only 128 by 128 pixels.

• 33:15Speaker 00

  Exactly. Exactly. I was, I was actually, I remember like, we had a version of like fairly early like gun system that did text to image translation. We actually still have them online. And the outputs were sort of like this 128 pixels, like exactly what you're saying. Images that were just blurry. It looked like abstract paintings, right? It was just like, I mean, you typed, I don't know, a blue ocean and you get like a blue form with something. So we see, if you close your eyes, like 10 meters away, maybe.

• 33:39Sarah Guo?

  And you saw beauty or the future.

• 33:41Speaker 00

  I really like it. But at the same time, I remember like showing it to advertisers and like I went to like this, like, say, the meeting at this top agency in New York. And it's like, here, guys, here's the thing you will be using to work, right? And they're like, Chris, this, this is a toy, like, great. I mean, fascinating technology, whatever, but like, we have work to do. Come on, like, move on, right? And I think the main mistake for me was like, you're looking at the singular moment in time of that technology. You should really be looking at the rate of progress, right? That thing that I can type a word and send an image wasn't visible a year ago. Just it didn't exist, right? Now we have this. So just compound and try to like imagine where we'll be in like four or five, six years, right? But the thing is, it's really hard because you can't imagine it.

• 34:26Speaker 00

  And I remember people at the time, like when I show some of those demos, specifically for journey models is people are asking me like, hey Chris, how are you collaging these images? Like you're taking existing images and you're pasting them together, right? And I was like, no, this is, you're generating them. This model has learned patterns around for sure a data set. And you're then generating them on the fly, but these images don't really exist. Just don't exist. And so there's a lot of, I think, mental models that need to be adjusted to really understand it. And we've been adjusting that, that those mental models. And from a product perspective and from a product market fit perspective, I think there's the right moment for the market to use technology. And I think that moment has matured and we've seen it more as more people have been exposed to journey models and the potential of them.

• 35:09Speaker 00

  And for us, it's still like, there's a lot to build and to develop and to kind of like, you know, improve. But a few realizations were like when people were starting using Runway as a bird, right? You just run with that. Okay, that, that means something. Then you start seeing people just like creating tutorials and like speaking about their product online, right? With no, we don't, for a long time, never had like a marketing team or a content strategy team. Like everything was just basically people making things and then sharing them online. I think that really drives, drives, I would say, realizations. Okay, we're into something like people are using this every day. They're coming and they're sharing with their friends, right? And they're, they're thinking about it every day and they're like, I remember Arnaudius and a person who early Runway adopter, which just fell in love with a product.

• 35:51Speaker 00

  He sent me like, he painted a picture like, and he just sent me the picture to my home. He's like, here's like, just once you have the first piece I ever made with AI. And that was like 2018.

• 36:01Sarah Guo?

  What was it? A cat?

• 36:02Speaker 00

  Uh, no, it was an abstract painting. It was like he, he generated something that was very abstract, right? And then he painted a canvas and then he used like mixed techniques to just like improve some sounds and like change some colors. It was very new and novel at the time. It's like, wow, that's just, I don't know, interesting and fascinating.

• 36:19Speaker 02

  One thing I'd love to get your perspective on simply because you have such a unique mix of, of background and skills and customers and everything else is, you know, there's this emerging debate in the art world about the role of AI in art. And I think if you go back through art history, there's always been ongoing questions and contentious, not just around technology and art, but the role of an artist. Relative to the art they create. And I think like the sort of old school canonical example was Marcel Duchamp signing the urinal with Armut. And I think it was called the Fountain or something, right? It was a piece that he submitted and it got refused and it created a bunch of scandal at the time. Or Andy Warhol had the Factory and other people would assemble a lot of the art, actually, with him overseeing it. And

• 37:03Speaker 02

  so it seems like there's been a long history of different approaches to art that, at the time, seemed very controversial. And now you're just like, yeah, of course, that's how you do things or how things were done. What do you think about the debates right now in terms of art and AI? And what do you think of the important threads that people are talking about? And what do you think of the areas that in 10 or 20 years, people will look back and say, yes, it was just part of this art history debate, but in hindsight, wasn't really that important?

• 37:27Speaker 00

  Yeah. I like to think a lot about what, I guess, previous moments in history and time, as you were referring before, that has taught us something about how to both understand art and look at the tools that we use for art. For me, art is a way of looking at the world and expressing that view of the world in a particular way. And an artist's role, I think, should be to explore and experiment with different mediums that would allow you to express that in the best way you think possible, right? And so people experiment with different techniques and different systems and different structures and pigments and tools themselves, right? Even before Duchamp and even before Warhol, you had previous

• 38:09Speaker 00

  moments in times where technical revolutions enabled people to look at their world in very different ways and then express those views of the world in very different ways to what was visible at the time or possible at the time. And an example I go back to often is this idea of the 1700s, like before even painting was a massive thing that you can do in any condition, situation, or location, painting was the realm of these very sophisticated painters that were painting in studios, right? Painting was the realm of people who can afford and were able to understand and master the techniques of the masters, right? And more importantly, from a tools perspective, it was really hard to get pigments, right? It's a very practical thing, but you couldn't just, pigments didn't exist. You could just

• 38:51Speaker 00

  go to a store and get red, white, yellow, and I'll paint something. I have a canvas. The way you mix pigments was this very sophisticated thing where you had to hire a master that knew like this obscure techniques and you were like maturing them and then you you store them in like this sophisticated bladders and you seal them and it was incredible like complex and expensive process. And then someone was like, hey, we should just like build a tube and then like have this and carry it around, right? And this maybe it's easier. And it was and it was it was a very radical innovation, very simple at the time, it's very simple for us now. But that would allow was like for a whole new generation of artists to look at art and be like, great, I'm

• 39:33Speaker 00

  gonna take this painting and there's some montine that I really like there. You don't have to sit on a canvas. I'm gonna paint in plein air, which is that which is which is the thing you paint in plein air, right? You're painting in air, in the in the wild. And you're able to look at the world and the sky and you're able to like quickly brush the light, right? And this being outside of the studio was just not physical. It was just ten like before that. And then gave birth to Impressionism, right? And impression was like a whole revolution, like Impressionism was not really well received, because it's like, hey, this is this is not art. This is like, these are just brushes of like, things like they're not, I mean, no, right? And then Impressionism really started to beg up, people like started to really understand the medium. And then it evolved, it

• 40:17Speaker 00

  continuously evolved and evolved, right? And you find similar moments in time where the paint to battle for becomes relevant and like photography for me was a very similar one, right? And then cinema for sure. And then the digital world, the tradition to like film to find out digital is another one. And every single step of the way, you have artists experimenting with this analogy and using them to put a perspective of the world. I think right now, what we're seeing right now with AI, I like to think of two AI art waves. There was like the 2015 to 2022, where like the the VQ gun and like the early gun experimenters and there was a lot of artists experimenting with it. And then now the diffusion kind of like and transformers kind of like world has enabled

• 40:59Speaker 00

  a whole new wave of people to experiment with it. But at both the first wave and now this particular way, I think we're in the paint tubes moment, right? Where people are taking it and are using it to express something, right? To think of the world and then type that on the world and generate something, right? I think the artist still remains pretty much at the center, because that's what really art is about. And these are just tools, right? It's hard to understand them first because they're just new, like every new piece of art is. And I think, well, I guess to your point, like, what are we going to be asking ourselves in like 10, 20 years, 30 years? I think it's their decision that will look back and will look at this moment as in like, yeah, it many was a natural transition and we needed it. It allowed us to do so many things that we just couldn't have thought of before. Great that we had it. And I think we're still early at realizing that.

• 41:44Speaker 02

  Yeah, it seems like an extremely exciting time from a arts perspective. And I remember in 2018, the first sort of GAN-based artwork sold at auction. I don't know if it's Christie's or Sotheby's or something. And then it almost felt like everybody got really excited and then there was silence, you know, until this next wave of diffusion-based models and, you know, everything else. Is there anything that you think is needed to encourage that art scene? Or do you think it's literally just time now because we have the tools and we have really interesting things happening? Or do you need to be able to print the artist or in a way? I'm just sort of curious, like, what are the obstacles for this becoming a bona fide sort of fine arts moment or movement?

• 42:20Speaker 00

  I think it's convenience and it needs to be like accessible and usable and understandable by people. I think in the analogy of the paint tubes, it's we're not yet at the stage where you can just buy a paint tube and use it. We're still like in the stage of where transitioning from like this sophisticated pigments to like some sort of like a paint tube, right? But early like GANs and like Robbie Barat, which I was thinking, I think is the artist you mentioned that that was behind a lot of the early works on on on that auction in 20, I think it was 2017, 2018. It was very hard to just get started with a model, right? It was very sophisticated process and now you can just do it from your phone, right? You're coming closer to the like that we're putting the cap on the painting, right? Almost there. I think we're just a rate of like the rate of progress and the expectation of it will become easier and better.

• 43:04Speaker 00

  And I would say two things. These models and the systems need to become really expressible and controllable, which is somehow the way I like to think about enlightenment is like. You have an intention and you want to express that intention in a very controllable way, right? These models are yet not controllable, right? Not exactly as we would like them to be, right? And the reason why, because of that is we're too very early. Like, there's a lot that had to be invented to control them and have them be very expressive and have them work in the way that you really want them to work.

• 43:37Sarah Guo?

  So the art movements that you mentioned, they're art movements, but they're also like cultural movements fundamentally, right? And like, we talked about the tools because you're a toolmaker and it's like, we gotta have the paint too. But if you take Impressionism or Futurism or something, like it's also like, it had an aesthetic, it had tools, but it was also very Italian at a certain point in time. It was about optimism, about like urbanism and cars and everything. Are there like schools or philosophies or scenes that you think are like worth paying attention to right now?

• 44:04Speaker 00

  Yeah, I mean, I'm biased because I've been part of a particular scene in New York, the media art scene that I like, I particularly think will heavily, has heavily influenced a lot of this earnings in this stage. To your point, I think every art movement sits in a particular cultural context and historical context. And Futurism was like in particular moment in time about like technology and like also fascism was around and there's a lot of like things and you just look at the world in like this particular way and you're expressing this particular way and there's an aesthetic and a line and a system that if you look back, it's like, oh, of course, like the, and cinema was the same, like movies early on were, were a way of like perceiving the world and expressing them because it was in a very contextual, like historical moment in time.

• 44:46Speaker 00

  I think for me, if you apply that same kind of like principle now, I would tend to look a lot at the weirdos of tech, right? People who are at the fringes, people have, who've been always considered like, oh, you're just toying around, this is like an experiment, like there's a lot of creative coding communities and people who are experimenting with CoZo's art, there's like a lot of conferences and these communities of people, Baby Castles in New York or Workhack or like I O or you have all this, they're just like interesting, just art, it's just very, very, very highly creative and niche. I think those folks will define a lot of what we'll see next in tech.

• 45:22Sarah Guo?

  Yeah, well, New York or otherwise, weirdos are a pretty good bet in general for people who come from the technology world.

• 45:28Speaker 00

  Yeah.

• 45:28Sarah Guo?

  Chris, this is, this has been amazing. That's all we have time for today. We're looking forward towards unlocked creativity and the next Paint Tubes. Thank you so much for joining us on the podcast.

• 45:37Speaker 00

  Of course. Thank you for having me here. It was great.

• 45:41Sarah Guo?

  Thank you for listening to this week's episode of No Priors.

• 45:44Speaker 02

  Follow No Priors for new guests each week and let us know online what you think and who in AI you want to hear from.

• 45:49Sarah Guo?

  You can keep in touch with me and Conviction by following at Sarah Normas.

• 45:53Speaker 02

  You can follow me on Twitter at @AladGill. Thanks for listening.

• 45:58Sarah Guo?

  No Priors is produced in partnership with Podpeople. Special thanks to our team, Synthel Galdea and Pranav Ruddy, and the production team at Podpeople. Alex McManus, Matt Saav, Amy Machado, Ashton Carter, Danielle Roth, Carter Wogan, and Billy Libby. Also, our parents, our children, the Academy, GovGPT, and our future AGI overlords.