Roblox Studio Head Stef Corazza: Using AI to Empower Creators

Stef Corazza leads generative AI development at Roblox after previously building Adobe’s 3D and AR platforms. His technical expertise, combined with Roblox’s unique relationship with its users, has led to the infusion of AI into its creation tools. Roblox has assembled the world’s largest multimodal dataset. Stef previews the Roblox Assistant and the company’s new 3D foundation model, while emphasizing the importance of maintaining positive experiences and civility on the platform. Mentioned in this episode: Driving Empire: A Roblox car racing game Stef particularly enjoys RDC: Roblox Developer Conference Ego.live: Roblox app to create and share synthetic worlds populated with human-like generative agents and simulated communities| PINNs: Physics Informed Neural Networks ControlNet: A model for controlling image diffusion by conditioning on an additional input image that Stef says can be used as a 2.5D approach to 3D generation. Neural rendering: A combination of deep learning with computer graphics principles developed by Nvidia in its RTX platform Hosted by: Konstantine Buhler and Sonya Huang, Sequoia Capital

• 0:00Speaker 00

  We have this unique synergy and collaboration with the community, where basically we told the community, hey, give us access to your data to train AI. We're going to make the best AI companion, the best AI assistant that we can. And that assistant goes back into studio and is free, right? So we're not making money of your data. We're actually helping you create more. And so we found the overwhelming majority of the creators in our community gave us permission to use their data for training. And that's why I was mentioning earlier, we have not only one of the largest data set in the world, but also the most multimodal.

• 0:57Speaker 01

  Welcome to Training Data. Today, we have an amazing guest in Steph Corazza. He leads Generative AI at Roblox, one of the largest gaming platforms on the planet. Roblox has 79 million daily active users, and they have a creator economy that pays out hundreds of millions to creators. Because of this, Roblox is uniquely positioned to transform how games are made and played with AI. Steph is a founder at heart. He started Mixamo, a pioneering AI company for character animation, and was acquired by Adobe. Roblox brought him in to revolutionize how games are made with AI at their own platform. Under Steph's leadership, Roblox is pushing the boundaries of AI in gaming, from their groundbreaking AI assistant,

• 1:42Speaker 01

  which lets you generate games with simple, natural language, all the way through their 3D foundation model technologies. Welcome, Steph, to Training Data.

• 1:53Speaker 01

  Today, we get to talk about games. In particular, we're talking about AI at Roblox. Roblox is one of the largest gaming universes on the planet. And I say universe instead of platform. Platform is a pretty overused term. And really, Roblox has created much more than a platform. It's a place where you can create. It's a place where you can play, where you can meet new friends, and it's all done online, virtually. Now, we are technologists, but we're also investors. And I want to spend a moment on how remarkable of a business Roblox is. It's obviously an amazing technology, and we'll get to there. But the business is exceptional.

• 2:33Speaker 01

  You've got a $29 billion market cap company with over $3.5 billion of run rate revenue. An amazing stat here is they're really building an economy. It's not just a selfish company. It's a company that actually has produced over $800 million for their creators, for the people actually building on Roblox over the course of a year. 70 billion hours of gameplay on Roblox per year, 70 billion. And they're able to deliver cash operating profit. So $600 million operating profit. That's because they've catered to a huge audience. 79 million daily active users. Over time, they've actually shifted up in age.

• 3:17Speaker 01

  And you've got 46 million of those daily active users are actually now over the age of 13. 3.8 million daily active voice users. And the numbers just continue to grow on this amazing business. And so, Steph, we're so excited to have you here today. We could not have asked for a better person in the category of AI gaming. From machine learning to computer vision to biomedical engineering, you have a pretty impressive technical background. An amazing journey from what was initially biomedical to a generative AI efforts. And we were hoping that you could kick it off by just telling us how you got here. How did you get to becoming the head of generative AI when you started off as an engineer in a very different field many years ago?

• 4:03Speaker 00

  And thank you for the great introduction about Roblox. It's really like an amazing example of a compounding effect. And we are, every day, we are mesmerized ourselves to about the success. Well, my journey started at Stanford about 20 years ago. When I came from Italy as part of my exchange program. And I was focusing on computer vision, machine learning for the measurement of human motion. And so you have basically two markets there. One is the biomedical that you mentioned, and the other one is animation. And so we were basically at the boundary between the two. And at some point, I realized that there was a much bigger opportunity in the animation space.

• 4:42Speaker 00

  And so I basically worked on like video-based motion capture and animation creation solution. That then led to the spin-off of Miximo, the company that I started in 2008. They get later acquired by Adobe in 2015. And it's still today, one of the most used machine learning services in the industry to rig and animate characters. And so after a few great years at Adobe, actually seven of them. I helped build the 3D offering there, including products that my team built, like Adobe Stager and Adobe Arrow for AR. And then we acquired Legoric, and so we built the full 3D portfolio.

• 5:26Speaker 00

  And then after that, I was really passionate about GNI. I was working with the CTO of Adobe, trying to figure out completely new ways to generate things. And that's when basically Roblox reached out. And I had, you know, breakfast a few times with Dave, and we talked about it. And I really realized that potential of the GNI I had was really finding in the gaming space. And specifically in Roblox, being a platform where so much data exists. You know, 15 million experiences every day, every year are played. I found basically Roblox be the place where this could really blossom with a massive impact worldwide.

• 6:05Speaker 01

  So before we get into the AI components, can you tell us a little bit more about the platform? You're an engineer by background and so scale is incredibly exciting to you, I would imagine. The sheer scale at Roblox is pretty mind boggling. I think to anyone, especially users, there are obviously platforms that have more daily actives, the Facebooks of the world, there are platforms that have more monthly actives, et cetera. But it's very rare to have the amount of bandwidth and compute and graphics and everything in one place. Maybe you can share a little bit about what that means to have 15 million daily sessions and 79 million monthly actives and a little bit about the stats and what that means technically.

• 6:49Speaker 00

  Yeah, we always like to talk about the daily active, which is the 79 million that you mentioned. But the monthly active, I think we don't communicate to the outside world. It's even more staggering being in the several hundreds of millions, right? So it's a massive community that is growing very healthy and pretty fast. And sometimes people ask us, you know, where are those games coming from that people play, right? Some of those games are now like worldwide franchises with like tens of millions of concurrent players. And then sometimes I give this number, which is also like a reminder of the scale.

• 7:29Speaker 00

  So every day we have roughly 90,000 experiences and games published on Roblox.

• 7:38Speaker 01

  Wow.

• 7:39Speaker 00

  And so that gives you the scale of the human creativity, if you want. And how much really this is becoming more and more of a creation platform and game development platform that has like incredible numbers in terms of scale of creation. And also like it's an economy on its own. As you mentioned, you know, we paid out, you know, $800 million to our creators. And so people have jobs, people buying houses, people have like companies. They are now, you know, we have game studios, they have like more than 100 people. Some of them are VC funded. So it's basically creating its own like creation economy and tied to that. And so that I think is very humbling and at the same time.

• 8:21Speaker 00

  Has incredible potential because the uniqueness, I think, of Roblox is that we are one of the most vertically integrated companies on the planet. You know, we own our own data center. We have many data centers around the world where we own like the hardware. But then we also own the app that distributes all these games and the players and all the services like video chat and live chat and chat translation and so on. But then we also have the creation tool, which is a Roblox studio that I have the privilege of leading with my team. And then also, we basically have all the services for creations that go with it. So it's all the way from like the bare metal CPU GPU cluster, all the way to the creation tools.

• 9:06Speaker 00

  With the big difference that we only charge our, we only basically take some revenue ourselves when our creators make money, right? We make money when they make money. There's no upfront fee to get into the game. There's no upfront fee for the tool. The tool is free. A lot of services are free. And so there's really very little friction to start using Roblox. And if your game has one user or a million users, you don't have to worry about anything. We scale it for you. We pay for all those like CPU GPU instances, storage in the cloud and everything. And it's completely opaque to you as the creator. So I think that's the uniqueness.

• 9:48Speaker 00

  And as part of the fully vertical integration, we are also able to subsidize AI, right? So we are one of the few companies out there with a full fledged AI assistant offering for game development, for code creation, material, texture, asset, everything. And it's all free to the creators.

• 10:10Speaker 02

  I'd love to get into that. Maybe, can you just walk us through today? What is the experience of creating a game? Like, who are the typical creators on your platform? Is it a high school student? Is it a professional game developer? And what types of games are they creating?

• 10:24Speaker 00

  That's a good question. So we have several million creators on a monthly basis on the platform. So big numbers there as well. And usually, I think the average age is in like the mid twenties. It's a little bit older than our player demographics, of course. And typically, the majority of them are doing like world building. They're like building stuff. They're artists. They are making games. And then we have about thirty, forty percent that actually are coding, right? And then, of course, there's an overlap between the two audiences. But basically, roughly, this is what we're seeing. And it's used for the most different creations you can imagine.

• 11:09Speaker 00

  From like natural disaster simulations to learning, to the more classic gaming experience, events, concerts, fashion, design. We are seeing like new type of experience popping up on a daily basis, which is very fascinating.

• 11:26Speaker 02

  What's your favorite game?

• 11:29Speaker 00

  I've been playing really, really Racing Empire quite a bit. I like our games. So that's a good one that I really like.

• 11:38Speaker 01

  Steph, I was actually surprised to hear that you said thirty to forty percent of the developers are actually coding as opposed to world building. Why is that? And maybe that kind of parlay's into what you've created with Assistant.

• 11:52Speaker 00

  Yeah, so there's just, it's a skill that is harder to master and to get into, right? A lot of people just like go from players and they want to create something. And so they start like building the world and that is probably more intuitive than writing code, where you have to understand these like high level constructs and apply them to gain interactivity. And so that is one of the things that we wanted to tackle with Assistant. We wanted to basically remove that friction of having to learn to code, having to learn like programming language in order to create interactivity. And so that was like one of the initial inspirations. And that's why Code Assist was the first feature that we released.

• 12:35Speaker 00

  And then now, this was like in March twenty twenty-three, which feels like two decades ago.

• 12:43Speaker 00

  And now, you know, a year and a half later, we basically have a full a full basically assistant that has game development capabilities. They go from writing code, auto completing code, explaining code, debugging code, applying scripts to parts and objects in your scene. So that's all the coding stuff. Then we have documentation, right? People ask, how do I do X, Y, Z? And usually they have to browse through their forums and documents on the internet. Instead now, assistant can basically summarize those information for them. And then the third aspect is creation of assets. So we rolled out a material generator.

• 13:25Speaker 00

  We rolled out a texture generator, which was a lot more complex, where you can basically texture any 3D object with a quite good fidelity and resolution just from a text prompt. And so all these together is what we call an assistant with an umbrella name. And it basically is allowing now to create entire, like, simple games from scratch just by typing natural language. In the future, there will be more like multimodal input through images. But basically, right now, we have like simple games that people are making also as a test, where like they are only using assistant. And so you can imagine, you can make that game on your phone,

• 14:07Speaker 00

  you know, with a microphone. You just speak to it and then assistant will just generate the world, will create the forest, will create the enemy, the boss that you have to fight, and then also add all the game mechanics to it. Everything automated.

• 14:23Speaker 02

  Are the games created with assistants? Like, how good are they? If you had to give a score out of 10, the games created from coding versus what people are doing right now with AI versus where it's going?

• 14:34Speaker 00

  I mean, if you can code, of course, you get to another level of sophistication in terms of the complication of the gameplay and all the nuances that make like a game fun to play. So of course, we're not there. The examples that I see are mostly like we do game jams and, you know, we spend like two hours to make a game with assistant and what can come out is pretty incredible. But we haven't had, you know, in the community, I'm sure they're gonna take it to the next level. The goal is not to exclusively use assistant. The goal is to basically combine assistant and learn skills through assistant. So maybe at the beginning, the first script, assistant will write it for you

• 15:18Speaker 00

  and will attach it to a part. Then you know where the script should go. And then you know, you know, how to, I don't know, make some hobby platform move up and down, and then you learn on the way. So we see assistant as a companion that it shows you by doing how you make a game and then over time, people really develop skills. Otherwise, it's hard to acquire.

• 15:42Speaker 01

  Stefano, have you seen that the type of development has changed? As in, the assistant not only has language, like the documentation you described, it does also have code. It even has images. I saw that it's, and we'll get into the technical specifications in a little bit. But as a user, you have language, you have code, you have images. Have you seen the behavior change for how people develop in Roblox? As in tactically, have you seen the number of people actually coding going up? I would have guessed that outside looking in because the barriers to entry of coding have gone down. But does that just mean that more people are developing in general and the ratios actually stay the same?

• 16:19Speaker 00

  That's a great question. So what we have seen, we have measured the productivity of people that use assistant versus not. And so we have found that people that use assistant create 180% more code.

• 16:33Speaker 01

  Wow.

• 16:34Speaker 00

  And so the individuals are a lot more productive.

• 16:36Speaker 01

  And that's benchmarks to people who already wrote code? Or does that also say just more?

• 16:40Speaker 00

  Yeah.

• 16:41Speaker 01

  Oh, wow, okay.

• 16:42Speaker 00

  Two cohorts of coders, one using assistant and Code Assist. Code Assist is the, you know, suggests code and assistant that creates from scratch. But they're all kind of integrated into a similar user workflow. And then if you're looking the same cohort comparison for material generator, creators that use material generator create sixty percent more materials. And so also on the art front, there's more productivity. And then if you look at the final goal, which is how much they publish, right? Because publishing the game is the ultimate goal, the lift is about thirty percent. So people that use assistant publish thirty percent more than people that are not using it.

• 17:26Speaker 00

  And remember, this is now still in beta, mostly. It's gonna get out of beta soon. I can, I can give you the exact day, but you know, it's gonna be relatively soon. And so we're gonna see even, even a broader adoption, of course, there and impact.

• 17:45Speaker 01

  One more question on this, on Sonya's quality question. Like, what about usage? So they're publishing thirty percent more. Do you have any KPIs that actually give you a sense of if the assistant games, the hours spent, the Roblox spent on them, whatever the KPIs might be, if they are also that thirty percent lift?

• 18:04Speaker 00

  Yeah, that's a good question. So our number one KPI right now is around retention. And so we are seeing like a week over week retention of people using assistant. That is much higher than any other features that we roll out. And also we are seeing the over time that retention in the long run increases quite a bit. So we have people that are using assistant with studio. We are seeing significant lift in the overall retention of studio. And then, of course, we see a higher retention on system. So the number of daily users has been growing organically and very steady. We don't do any marketing, of course, on this. And, and it's free, which is, I think, the best marketing.

• 18:45Speaker 00

  You know, like, it's not, it's not cheap, let me tell you that, right? I think Roblox is very generous on that. But we have this unique synergy and collaboration with the community, where basically we, we told the community, hey, give us access to your data to train AI. We are going to make the best AI companion, the best AI assistant that we can. And that assistant goes back into studio and is free, right? So we're not making money of your data. We're actually helping you create more. And so we found the overwhelming majority of the creators in our community gave us permission to use their data for training. And that's why I was mentioning earlier.

• 19:28Speaker 00

  We have not only one of the largest data set in the world, but also the most multimodal. If you want, because we have, we have code, we have images, we have 3D assets. We have audio, video, all that is part of our gaming experience. And also with the interactivity, there is the glue for all of that. And with the analytics on the usage, right? So it's a very, very powerful data set that we are very, you know, treating with a lot of respect. And, and a lot of, you know, the best practices to, to keep that, of course, very secure. But at the same time allows us to really harvest the value. And ultimately, what we are doing is we are teaching AI game development. Basically, that's what we are doing, right?

• 20:12Speaker 00

  We're not teaching how to make an image. We're not teaching how to write code. We are teaching game development. That's the ultimate goal. And so it's gonna, all these tools, they are at the beginning will feel a little bit. Hey, this is a tool to make a material. This is to make the textures, to write code. We are already seeing that they are converging. We already started that process of converging. Some of those like lower level tools into larger ones. Where basically the AI is actually learning how to develop a game as opposed to just how to do like a small task.

• 20:46Speaker 02

  I love the vision of the AI learning how to develop a game. And the question I have is if you break up game development into its component parts. Which parts do you think are most likely to be taken over by AI? Well, in the, in the near term versus which, what do you think humans will be uniquely good at for a while?

• 21:06Speaker 00

  That's a great question. So we don't see AI as taking over, by the way.

• 21:11Speaker 02

  I know.

• 21:11Speaker 00

  I think we made with the best parlor that we made in at RDC like a couple of weeks ago was AI is your dishwasher. Right. Right. Nobody wants to wash the dishes, right? And so we are really focusing on tasks, especially with the last release of assistant actions. We are focusing on the task that you don't want to do, right? Washing the dishes, doing the laundry. And so we have introduced amongst the assistant capabilities, a new capability where assistant can basically make large scale modifications to the data model. A given example. I have made a beautiful open world game with a huge forest. This forest has a hundred thousand trees.

• 21:55Speaker 00

  Now, all of a sudden, I want these trees to actually follow the seasons and, you know, get more leaves more yellow because fall is coming. The amount of work that will take me to implement that will be huge. Assistant can do that with three lines of text. I can say, select all the trees, or I can say, select all the pine trees. Actually, those don't become yellow, so that will be the wrong one, sorry. Select all the trees other than the pine trees and make the leaves yellow, right? I can just give you these three liners. Assistant can go, can select, you know, the fifty seven thousand trees. There are no pine trees. It can change the color of the leaves for me in just a few seconds. So

• 22:36Speaker 00

  that's the kind of task that we are seeing a lot of value. And honestly, this was the feature that the community loved the most. Of all the AI features we released in studio in the last two years, this was by a landslide, the one that got the the highest score and appreciation for the community because again, it was the dishwasher. We are not replacing your talent, which we believe is irreplaceable. But we actually helping you on the task that you don't want to do.

• 23:01Speaker 01

  I am always so impressed by Roblox's emphasis, genuine emphasis on community and teaching. You've said this in the past few minutes. You've talked about community quite a bit, about teaching how to code and how to create games, and really uplifting the entire community, frankly. And I just want to say that this is very much true all the way to the core of the business. I got to follow my really good friend Craig Sherman to board meetings in 2017 and 2018 at Roblox. And even behind closed doors, you know, years ago, this was always the focus. It wasn't the banality that a lot of board meetings on monetization and financials. It was community.

• 23:45Speaker 01

  It was uplifting community, teaching them how to code, teaching everyone how to use Roblox in a way that actually benefits themselves and their own learning. And this dishwasher analogy sounds very consistent with the ethos of your business. How impressive. How did you implement it? It sounds like a segment anything type of algorithm. Was it a segmentation approach or this particular feature? How did you do it?

• 24:12Speaker 00

  That's a good question. So we basically have found a way to give a system. So Assistant is really good at generating code, right? As you know, L is based on LLMs and some of them, actually, we are supporting as open source project like StartCoder and we train with our own data. So over time, Assistant is getting better and better at generating code. Some of the code, instead of running runtime, you can actually run it at the time in Studio. Studio has a command bar where you can just execute some of that code. And so Assistant Actions basically is creating code that is executed directly in Studio at edit time.

• 24:54Speaker 00

  And because we integrate in Studio has full awareness of the data model of your scene. And so it knows, oh, this is a tree, this is a car. It knows what you did and it has full awareness of that. And so we have combined the ability to generate code with the ability to create, using the same code, Lua commands in Studio and the awareness of the data model. Those three things coming together have basically unleashed the power of the LLM onto like data model manipulation.

• 25:26Speaker 01

  Is that, do you have these things labeled? Or is it dynamically determining that this is a car?

• 25:32Speaker 00

  There's a good amount of inference that just happens.

• 25:36Speaker 01

  Cool.

• 25:37Speaker 00

  Yeah.

• 25:37Speaker 02

  How do you imagine the creation experience evolves? Let's say 10 years from now.

• 25:43Speaker 02

  How do you see assistants evolving?

• 25:45Speaker 00

  So when we, when we met about roughly two years ago and we said, okay, how, how is the AI going to impact creation and where should we start? The one paradigm shift that we envisioned that we thought was gonna happen in the industry, not just the Roblox, was a shift between control on, on the, fine control on the creation to capturing intent, right? And so I, I spent quite a bit of time in the Photoshop land where basically there you give control to the color of the individual pixel of an image, even though most people don't need it, but some might, right?

• 26:27Speaker 00

  And so there it's all about 100% control, non-destructive workflows, but 100% control on the actual artifact that you're generating. And, and then we are now moving to, towards a complete different generation of tools where the tool is successful and can produce good outcome only as long as it can capture the intent from the user. And so all the, all the digital tools that we have seen in the last 30 years, it's all about surfacing more control to a user and then the user will, will figure out how to, how to use that control, right? Instead, we are migrating from control to capturing intent. And so we're gonna see probably quite a big change and there's gonna be, you know,

• 27:11Speaker 00

  a thousand startups trying to do in different ways and there will be new UX paradigm popping up, but we can see audio as an input, we can see high level gesture, not, not just, you know, with your hands, but also with mouse and keyboards. So things that are just providing an input into what you want to do. We are seeing multi-model input. I can describe a word better if I can type. And I can provide maybe a concept art and maybe some high level sketching on top of it. Right. So this is very different from having that phenomenally granular control. But at the same time, the velocity is like two orders of magnitude faster.

• 27:53Speaker 00

  The challenge in all this is that for, you know, for casual creators or people that are, you know, have limited amount of time to spend on it, it probably what AI is providing is already good enough to share on TikTok or to make an experience on Roblox and invite your friends over. But for people that want to spend a lot more time, you still have to provide that fine control. Right. And so the challenge is how do I make, how do you make a tool that is really good at capturing all the initial intent, but at the same time allows the real pros to iterate with the same level of control of the traditional tools. So that's a little bit the, the challenge that I think a lot

• 28:36Speaker 00

  of companies like, like we are facing in, in studio, a lot of other companies are also facing.

• 28:41Speaker 01

  We have a really fun episode of training data with, with this team that created a company called Dust and one of the founders, Gabriel Hubert, talks a lot about rasterization versus vectorization. And this is definitely your language as a graphics person. They also were Stanford computer vision type folks and it seems like that transformation has happened. And the vectorization, kind of what you're describing, like you can, you can expand it to a great deal based on intent, you can shrink it to very small and almost like fractals go down into more and more detail. When do you think we will get there? What do you think will get to

• 29:22Speaker 01

  the point where you can say, hey, this is the intent and then you can go in and, and a fractal level of detail, change each pixel, also based on intent, but on a much smaller scale?

• 29:34Speaker 00

  Yeah, that's a million dollar question. So what we're trying to do is allow assistant first to be able to do, to perform those operations, right? So the same AI that gives me the like the Uh, you know, slot machine type of input, you know, I, I throw in some text, an image. I, I pull the lever, let's see what we get, right? We want basically a system to also be able to go in and do the fine grain change. So, hey, only the trees that are above, you know, uh, five feet, can you just change the color of that? Right? So, uh, and then can you take the texture of the tree and then open up? And I'm gonna like paint over it, right? And, uh, and so we, we want to allow AI to already be able to go beyond the like one shot.

• 30:19Speaker 00

  And I allow for iteration with, we believe iteration is, is a fundamental, uh, um, way people create. And so we want to make sure the AI can support that from the get go. And then we will always have a fallback, uh, with some tools. And maybe there, you know, it's gonna be more like a progressive disclosure where we don't throw it in front of every user, right? But only the users that wanna go deeper, then, you know, we can basically pull the curtain and then allow them to, to go a little bit deeper. In some cases, honestly, we would just like, uh, interrupt with our other tools. Uh, there are things that only in Blender you can do, or only Photoshop or some painter, uh, in our studio.

• 30:58Speaker 00

  Uh, won't become, you know, uh, this, uh, crazy place where you can do everything, but it's hard to do, uh, uh, anything, right? So we are very, I think, committed to, uh, um, studio being good at what it does. And then the, it's okay to have, you know, for really the, the pros that wanna go deep to have a great interoperability with external tools. We talked a lot about studio, but actually the thing that we are the most excited is taking all this AI goodness and bring it to inexperienced creation. And so we think that's gonna be the next frontier. So two aspects in the industry, we are at the very beginning and we are curious to, we, we don't know what's, how it's gonna play out and we are super curious.

• 31:39Speaker 00

  Using AI for substantially different gameplay, uh, like, uh, you know, companies like, uh, ego dot live are experimenting on, on that. And then taking all the ability to create the AI has unleashed and bring it to inexperienced creation. So I'm there, I'm playing my, my game on Roblox, now I want to modify this level. I want with my friends, create something new that we can play together. Those are like completely different type of of creation experiences that we're gonna see be unleashed by AI. So right now, we are incubating in studio. We're making it solid, robust. We're making sure the output is high quality. But we are very excited to bring, you know, those APIs to inexperience. And we think that's gonna be the the real impact.

• 32:22Speaker 02

  I'd love to dig more into that, maybe starting with just the in-game, the gameplay experience. How do you think, how do you think from the player's perspective, these games will be different?

• 32:32Speaker 00

  Uh, I think they can be, uh, with basically no effort or very little effort from the developer. Uh, these games can be more personalized and also they can be always different, right? Like, if you, if you play the same game, uh, but there's an L L M and he's aware of your past, uh, uh, you know, all the, you know, all the things that you have done in in past sessions. And then you come back, then you can make the game different, more interesting, can change the challenge, can spin a little bit the story, right? So all these things, when you have like a, um, a very smart, you know, AI backend, uh, who keeps track of what's happening in the history and who and knows also who you are.

• 33:13Speaker 00

  Uh, it can really morph the game onto something that is really more enjoyable for you specifically. So we think that will be a big opportunity. There's lots of companies that are now doing experiments. Uh, and we are very excited to basically provide AI as a platform and then let them experiment with different gameplays.

• 33:31Speaker 01

  That sounds, Stefano, technically very hard. And I remember even, even for the simplest type of of Roblox gaming, which there, there, there isn't really a simple type. You're generating digital worlds continuously. Like, it's actually a very, very heavy lift. You mentioned having your own data centers, right? Like, low latency, being able to play with people internationally, a- a- across the world. How do you think about adding this new level of complexity, this AI inference, uh, complexity for gameplay? As in generating the world, varying it on the fly? What will have to change from an infrastructure perspective?

• 34:11Speaker 00

  That's a great question. So I think the first step towards that will be NPC. So non-playable characters, the you who cap to an LLM. That doesn't need massive changes in interest. Right, that's pretty straightforward, and I think that's gonna be the first rep. And we're gonna see what kind of impact that creates. Then we have seen other experiences where they actually want to create the whole world. And of course, there are some challenges there. And let me tell you, I think what I think is the biggest challenge is actually moderation. Because Roblox Safety is our number one product. And we want to keep the platform safe and we want people to connect with civility. And when you allow people to create anything,

• 34:54Speaker 00

  of course, the bar goes a little bit higher. And so, you know, we have, we open this Pandora box, but we also have to build the guardrails so things stay, you know, stay positive. And we have, you know, everybody can have a positive experience on the platform. So I think that is more of a challenge than like latency in infrastructure. Because we can, you know, CDN, we can cache things. We can pre-generate some of the content. We can use a level of detail. So there's a lot of things that the gaming industry has developed in order to basically stream. We are using streaming as well, right, in the platform. So we can basically in real time generate and stream more assets.

• 35:37Speaker 00

  I think the challenge will be more like if you have now the control of the game that you're playing, how do you make sure that everybody else has a great experience? I think that's gonna be the challenge, both on the moderation side and also like in making it fun. We have infinite respect for game developers because they know how to make things fun. And not every player, you know, has mastered the same skills of like making games fun for the last, you know, 20 years, right? So it's like, how do you give this freedom to create while you somehow control the gameplay and the story so it stays compelling? So these are all things that we are very eager to learn from the community, honestly. And I think that's the beauty of being a platform.

• 36:22Speaker 00

  We don't need to have any opinion about it and we don't need to figure it out ourselves. We would just provide the APIs to the community and the community with the infinite creativity it has will figure it out.

• 36:36Speaker 02

  So I've heard NPCs as that first stepping stone in game AI frequently. What do you think is going to be the next stepping stone for how, you know, game mechanics or gameplay might change? Because I imagine you're not going to go from NPCs to entire worlds, right? Like, is there another kind of, what's the second stepping stone?

• 36:55Speaker 00

  I've seen like games where they limit the creation to one specific item. So for example, you, there's a very popular game on Roblox called Build a Boat. You go into this game, you're building a boat, and then you're sailing that boat, and then you're racing other players. And so you don't create the world. The world is predefined by the developer, of the experience. But you're basically allowed to constrain creativity as in, this is, these are the materials that you can use to build a boat, and this is the size that it has to be. And then you can go crazy and build whatever thing you want. And because Roblox is such a physics sandbox, right?

• 37:36Speaker 00

  We have our dynamics, fluid dynamics, and so we can actually see what you create and like simulate the wind, and you're going to see the outcome of what you created. And so I think experiences like that right now are a little bit more difficult without AI. And AI, I think, can really power those up quite a bit, where you can create your race car, you can create your boat, your airplanes, like some elements of the gameplay. The, based on what you create, you can have, you know, an advantage in the game, but you're not completely changing the game itself.

• 38:10Speaker 02

  Constrain creativity. That's a great answer. Thank you.

• 38:12Speaker 00

  Exactly.

• 38:13Speaker 01

  I have two follow-ups from that. The first is you just mentioned physics engines. And there's been a lot of enthusiasm for years now around PINS and these neural physics engines, et cetera. Is that now in the physics engine at Roblox, if you can disclose? Are you using neural networks as part of the physics engines, as part of the, you mentioned aerodynamics, right? Like, are you guys at the point where you're estimating Navier-Stokes because it's cheap enough, because you can use neural networks? Or is that maybe in the future? And then my second question's about NPCs. Would you guys ever allow a world or a game that is purely NPCs? As in, something you watch as opposed to something that you participate in?

• 38:52Speaker 00

  That's a great question. So on the physics side.

• 38:57Speaker 00

  I think the use of neural networks for physics is not being proven as effective. I think there is more like this unlimited amount of like real world approximations that you can use. They are like very computationally efficient. And I think that's more important than than how good you're approximating the world. And so on that on that front, I'm not sure. Neural networks are gonna provide that much value, honestly.

• 39:27Speaker 00

  Especially when you have like already implemented all the basic functionalities and people are already using them successfully.

• 39:32Speaker 01

  Especially for a purely digital world. I mean, if you're trying to, if there's a noisy real world, I imagine it's different from like Roblox world, where everything can work in the physics estimations. Is that fair?

• 39:44Speaker 00

  Yeah, that's that's fair. So maybe, I would say, that's all I can say is basically maybe. I think at the moment is still not fully proven as a path. And then on the on the NPC front, you know, if you think about a game where all the players are NPCs and you're just watching it, to me, it sounds like TV, right?

• 40:07Speaker 01

  Yeah.

• 40:07Speaker 00

  So people watch this all the time and, you know, these NPCs can be super smart and do super fun things. Can race, can do whatever they like. And maybe it's just fun to watch. So definitely, we are seeing such a huge community of people that just watch other people play games. And so those players could be LLMs in the future and still generate great entertainment and fun. So I will not exclude that. I think at the beginning, we're gonna find a hybrid model, like which is like, you know, NPCs existed, where people will inject NPCs in the game to just make it more interesting. And have people that, you know, you also can populate games that just launched.

• 40:49Speaker 00

  They don't have, you know, a thousand concurrent players yet. And so you can basically populate. Huge world with with characters that are interesting to talk to. Without having to have, you know, the the real people there right at the beginning. So I think there's going to be a lot of potential on that front, for sure.

• 41:08Speaker 02

  What about for user generated content? Like, how do you imagine UGC changes with now that generative AI is really coming into the experience?

• 41:17Speaker 00

  That's a good question. We have seen some of this with the Gen AI features that we released for avatar creation. So avatar creation is another huge area of creativity. There's a massive community in Roblox. They work on like making assets and clothing and accessories for avatars. And a lot of people make make a living doing that. And so we have found that there were a lot more people, like always, that had amazing creative ideas, but may not have the tools to like do 3D modeling, for example. 3D modeling is a very narrow, specific skill. They would have to learn, you know, Blender or Maya. And and so not all of them were up for for the challenge, but they had a great idea about what outfit to create.

• 41:59Speaker 00

  And so there we experimented with using images as an input or text as an input and generate avatars. And we found that it substantially democratized how many avatars could be created. And this is an area. We launched like an early beta of our avatar auto setup. And we are doubling down that we're going to do a lot more in the coming months that you're going to see. But basically, we want to allow people to create avatars, clothing, accessories, with just multimodal, easy input. Again, there's more people that have that have great ideas than the people that can actually execute on those. And so we want to basically remove those barriers and really leverage AI for for that type of creative expression as well.

• 42:43Speaker 02

  Do you think that generative AI blurs the line between what it means to be a user versus a creator? Like, do you imagine those two become kind of the same thing? And I imagine you think of them distinctly today.

• 42:53Speaker 00

  Oh, for sure. Yeah. And I mean, think about, you know, what happened when the music scene, right? The the boundary between the composer and and and the people that could play an instrument and people in the audience were just so set at the beginning. And then, you know, I think karaoke like completely blurred out, right? And then now everybody can create things on GarageBand. And now with AI, people can just type lyrics and the genre, and then a full song is created. So they are like, the boundary got completely blurred, I think, in the audio and music space. And, you know, for game development is going to take a little bit longer because it's just

• 43:31Speaker 00

  more complex as a type of content. But I think it's, you know, just entropy only only increases. And so we're going to see that for sure.

• 43:41Speaker 01

  Stefano, you mentioned that you guys have an absurd amount of data. You guys have an insane amount of video data. I think Crowbox is still the number one VR application in the world by a lot, including on VR headsets and then also, obviously, personal computers, all sorts of devices, etc. Ton of audio data, ton of textual data, kind of everything data. And so you kind of you mentioned this, I've been kind of walked by it, but obviously Sony and I are very curious. With all that training data, yes, the name of the show, are you guys going to have a world model? I think I know the answer to this. I think it might even be announced already, depending on when this airs. Are you going to have a world model? What's it going to look like?

• 44:26Speaker 01

  What will be its boundaries? And where do you go from there?

• 44:30Speaker 00

  Yeah, that's a great question. So yeah, we do have an enormous amount of data. And I think the challenge for us is less about gathering data and more like being able to use it. Because there's there's a lot that goes into from, you know, the raw data to be able to to train. An L L M or AI in general. So that is where like the work is focusing. We have announced just a couple of weeks ago that Roblox is working on a 3D foundational model. Our intent is to open source that. And basically, what it does is it would allow the digital synthesis of scenes and the world from multi-model input. So that's the goal.

• 45:13Speaker 00

  And I think also we can go a little bit beyond that. As I said before, our goal is basically to teach game development to AI. And so it's not just about world creation. I think that will be the first maybe area that we can really attack in a meaningful way. But then there's gonna be all the interactivity of that world. The ability for things to move, bend, doors to open, characters to run around, right? And then the next level will be all the interactive gameplay. And so we are seeing the data and we also can see from the data what works, what is fun for the for the user.

• 45:53Speaker 00

  And so we will be able to guide AI to not just make games, but actually make games that are fun. Because you can see which games get more traction or like which specific levels get played more. And so they could be in the inference aspects of quality of the experience, not just like, hey, I'm just gonna generate whatever you ask me, but I can also like, if I can pick, I can pick things that are like more fun to engage with. And so again, that one is gonna be, you know, it's gonna take a while to figure out. But, you know, we have the data that they can support it. I think it's more about on us to be able to pick up, you know, there's gonna be a lot

• 46:37Speaker 00

  of like unsupervised learning that we have to do. And so we have to pick up, we have to find ways to pick up signals of intent from our creators that then we can use in an unsupervised way to classify things and then figure out what AI should be learning from.

• 46:54Speaker 01

  I'd imagine because of the physics engine, you guys have such a competitive advantage with object placement, object interaction. The struggle with a lot of training data for these video and role models is they're two dimensional. And there's no concept of, I mean, maybe you can infer from it, the concept of what is three dimensional and how objects interact. But is that something that you guys are leaning on particularly heavily? I'd imagine it's a big advantage to the 3D model you create.

• 47:23Speaker 00

  Yeah, it's a huge advantage for gaining also consistency over time and spatial consistency. If you actually have the full 3D model and if you have a full 3D scene, then you can anchor the spatial and the temporal coherence in a much stronger way than if you're estimating those things from a 2D. I mean, the classic example is if you're trying to stylize a video and you use just video to video, you're gonna see drifts and artifacts and everything. Then with ControlNet and the depth map, which is like a 2.5D approach, this video to video type of stylization became a little bit more reliable

• 48:08Speaker 00

  and more consistent temporally. But you still don't have, I mean, you have basically the situation where if a character is looking at you and then you stylize that face and then they look away and then they look back, they're like a different person, right? We have all seen those examples. And so the only way, I think, to overcome that is to actually train with 3D data. And the industry went so fast, so far with just 2D, right? And is now making magic with 2D. And I think there's still like the work to incorporate 3D information into those algorithms still has to come. And I think it's gonna bring to fruition this temporal and spatial coherence that right now we don't have

• 48:51Speaker 00

  or that is so hard to generate out of 2D data, right? Because it's not rich enough.

• 48:57Speaker 02

  Just to make sure I understand what you just said. Do you think that 2D is gonna lead to a dead end? And you have to start over from 3D? Or do you think that 2D can kind of get there with scale as well?

• 49:07Speaker 00

  I think if the goal is to operate on a single image, 2D has done fantastic. And so if you need to texture a character or stylize an image, all these things is fantastic. If you want to actually have a video, so if you add the basic time on the axis and now you are also moving the camera, so the camera is moving and then there's like 2000 frames ahead of you, they have to be consistent. At that point, if you're not using 3D data, you are at a disadvantage and you're gonna be dealing with drift. And yes, you can enforce it in many possible way, but the problem of 2D data is that it has occlusions. Right? So if you're looking at me and my arm is behind my back

• 49:51Speaker 00

  and then at some point my arm pops out, you have nothing to enforce consistency of the look of my hand because you just have not seen it before, right? So all these problems don't exist if you're operating on a 3D representation of your scene because even the data they are not seen from the camera is there and it's available. And so I think the industry has gone like, you know, in a amazing, made amazing progress in trying to cope with that, which is a fundamental lack of data. And then if that data is actually used and can be incorporated in, let's say, a stable diffusion, then we're gonna see a much better outcome.

• 50:31Speaker 02

  Steph, in our homework for this episode, we heard that you're passionate about neural rendering. Can you tell us what that is?

• 50:37Speaker 00

  Yeah, I don't know where you read that, but yes, it's true. I'm very passionate because I have a strong belief that neural networks will change the way we're gonna stylize games and make games visually a lot more compelling than they are today. And so if you look at the history of game development, the creation of assets and the style of the game were always tied together. You know, if you're making like a Super Mario and that's the style of the game, you can't just like on the flip of a coin say, okay, make it look like Call of Duty. It would just not work. You will have to redo all the assets from scratch. But if you're using neural rendering or generative rendering techniques,

• 51:19Speaker 00

  you're able with some text description and maybe some reference images to restyle your game in real time. And then you leave the geometry where it is, so it's physically consistent, right? Your physics capsule is in the same place, but the visual look can be very different. And this can be used also to make games completely photorealistic, even though you're using, you know, some low resolution meshes and textures. But you can use this as a final pass. Like, you know, in games, you add, you know, typically bloom at the end to make things look really cool. And so it could be like a final, you know, a generative rendering pass, make it photorealistic or stylize it in this specific way that I like.

• 52:01Speaker 00

  And you can restyle your game in a way that actually doesn't add any extra assets that you have to download, doesn't change the assets. It's really like a compute layer that happens at the end. Right now it's very compute intensive. But, you know, looking at the speed at which things have moved, you can imagine that in the future, people, developers can just add this filter to their game, describe the style, give a reference image, and then have the game look beautiful without changing, you know, a single geometry in the assets.

• 52:35Speaker 01

  Wow.

• 52:35Speaker 00

  So I'm a believer that in five years, this will be the way games will be built. And it was gonna run also at least on the high-end phones. And yeah, we're very excited about it.

• 52:47Speaker 01

  Steph, does it ever become the end gamers' choice? The way it's rendered as in I put my skin or style into whatever game I'm playing?

• 52:58Speaker 00

  That's a good question. I think if the developer of the game allows that freedom to the player, why not? Right? I think it's gonna be an artistic choice. Hey, you can play my game and you can make in any style that you want. And again, you can still play with people that will visually see a different game, but the game is still consistent. All the physics, all the gameplay still hold.

• 53:20Speaker 01

  Stefano, thank you so much for joining us today. We learned an incredible amount, certainly about the scale and complexity of Roblox, not just from technology, but also the scale of impact. The 79 million daily actives, the hundreds of millions of monthly actives, the amount of technical depth that's necessary to get there. We learned about how you're using AI as dishwashers to empower your customers and developers and gamers, not just in the assistant, but also in the code assistant and the material generator. And we learned a little bit about what you think the future is going to look like. We might be using infrastructure for guardrails to make sure that the civility of Roblox is preserved. We might be watching some NPC television.

• 54:04Speaker 01

  And we certainly will have a 3D world created by Roblox. Can't wait to live in that oasis.

• 54:11Speaker 00

  Thank you so much for having me. It was a pleasure.

• 54:14Speaker 02

  Thank you.