Muse Wonder: Zuckerberg and Wang's $100B AI

As Meta scales its AI infrastructure past five gigawatts, Mark Zuckerberg (founder and CEO, Meta) and Alexandr Wang (Chief AI Officer, Meta Superintelligence Labs) join John for a pint to discuss the launch of Muse Wonder and the future of personal AI agents. In this episode, Mark and Alex open up about the creation of Meta Superintelligence Labs (MSL), the decision to execute a complete talent reboot, and what it really feels like to make a $100B+ CapEx commitment before knowing if the model will work. They dive deep into the architecture of Muse Wonder—including active memory, multimodal perception, and frontier coding—while laying out their vision for agentic commerce, hardware form factors like smart glasses, and why superintelligence belongs in the hands of billions. Timestamps (00:00:24) Muse’s breakout use cases (00:07:28) Why the model is the product (00:14:35) Meet Muse Wonder (00:24:56) How frontier models are actually built (00:48:26) Betting $100B (01:07:21) Agentic commerce (01:19:34) OpenRouter (01:20:06) Beyond the smartphone (01:27:00) Funding (01:34:06) Why AI won’t be winner-take-all (01:40:46) Meta Superintelligence Labs (MSL) (02:08:11) Working for Mark Zuckerberg

• 0:00Speaker 00

  At the end of last year, we're like, all right, we are quite excited about the progress.

• 0:04Speaker 01

  Yeah.

• 0:05Speaker 00

  We haven't seen the models yet. You know, we're going to write this check for like, I don't know, like $100 billion or $200 billion. I'm like, Alex, are you sure this is gonna work?

• 0:16Speaker 01

  Cheers.

• 0:17Speaker 02

  Cheers.

• 0:17Speaker 00

  Yeah. Cheers.

• 0:19Speaker 02

  Cheers.

• 0:20Speaker 00

  Cheers.

• 0:20Speaker 01

  Let's launch it.

• 0:24Speaker 01

  So, give me the updates on Muse.

• 0:27Speaker 00

  If you were looking at the third-party reports, you would see that this might be the fastest-growing agent or AI app that I think has existed yet.

• 0:41Speaker 01

  What are your guys' personal favorite Muse use cases? Either yours or other people's?

• 0:47Speaker 02

  I was actually talking about this at dinner yesterday. My favorite thing is I have my Muse, whose name is Euler, constantly source increasingly obscure teas from increasingly sort of like obscure vendors of tea. So at this point, he's on a weekly basis, my Muse is buying some random tea from some, you know, farm in Taiwan or some farm in Japan and sourcing it over and bringing it back.

• 1:16Speaker 01

  What's your favorite Muse use case?

• 1:18Speaker 00

  I don't know. I mean, I use it for a ton of stuff. I still like, my favorite both use case and probably eval for the product is having it coach me in MMA. So I put cameras in my gym and I basically, it watches the cameras when I'm in there and it gives me feedback in between the rounds, which is a very hard problem, it turns out, because not only does it have to, it basically has to process all the video quickly enough because you only get one minute in between the rounds.

• 1:52Speaker 01

  Oh, it coaches you between the rounds?

• 1:53Speaker 00

  Yeah, I want feedback. I want feedback on how to win, you know? And it's like, come on.

• 1:57Speaker 01

  Did you have to instruct us? I've wondered about this too, because looking at videos is a really good AI use case. It feels like you can't just give it a multi-gigabyte video file. So do you tell it how to slice stills out of it or something?

• 2:10Speaker 00

  No, basically, yeah, it pulls stills. I think it kind of figured that out by itself. It turns out that Muse has pretty good native understanding of MMA. So that's good. So like the multimodal-

• 2:23Speaker 02

  Pre-training AI.

• 2:24Speaker 00

  Yeah, that's, yeah, someone on the team I knew. Now, or just kind of out there in the corpus.

• 2:34Speaker 00

  I think it's a good test of the kind of coding ability that, because really, basically, what it's doing, it's like, you know, within the agent, I mean, it's like, I'm not seeing any code, but it basically is writing this whole video processing, image processing pipeline.

• 2:47Speaker 01

  Yes.

• 2:48Speaker 00

  And I'm constantly telling it like, all right, like, you look like you took 30 seconds to give me that feedback. I'm gonna have a 60-second break. Like, you gotta go faster. Just optimize yourself, optimize yourself.

• 2:57Speaker 01

  Yeah, yeah, yeah.

• 2:57Speaker 00

  And it's, um, so it's, I think, a good example of how stuff is gonna go in the future, where basically like everyone is actually gonna be writing tons of code. They're just not gonna know that they're writing code.

• 3:08Speaker 01

  What kind of MMA steers does a model give you?

• 3:13Speaker 00

  You wanna see?

• 3:14Speaker 01

  Yeah.

• 3:14Speaker 00

  Basically, let me see if I can pull some stuff up. But it's basically, it gives me a lot of stuff about balance and like hip position or it's like, oh, you're like, when you're kind of in side control here, it's just like, you're like, you didn't get this underhook in this place, so you're gonna, you're vulnerable to this getting rolled over here. Let me, I'll pull up some stuff. You guys should go to-

• 3:35Speaker 02

  It was definitely one of his first use cases, actually, even the initial prototypes.

• 3:39Speaker 01

  Yeah, yeah, yeah.

• 3:40Speaker 00

  I mean, I did this with OpenClaw before we even had Muse working. It was just like, because, and then that's why I was saying, it's kind of like become an eval for this too.

• 3:49Speaker 02

  It's, I actually didn't know that it was an eval for the product, so noted.

• 3:57Speaker 01

  How is all this in the model's training data? Like the fine-grained, this is what an MMA fighter looks like when they're appropriately balanced and this is when they're like too much on the back foot. Or just, I'm wondering, how is it consumed so many images of appropriate versus inappropriate MMA posture?

• 4:17Speaker 00

  Here you can see, it's just like, it just pulls out frames from it and it's like, okay, I'm like too much on my back leg there.

• 4:21Speaker 01

  Huh.

• 4:22Speaker 00

  Yeah.

• 4:23Speaker 01

  But is this the emergent intelligence? Like, it does actually, again, does it know from seeing lots of images of MMA? Or does it know because it understands the concept of where you're meant to be balancing? It's read books about MMA and then it sees that you're too much on the back foot or whatever?

• 4:37Speaker 00

  Well, we did train the models with a lot of visual perception data. So, I mean, I think that that's like kind of making good multimodal models. I mean, this, this gets into like, I think, a kind of core approach thing that we have, which is we're training the models end to end to be good personal agents. Right? So like the question is like, why would Meta like want to build these models? I mean, first of all, I think, you know, just superintelligence overall is going to allow us to run the company better and build all kinds of different things. But for the products that we're building, there are certain capabilities and certain mixes of skills that basically are just much more applicable to these cases. So I mean, this is one of the things that kind of Alex focused on from the beginning

• 5:20Speaker 00

  was like, all right, we really wanna make sure that our multimodal program is good because that's disproportionately important for the types of things that people use all these services for.

• 5:29Speaker 01

  How do you train a good multimodal model? Is this where Meta just has an advantage because you guys have so much image and video data?

• 5:35Speaker 00

  It's very helpful.

• 5:37Speaker 02

  Yeah, and I think ultimately, like, a lot of this, it comes down to how are you focused on it, right? Like, I think, I think because of the fact that we wanted to build personal superintelligence and fundamentally, I think most people in the world really experience the world in a highly multimodal way. They like talking to other people and listening to things and seeing videos and like, you know, filming things of their own. Like, I think most consumers have this like highly multimodal experience. And especially earlier this year, I think there was, you know, there was actually like a point of tension almost in the industry because I think so much of the consensus view within the AI industry was that it was just all about coding and coding was all that mattered.

• 6:19Speaker 02

  And I think that, I think that for the success of a product like Muse and the success of a lot of these consumer use cases, it really depends on the ability to be strong in multimodal.

• 6:30Speaker 00

  Yeah. I think we're just kind of scratching the surface of that. I mean, a lot of the initial theory that we had was that, I mean, most people don't actually like reading that much, right? So even the kind of text-based interaction model, I think it's something that people do. I mean, everyone texts, but, but, and when you think about stuff like the glasses that we're working on or even our other products that are very visual, and the glasses are basically, will eventually just be a pure multimodal product, right? It's like, you know, you can let it hear what you are listening to and it can talk to you and it can see what you see if you let it. And then eventually, I don't even know if we're gonna have an app store in the glasses. It'll just be like a multimodal generation that kind of creates whatever you need right in your vision.

• 7:15Speaker 00

  And I imagine something like that will probably end up powering a lot of the kind of apps that we build too. But yeah, I mean, we're still in the early days of that, but, but for now, I mean, the perception work has been really impressive.

• 7:27Speaker 02

  Yeah.

• 7:28Speaker 00

  Yeah.

• 7:28Speaker 01

  I wanna ask you guys about the product space here in AI and try something out on you. If I was to describe to you, say with Stripe, what I think, you know, winning product looks like, it would take me pages and pages to talk about because we have the developer experience and we have to talk about all the international heterogeneity of payment methods, then we have to talk about stablecoin. It would take me a long time. Whereas I noticed a phenomenon in AI where it feels like the winning products collapse very quickly onto an agreed-upon product space that everyone is pursuing. And so Transformers worked and then like the chat agent UI really worked. And, you know, now we are seeing, you know, you mentioned OpenClaw earlier, the kind of OpenClaw persistent agent has a computer able to do things, you know, computer use

• 8:12Speaker 01

  out there in the world for you, like we're collapsing on that product space. But there's something about AI where the winning products tend to be very simple, pursued by all the companies that are kind of taking it seriously. You could describe on the back of a postcard, like you could describe very concisely. And obviously, implementing them is massively complicated. Like, we'll get into the new model. Like, there's, you know, huge amounts of frontier work that goes into making a good version, but the winning, and you know, now you're talking about, you know, the glasses or, you know, the future versions. But somehow, there's a very small number of like clearly correct products in AI as opposed to it being this like very broad and complicated product space.

• 8:54Speaker 00

  I kind of disagree.

• 8:55Speaker 01

  Sure.

• 8:57Speaker 00

  I mean, well, first of all, I mean, I think part of the theory from the beginning was that this kind of agent or whatever form it took was going to be quite a different thing from this like text-based assistant, right? Like, every company has one of those. Ours is Meta AI, it reaches a ton of people, but I mean, it's like the same thing that all these companies have. And we, I think specifically, think that that is used for very informational purposes, but people don't think about it as like a thing that gets stuff done for them.

• 9:26Speaker 01

  It's retrieval, not delegation or something?

• 9:28Speaker 00

  Yeah, and even like when you say it has a computer, yeah, but there's lots of different ways to do that, right? You know, I mean, I think that it's only kind of this most recent wave that gives the agent a dedicated VM, which I think is actually incredibly important for making you feel confident that the security and privacy of your information is, it's not just like, okay, it's not connecting to some elastic compute that like that just kind of everyone uses, so.

• 9:56Speaker 01

  But my claim isn't that it's easy to implement, nor that it's obvious.

• 10:00Speaker 00

  I think there are going to be a lot of things.

• 10:01Speaker 01

  Yeah, okay, you think there's going to be a lot of different things? Because again, my claim is that now that people see Muse, everyone looks at that and says, oh yeah, that's clearly how you want the like AI agent product to work, and that the future AI agent products that we'll see will be fairly Muse-shaped.

• 10:17Speaker 02

  So I think, my take on this is like, I think we're just so early in the realm of what AI products will look like, that I think a lot of the interesting differentiation about the products kind of gets collapsed in how we view the products today. Like, I think the traditional way in which we wrote down and think about product specs, like, that is not, those aren't the relevant dimensions and levers and axes to consider the important design decisions on AI-native products. And so like Muse, for example, like, I think so much attention was put in the design of the model for the product, and there were a lot of very intentional decisions around, in all these situations, how exactly should the model behave? And what is the personality of the model in these situations?

• 11:02Speaker 02

  And how should it handle these sorts of tool use scenarios? And these are like very intentional decisions, and probably in the same way that like, you know, the full spec of how we wanted the model to behave in all these scenarios would be an extremely long document, but like, because of just how we think about products today, it's like, you know, it's just, it's just AI. It's kind of like how, when mobile first started happening, it was like, oh, it's just an app, and everything is just an app. Well, turns out, like, there's a lot of nuance in like what exactly an app is, and we're kind of in this phase where, oh, it's just AI, or it's just an agent, or, you know, it's just an agent with a VM or something like that. But, you know, the interesting differentiations, we have to like tease out.

• 11:44Speaker 01

  Okay, so you guys think I'm making the classic mistake, which is like, my domain is complex and this domain is, again, from a product point of view.

• 11:49Speaker 00

  It's like all payment processors seem the same. Yeah, no, it's, no. I really agree with what you're saying. One thing that we talked about from early on is that basically, there is this pattern where I think the products are getting thinner, and the model is getting to be more of the product, right? So one of the things that we say internally is like, the model is the product.

• 12:05Speaker 01

  Yes.

• 12:06Speaker 00

  Right, so, and when you're talking about an agent, a lot of the complex product work that went into Muse is basically around the privacy and security model for handling your data and the things that get connected to it, and that, I think, is something that we're just very proud of, that we really focused on, but that's like a lot of the product surface that we need to innovate on, but the capabilities, the personality, a lot of this stuff is like, you're training into the model. And whether there are the people in the lab who are working on that, or the people on Nat's team who are working on the product, I think when Nat is thinking about like, what is the key stuff that we need to make the product feel good, he's talking about post-training the model, right? Not like, I mean, of course, the UI of the product matters too, but the most important part of the UI is the product.

• 12:47Speaker 02

  Yeah, exactly. Like, I think there's like, there's still, very viscerally, I think from a user, there's like a feeling of using Muse over time. And like, maybe some meaningful portion of that is just like the reliability of Muse to handle certain tasks, but then another meaningful portion is like, how is it proactive? And in what context is it proactive? And, you know, when it talks to you, how do you feel about how it talks to you? And so there's like, clearly a lot of, and I'm starting to see more and more of this actually show up on the X feed, which is, which are, people are starting to tease out these like, you know, they're all proactive agents, but what are the actual like meaningful differences between them? And these, I think, are the interesting product decisions, which, you know, the funny thing is like, how do you specify these product decisions?

• 13:32Speaker 02

  Well, it's ultimately in model design decisions, which ultimately boil down to like, training methods, or data, or vowels, or, you know, all these things.

• 13:39Speaker 01

  Yes. As we talk about product space, one notable Muse detail is that there is no model selector and no effort selector. Do you guys think that's just obviously correct in a consumer AI product, or?

• 13:52Speaker 00

  I just want to make it simple.

• 13:53Speaker 02

  Yeah, I think, I think that the, we've been, I think somewhat infamously, a lot of the AI products have been designed by a highly technical group that like, do not have these sort of like, you know, randomly sampled American in their like, you know, in their mental model. And so the products like, I mean, you know, the naming of some of the models is like so egregious.

• 14:18Speaker 01

  The numbers go backwards.

• 14:19Speaker 02

  Some of the times the numbers go backwards, and sometimes it's just like a totally different word, and you have another set of words, and like, I just like, how is somebody who's not like carefully reading, you know, API design specs, like supposed to know what they're supposed to pick? Like, it's just far too confusing.

• 14:35Speaker 01

  Okay, we're talking about models a lot. Tell me about the new model, it sounds awesome.

• 14:38Speaker 00

  Yeah, I mean, I think we're quite proud of this. Yeah, the new model, Muse Wonder, it's a new class of model that is at the frontier. So we've basically been working on this since the beginning. I'm basically like, when you're doing these pre-trains, you, you don't just go to whatever the biggest cluster is that you have, you want to, you know, work your way up the ladder, training models that are, you know, maybe four or five times bigger, and then you just keep on repeating that, that process. So the first set of models that we've had Muse Spark was all based on this initial pre-train that we did, and what was it, December?

• 15:13Speaker 02

  Yeah, at the end of last year.

• 15:15Speaker 00

  Yeah, and, and it's actually, I mean, I think we've all just been quite surprised and impressed with how much intelligence and capability the team has been able to kind of build into a model which is like, frankly, not that big, right? I mean, it's like, and then after that, we were working on this next model, and the pre-train has been done for a bit, and we've been working on the post-training, and now it is here. So it's the next kind of class of model that is significantly bigger, so I think you will kind of feel that in terms of the nuance in which it can think. And it really, a lot of the areas that we focused on, we're making it the best personal and professional agentic model.

• 15:57Speaker 00

  Some of the different things that you can optimize for, that's it for us. And then, yeah, I mean, a lot of the basic capabilities that we focused on building in, I mean, very good memory, kind of very good ability to persist through complicated tasks, but, I mean, a ton of work went into this across the lab and the product teams and all of this, and, and I think it's also just sort of a testament to the, the team being able to methodically figure out the process to scale these models. So I think this is also just, you know, a milestone on the way towards the time between each subsequently bigger class of models, I think will now start to kind of compress, and that's very exciting for the type of stuff that we can build.

• 16:40Speaker 01

  At Meta or in the industry generally?

• 16:41Speaker 00

  At Meta. I mean, I think you're seeing something like this everywhere else, but.

• 16:46Speaker 01

  Is this an RS-I effect or something else?

• 16:48Speaker 00

  Well, I think at Meta, part of this is just, we needed to, well, I think quite famously, we were not happy with the trajectory that we were on with Alma4. We kind of like rebuilt the whole team. I mean, that's been this kind of crazy thing over the last year and a half that we should probably spend a bunch of time on.

• 17:04Speaker 01

  Oh, that's about it, yeah.

• 17:05Speaker 00

  Some war stories.

• 17:06Speaker 01

  Yeah.

• 17:07Speaker 00

  But the team basically had to go build up like different tiers of models to kind of get to this one, and, and now that we are kind of at the frontier, now everything that we do going forward will just kind of be pushing the frontier, as opposed to, there's a lot of kind of background work that we needed to do before that. I don't know, what would you say?

• 17:26Speaker 02

  Yeah, well, I think we're really excited about it. We will have two, sort of, two models in this series that are going to be available. So there is Muse Wonder and Muse Wonder Full. Laugh? Yeah, alright, there you go.

• 17:40Speaker 01

  Yeah, very good.

• 17:41Speaker 02

  Which is basically a much higher end.

• 17:43Speaker 01

  We have no studio band for it.

• 17:44Speaker 02

  Yeah, no. A much higher end model.

• 17:47Speaker 00

  Is the full version of Muse Wonder.

• 17:49Speaker 01

  I know, I got it.

• 17:50Speaker 00

  Yeah, no, it's two words, "Wonder". Full version.

• 17:56Speaker 01

  You're clearly a father.

• 17:57Speaker 02

  What's up?

• 17:58Speaker 01

  Excellent dad joke.

• 17:59Speaker 02

  There you go. And the performance, I mean, we've seen incredible performance across a slate of evaluations on personal and professional agent tasks. It's state of the art, based on our measurement. And then there's a number of, I think, specific highlights which we're really excited about, which we've designed into the model over time. So, active memory, like Mark mentioned, is one area where, you know, we think that to build personal agents, personal and professional agents, like, memory is just one of the clear requirements to build reliable agents, and is one of the areas where we've invested, I think disproportionately, actually, compared to many of the other efforts. And we have our own internal evals of memory bench, which demonstrate a lot of the capabilities there.

• 18:42Speaker 01

  What does memory mean in a model to a layperson? Like, is it just how much stuff it can, like, the working memory of if you're working on a task, how much stuff the model can kind of keep in its...

• 18:52Speaker 02

  Yeah.

• 18:53Speaker 01

  Like, how's it different from context?

• 18:54Speaker 02

  I think it's in the experience of using the model, how well does it remember things that you mentioned to it over days, weeks, or months, and then leverage those in the process of doing future tasks.

• 19:04Speaker 01

  I see. This is the stuff that's buggy AF in OpenClaw. Like, I think we're all extremely fond of OpenClaw, very happy OpenClaw user, but it ultimately does this compaction where it has like the long-term memory and the working short-term memory or whatever, and I think once people use OpenClaw for a while, they find the memory pretty buggy.

• 19:22Speaker 02

  Yeah, so we've designed basically both the model and a system for the model to use to develop a much stronger form of memory. We call it active memory.

• 19:31Speaker 01

  Yeah.

• 19:31Speaker 02

  In addition, as, you know, as Mark mentioned on the persistence.

• 19:34Speaker 01

  But so this is outside the model, like, the model exists.

• 19:37Speaker 00

  No, no, but some of the capabilities that you're talking about, like the compaction, are things that the model does. Right, so basically, being able to do this well, both with the agents.

• 19:45Speaker 01

  It's not a harness, it's in the model itself.

• 19:47Speaker 02

  Yes, it's in the model itself. Exactly, yeah, yeah, yeah. So it's aware of the, like, its memory as a core system.

• 19:55Speaker 01

  Yep.

• 19:55Speaker 02

  And it's about how it's able to leverage various sources to be able to, you know, conduct tasks in the future, as well as like, understand its ability to do compaction, you know, how it should do that compaction most effectively, et cetera.

• 20:06Speaker 01

  Okay, but this is a big deal, because I think a lot of people's early experience of AI agents has been buggy memory, and so, yeah, improvement is...

• 20:14Speaker 02

  And to be clear, there's still plenty of room to improve, like, our evaluations show this, but this is something that we're quite proud of, and we think this is a meaningful contribution.

• 20:22Speaker 01

  Okay, so active memory is one thing.

• 20:23Speaker 02

  Yeah, active memory, active focus is another, which is effectively just the ability for the model to, you know, when conducting tasks, be able to sort of carry them through, especially as we've seen Muse users, as well as Muse code users, give sort of like, higher and higher complexity, and more and more ambiguous, and more and more complex tasks. We want the model to be able to conduct those and do those very effectively, and this is one of the things, actually, in early dogfooding and testing, that a lot of people have noticed about the model that is sort of special and stands out with respect to other models that they've tried or used. The third one is domain expertise.

• 20:59Speaker 01

  What is focus in AI terms? Is it like, putting a lot of compute on the task? Is this NAS getting distracted with other, like?

• 21:07Speaker 00

  Kind of persistence.

• 21:08Speaker 02

  Yeah, like persistence, it's basically like, the ability to conduct complete tasks over very long horizons.

• 21:13Speaker 01

  Okay.

• 21:14Speaker 02

  And then, when faced with various setbacks along the way.

• 21:19Speaker 01

  Yes.

• 21:20Speaker 02

  The ability to sort of identify other solutions to those problems in an aligned way.

• 21:24Speaker 01

  Okay.

• 21:26Speaker 02

  Then there's, it's actually a big step up in coding capabilities, so frontier coding capabilities, so it is actually state of the art on DeepSui and Sui Marathon, and does very well on other benchmarks like Terminal Bench 4 and many others. So, we think that, you know, users who use it for coding, which will be a minority of all of our users, but the ones who do, will see a big improvement and a big step up.

• 21:51Speaker 01

  Okay, so we should also just be using Muse Wonder Full for coding at Stripe?

• 21:56Speaker 02

  Potentially, yeah. I mean, let's go out there and tell the...

• 21:58Speaker 01

  We're not even state of the art on Sui Bench, yeah. We'd be missing out.

• 22:02Speaker 02

  Yeah, yeah, no, it's a big improvement on coding. I mean, obviously, you know, we want people to try it and see how they experience it, but we've been very pleased in our own internal testing.

• 22:12Speaker 01

  Okay.

• 22:12Speaker 00

  Yeah, I think people will like it for a lot of things.

• 22:14Speaker 01

  Okay, so this is both just your latest frontier model, like, people can come, you know, developers can come directly to you, they can use it on an open router, like, they can just consume it as a model, and it's also a big upgrade inside of Muse that people will just feel as, you know, active users of Muse.

• 22:33Speaker 00

  Yeah, well, what we're gonna do in Muse, because it's a lot bigger, and it's more expensive to serve.

• 22:37Speaker 01

  Yep.

• 22:38Speaker 00

  We're, we're going to start off primarily in paid subscriptions.

• 22:43Speaker 01

  Yep.

• 22:43Speaker 00

  So what we want to do with Muse as a product, we want it to be free for everyone to use, so we have this huge amount of free tokens that people can use with Muse Spark as the model. And, but if you want to use Wonder for most of the hard tasks, then, then, at least for now, you'll use the paying version. But, you know, I mean, eventually, we'll, you know how quickly the field moves, and we hope to, you know, at some point, not too far from now, have a model that is as good as Wonder, but in the Spark form factor, and that will be a great thing to be able to serve hundreds of millions of people around the world.

• 23:18Speaker 01

  Okay, so you go to Muse, you subscribe to the, what's it called, like, the premium or pro plan, or pay money.

• 23:23Speaker 00

  Yeah.

• 23:23Speaker 01

  And just don't stop until you've taken out your credit card.

• 23:26Speaker 00

  Yeah.

• 23:26Speaker 01

  And then you'll be able to, you'll select it, or it'll just magically...

• 23:29Speaker 00

  Oh, it just gets better.

• 23:30Speaker 01

  It just gets better for you.

• 23:31Speaker 00

  Yeah.

• 23:32Speaker 01

  And...

• 23:33Speaker 02

  And then it'll also be available on the API, on the Metal Model API, and Open Router. And then it'll also be available for Muse code users. So, and then Muse Wonder Full will be available to subscribers of Muse Code. So, there's, it's pretty exciting, I think we want it, you know, as Mark mentioned, it is a much larger model, and so we're working through how to serve that most effectively to as many people as possible, but we're excited to get it in the hands of as many people as possible.

• 24:03Speaker 00

  It's a big step forward. I mean, this is like, and we really with Muse Spark, we were very happy with the performance that we were able to deliver as we were getting up to this point, but like, this is the beginning of like...

• 24:14Speaker 01

  Yes.

• 24:15Speaker 00

  MSL kind of being at the frontier. We feel like the models are quite competitive with the best models that are out there, and I think that this is...

• 24:23Speaker 01

  Okay, so you think it's like, Astro, Fable, and Wonder all together at the frontier?

• 24:30Speaker 00

  Yeah, well, I mean, you can look at the...

• 24:31Speaker 01

  Or just empirically, by the benchmarks, you think that's all?

• 24:33Speaker 02

  Well, yeah, say by the benchmarks, one would conclude that, yes.

• 24:35Speaker 01

  I see, yes.

• 24:37Speaker 02

  And I think actually, one of the, one of the things that we're especially proud of is, you know, we have taken the time, kind of as we've mentioned, to optimize models for the product form factor of Muse. So, this is the state of the art model. We think it really is the best model in the world for the use case of personal and professional agents.

• 24:56Speaker 01

  Yeah. People inside companies like complaining about Goodhart's Law, you know, a metric that becomes a target ceases to become a good metric. You hear complaints that models are benchmarked, and that at a certain point, kind of overfitting to the benchmarks actually makes the models slightly worse. As people who actually have to train AI models, is that true? Are the benchmarks pretty good at grading models? Do you start to see like, weird edge failure cases from them?

• 25:20Speaker 00

  The benchmarks have to keep moving, right? Because like, every, I mean, it happens very quickly, but they just get saturated so quickly. So, I mean, I do think that in terms of...

• 25:28Speaker 01

  But you think non-saturated benchmarks are fine?

• 25:31Speaker 00

  I mean, as we've gone on, we steer the models more towards our internal benchmarks for the use cases that we're focused on. So, I mean, early on, when we were kind of just doing the science of trying to climb the ladder.

• 25:45Speaker 01

  Yep.

• 25:46Speaker 00

  We cared more about what the external benchmarks were, just to make sure we were on the right track.

• 25:49Speaker 01

  Yes.

• 25:50Speaker 00

  And then I'd say, you know, each release that goes by, the larger share of what we're looking at are our internal ones.

• 25:58Speaker 02

  And one of the things that we actually quantified in our research blog post, as part of Muse Wonder, is this metric for big model smell, which is kind of, you know, big model smell and feeling of benchmarking are kind of like inverses.

• 26:10Speaker 01

  This idea that you could just like, tell when you're using a really big model, it just, you can tell.

• 26:15Speaker 00

  It just has nuance.

• 26:15Speaker 01

  That Janice Qua, yeah.

• 26:17Speaker 02

  And so we, the way we quantify that is actually almost explicitly what you're mentioning, which is the gap between in-distribution performance and out-of-distribution performance. So, what is the, you know, we kind of measure, what is, how well does the model perform on tasks that are in-distribution of our RL and post-training versus tasks that are out of distribution? And what does that gap look like for, what does that gap look like between the Muse Spark line and the Muse Wonder line? And we actually see this huge step up from the Muse Spark to the Muse Wonder line because of, as Mark mentioned, like, it's just a much better base model powering this new line, and there's an entire Janice Qua to that, but I think we've been able to actually quantify this effect.

• 26:55Speaker 01

  Yes.

• 26:55Speaker 02

  And it's a, it's an obviously dimension that we're excited to continue improving on.

• 27:00Speaker 01

  But if you're saying that like, external benchmarks have ceased to become as important, and you care much more about your internal benchmarks, is that kind of an argument against benchmarking, that you think that just trying to get the top score on the external benchmarks that Evan talks about is maybe not going to lead you to develop a great model?

• 27:17Speaker 00

  Well, yeah, but also, we're focused on specific things. I mean, part of the whole strategy is personal superintelligence, and just trying to build different types of products than other people are trying to build. So even if we did have the exact same evals as everyone else, it, you know, it just...

• 27:32Speaker 01

  Yes, and though I would say a personal superintelligence is not like a super niche interest, like, it's probably something that other models want to be able to do.

• 27:40Speaker 00

  I think that there are things that are very different about it. I mean, I think like the personality probably matters more. I mean, one of the things that we talk about is this idea of discretion. Right? So if you want to like, use the model to get something done, it needs to be able to know a lot about you and get things done in the world while disclosing as little as possible. So, you know, if you're, when you have an allergy, or you're pregnant or something, you're trying to like get a, you know, like a table at a restaurant, and you don't want to like, have to tell them your medical condition.

• 28:11Speaker 01

  Yes, yeah.

• 28:11Speaker 00

  You know, to make sure that it's going to be a good place for you. Like, okay, how do you handle that? And, I mean, humans have common sense. And the, the model so far, if you're building a coding agent that works in an environment where everyone has access to the same context, discretion isn't particularly important. So there are all these things that I think are just kind of more important for us.

• 28:33Speaker 01

  Yes.

• 28:34Speaker 00

  Aspects of coding where the code needs to work, but you don't, the consumers who are using it may not even ever see the code. I mean, that probably matters more in a personal superintelligence context rather than, you know, serving software developers. So I think these things are just a bit different.

• 28:50Speaker 02

  Yeah, ultimately, we care about making useful models. So we want models when like, you put them into people's hands. They have a great experience using. And I think that is the most important metric for any model that we or other people produce. I think that, that, you know, really, really focusing on the external benchmarks, like, if you really dig into the supply chain a little bit, like, it's some PhD student who made, you know, these benchmark, a benchmark as part of like, their, you know, as the course of them doing their PhD. And so you're kind of outsourcing all of your product thinking to that PhD student who happened to make a viral benchmark.

• 29:26Speaker 01

  Right.

• 29:26Speaker 02

  So clearly, like, that's wrong.

• 29:28Speaker 01

  Yeah.

• 29:28Speaker 02

  And so that's why we care a lot, like, a huge part of what a lot of people at Meta and MSL are focused on is crafting amazing evals.

• 29:36Speaker 00

  And now that we have them, we can work on our own. But if you're building like a, you're ramping up your lab, right? It's like, those public ones are not a crazy place to start. But it's just not where you want to end up.

• 29:46Speaker 02

  You're just, at that point, you're just looking for signs of life. But then when you're like, really in this mode, as we are now, where it's like, you know, we want to be very intentional about how we're improving our model over time to build better and better, better products. Like, a lot of the cases that Mark just mentioned, like, these are actually detailed, very rigorously in like, the spreadsheet of all the little capabilities and nuances of the product that we think really, really matter to the end user. And maybe each of them matters like 2% to the user, but in aggregate, that's like, the obsession that will ultimately result in these models that feel magical.

• 30:21Speaker 01

  Can you guys describe for me just what goes into training an RL model? Oh, sorry, a leading edge frontier model? I'm just curious, in that, I think a lot of people listening to this have a rough idea where there's, you know, there's a pre-training and a post-training step. So pre-training is you kind of ingest this huge amount of data. You know, you look at a lot of pictures, and these are cats, and these are dogs. And after a while, you start to, you know, develop a sense for what is catness, and what is dogness, and what is fur, and, you know, everything like that. And so that's kind of the, the pre-training step, where you just hoover up data, and maybe you can talk about the data acquisition, and just kind of what goes into that. And then there is this RL stage, which is the new-ish advance over kind of the past few years of AI.

• 31:03Speaker 01

  And again, it sounds like that is where a lot of the specific nuance of this model has come in, where you guys have better, have chosen better domains, better, you know, hills to climb for the model. But just at a high level, describe how many people, how long it takes, what the steps are, how you think about those in...

• 31:26Speaker 02

  How much time do you have?

• 31:27Speaker 00

  Yeah, I know, it's like, yeah.

• 31:28Speaker 01

  At a high level?

• 31:28Speaker 02

  This happens to be like, what we spend all of our time on, so...

• 31:32Speaker 01

  Exactly. And so summarize it in three minutes.

• 31:34Speaker 00

  Recommend some YouTube courses.

• 31:35Speaker 01

  Yeah.

• 31:36Speaker 02

  Yeah, I think at a very high level, you know, these are the stages. It's pre-training and post-training. Ironically, training is the superset of all of it, but that's...

• 31:46Speaker 01

  But how many people are on this whole effort? Like, when you develop Muse?

• 31:49Speaker 00

  Well, so now, so this, I think you're getting into like, what is interesting about MSL, and the approach that we've taken. Which is, I mean, from the very beginning, I think when I first reached out to you, I think we were just looking at this, because we were like, it was like, just like, wow, okay, how far we've come. And I was just like, I pulled up the screenshot of like, when I first reached out to Alex. And my first thing was like, hey, do you have time to catch up and talk? And then that's like, all right, we talked for 40...

• 32:11Speaker 01

  That's how they get you.

• 32:12Speaker 00

  44 minutes or whatever. And then, then after that call, two days later, Alex sends me this long thing, which is like, well, here's what I think I would do if I were in your, if I were in your position. And the very first thing is that you don't need a lot of people, and talent density is the thing that matters. So I'd say like, from the beginning of the plan, this was like, the core of your strategy, right? It's basically like, you actually don't need a massive number of people. But what you do need is for like, each of the people to be as good as possible. Which kind of informed this kind of crazy recruiting push, and like, very intense filtering that we had to take, you know, some of the best people from inside Meta,

• 32:57Speaker 00

  and a very intense process to get people from outside Meta. But I actually think like, a lot of what has made MSL special, and why a lot of people have wanted to join, is because, I think it's in some ways, it's just like more fun to work on a talent end smaller. I've heard it feels like the startup that's operating, in some ways, it has all the benefits of a startup with the stability and resources of a larger company. And this was like, this environment, that I think this was like point one in your kind of plan for what, before it, Alex was like, this is what we need to go do. So, I don't know, the answer of like, how big is a pre-training team? You know, it's, it doesn't need to be that big. But I think if you look at, you know, some of the other labs, it's probably quite big in some other places.

• 33:39Speaker 00

  So I think part of the reason why these models are doing well is because you, you have people who we're betting on, and they have a lot of resources, because we're, you know, we're taking our compute. We're not like peanut buttering across a lot of competing efforts, where basically just, you know, we're investing in like, the key people, like, a lot of, a lot of compute per researcher, and, and then they can, they can do their work.

• 34:01Speaker 01

  Was Wonder the one big training at first? Like, that was for most of the training?

• 34:06Speaker 00

  Well, it was the next pre-train. Right, it was climbing up this ladder. So I mean, we have this tiering system, and then kind of Spark. I mean, internally, like, Avocado was the code name for the Muse Spark line, and then Watermelon was the code name for this one. And we're like, out of fruits now, we're out of fruits. Yeah, so we're gonna have to switch to a, to a new convention, but...

• 34:25Speaker 02

  I think the only fruit that is technically larger is the durian. And you don't want to go to durian.

• 34:29Speaker 00

  No, no.

• 34:30Speaker 02

  Yeah, yeah, yeah. You want to stay far away.

• 34:32Speaker 01

  A durian is bigger than a watermelon?

• 34:34Speaker 02

  I think on average.

• 34:34Speaker 00

  It's definitely heavier.

• 34:35Speaker 01

  That's horrifying.

• 34:35Speaker 00

  Definitely heavier, spikier, spikier, yeah.

• 34:39Speaker 02

  But no, yeah, I think, I think as, as Mark mentioned, like, you know, each of the, I mean, each of the stages, stages I'll describe, like, it's tens of people working on it at Meta versus, you know, hundreds.

• 34:50Speaker 01

  Oh, so Meta's a huge company, but this is like a really tiny effort inside of Meta.

• 34:53Speaker 00

  Yeah, I know, the research team is less than 200 people.

• 34:56Speaker 02

  Yeah, it's very, it's very small. It's intentionally very, very tight. I think that the...

• 35:00Speaker 00

  It's probably closer to 100. And then there's a bunch of info folks.

• 35:03Speaker 01

  Yeah.

• 35:03Speaker 02

  Yeah, and then it's, it's a lot of info work. So I think at a very high level, to go back to the, to the, what is it question? You know, how, how does it happen? How does model happen?

• 35:14Speaker 01

  That's basically what I'm asking.

• 35:15Speaker 02

  Where do models come from?

• 35:18Speaker 02

  You, you take a huge amount of data for pre-training. And here, you're, you're...

• 35:25Speaker 01

  And is here, like, roughly, it's like, all the, you know, public internet, and like, all the books you can get your hands on, and just like, as much data as possible.

• 35:33Speaker 00

  But different types.

• 35:34Speaker 01

  Plus the proprietary metadata.

• 35:35Speaker 00

  But also different modalities. Right, so if you want it to be good at perception, then you want to make sure it has a lot of images, and audio, and video, and this kind of stuff. You know, if you want it to be good at certain domains, you don't just want to RL that in at the end. You want to have it, have exposure to that from the beginning.

• 35:53Speaker 02

  Yeah.

• 35:54Speaker 01

  Yeah.

• 35:54Speaker 02

  Yeah, so, and in this stage, you're kind of, you're much less, you care a lot less exactly what the data looks like, or how it's formatted, or, you know, all of, everything about it. But it's, but it's just lots and lots of data.

• 36:05Speaker 01

  When you talk about this being the best personal agent model, was there data you had to get for the pre-train? Where it's like, huge amounts of restaurant reservation call book? I don't know what it is, but like...

• 36:16Speaker 00

  You just kind of want to dump in large volumes of the modalities that people are going to care about. I mean, there were a bunch of things. I mean, I don't actually think, we talked about this when we were talking about where Muse Wonder excels.

• 36:27Speaker 01

  Yeah.

• 36:27Speaker 00

  But some of the areas that, it's like, all right, what do people care about, right? It's, I mean, there are things like shopping that people care about a lot, personal advice that people care about a lot. People have personal legal questions, personal health questions, personal tax questions. Muse Wonder, I mean, is like, quite good.

• 36:44Speaker 02

  Yeah.

• 36:44Speaker 00

  I don't know if there's any quantification that we want to give, but, but it's, um, let's say it's, it's, um...

• 36:50Speaker 01

  It's very good on those personal advice questions.

• 36:51Speaker 00

  Yeah, and so those types of things, I mean, you can RL them to some degree, but you like, just want to put in a large amount of data. Like, you can imagine health is a very, there's a lot of data out there about health. And in, in pre-training, you just wanted to like, see all these different things. And so you, you kind of just put these different corporate end.

• 37:11Speaker 02

  Yeah. So, so that, so as, as Mark was mentioning, like, the, we...

• 37:17Speaker 02

  One of the highlights of MuseWonder, actually, is its ability on various domain expertises. And I think our philosophy here is that the best personal agent should feel like a team of experts in your corner. Like, it should feel like it is your legal team, and your finance team, and your medical team, and and and and. And so, on a bunch of these domains, MuseWonder is actually state-of-the-art on, I think, a lot of the finance evals, as well as a lot of the legal evals. It is the state-of-the-art model. And these domain areas, that starts in pre-training. We look for a lot of domain data to put into pre-training.

• 37:57Speaker 01

  Yes.

• 37:58Speaker 02

  But a lot of the art of pre-training, I mean, pre-training is, there's this one Rune tweet where it's like, "Pre-training is full of the researchers who chew glass and stare out into the distance, into the abyss," or whatnot. But I think there's a truth to that, because it is like, it is both the highest stakes of the research areas. Because you, we invested a huge percentage of our compute into training MuseWonder, into pre-training MuseWonder. And it was very high stakes. I think probably both of us, at various points in that process.

• 38:30Speaker 00

  Yeah, you went through troughs of sadness.

• 38:31Speaker 02

  Troughs of uncertainty. You know, all the seven stages of grief happen while you're pre-training a model. And it is the most fundamentally scientific in an interesting way, because you just don't have very many samples. So, like, your ability to infer exactly what will happen scientifically really depends on your ability to, sort of like, you know, big-brained reason about everything that's happening in this process.

• 38:57Speaker 01

  Okay, so in the pre-training stage, everyone who's training frontier models basically is trying to get as much data as possible for the pre-train, with maybe a bent towards, if you want MuseWonder to excel, ask legal and finance and tax questions and things like this, then you want to have as much of that specific corpus as possible in the pre-train.

• 39:17Speaker 02

  Yeah, you still want as much intentionality about where ultimately you want your model to be. But again, it just benefits from a lot more data. And this is sort of the massive data, massive compute stage.

• 39:28Speaker 01

  Do you hit some saturation in the data you put in the pre-train? Or ultimately, is it just more data is better?

• 39:34Speaker 00

  Well, there's a relationship between if you want the model to be bigger, you need more data. Otherwise, it overfits.

• 39:41Speaker 01

  Yes, but my point is like, is there data where it's like, "Oh, we can't distract the model with that. We can't clutter up its brain attic with knowing that stuff," or is it just more data is better?

• 39:50Speaker 02

  There is some, there are some of these effects, I think.

• 39:53Speaker 00

  But this is part of the science.

• 39:54Speaker 01

  Okay, we're getting the secret sauce, sorry.

• 39:55Speaker 00

  No, it's not even that, it's just like, I mean, I think that there are all these questions about like, "Okay, well, we put a lot of these tokens in, is there any zero-sum-ness in a bunch of places where it isn't, or it's actually positive transfer?" But then, like, depending on how it's set up, like, you can have some negative transfer. And so, I don't know, there are all these interesting questions. But in general, one of the bigger challenges of scaling bigger models is it's not just about how many flops and how much compute. You want to be able to scale data with that. I mean, this is like, part of what, when the original kind of scaling laws for pre-training were written. I mean, this is sort of what came out of that, is that. So, if the internet isn't growing that quickly, and you want to scale bigger models that can be more intelligent,

• 40:39Speaker 00

  I think one of the interesting problems is, all right, where's the data coming from?

• 40:42Speaker 01

  Yes.

• 40:42Speaker 00

  So, I think that that probably gets to the end of what we're going to talk about on that. But we can go to the other.

• 40:51Speaker 01

  Excellent CEO energy over here.

• 40:54Speaker 02

  But so, post-training. So then, I think, post-training is kind of, is a very exciting field now, I think. And is an area where, I think, actually, I think, almost on a continual basis, the entire field is learning about the optimal recipes to be able to post-train these models.

• 41:12Speaker 01

  And again, for people, like, post-training is where you give the model a task that you expect to do greatest on how it does. And so, if it's a chess-playing model, it plays chess games, learns this.

• 41:22Speaker 00

  Well, there's actually two stages. There's the SFT, so there's supervised learning. So, that's like, a lot of when you take this kind of raw thing, and you give it both the question and the answer that you would want it to have, all right? If you're having a chat conversation, how you want it to respond in this case. So, when you're training things like personality, the SFT is actually like, a very important part for molding it. But then, I think, most of the advances in the recent years have been in RL.

• 41:47Speaker 02

  Yeah, yeah, so I think, I think one mental model that can be helpful is like, in SFT is where you kind of initialize a lot of...

• 41:54Speaker 01

  SFT is supervised fine-tuning.

• 41:55Speaker 02

  Supervised fine-tuning, yeah. Where you initialize a lot of the behaviors that you want the model to be able to exhibit later on. And then, you know, maybe at the end of SFT, it has those behaviors kind of baked in, but it'll only do it like 10% of the time. And then, in reinforcement learning, you really sort of give the model reinforcement in various situations to bring the elicitation of those behaviors dramatically up. And I think that the, yeah, so I think there's a bunch of interesting things that are happening in RL and post-training. I think one is that, just the scaling of RL and post-training has been, has been one of the clear axes, just spending more compute and gathering more data.

• 42:37Speaker 01

  So, just doing much more of it before you release a model?

• 42:40Speaker 02

  Yes, doing much more of it, especially as the pre-trains scale up, doing much more.

• 42:44Speaker 01

  It's like anything in scaling laws, like, it has to go with the rest of the factors, or it doesn't work.

• 42:50Speaker 02

  Yeah, in general, I mean, if I were to give, say, like, one broad statement about AI, it's just scale everything, you know?

• 42:55Speaker 01

  But... But in particular, you have to scale everything, like, scaling one thing in isolation does not work.

• 43:01Speaker 02

  You technically, you can scale one dimension, you just will get much more optimal performance if you scale many things altogether.

• 43:09Speaker 00

  Well, I mean, so one way to think about, like, MuseWonder, right? It's like, all right, so we did the pre-train, we've done some RL, right? When you look at, like, MuseSpark, right? So, version 1 to 1.1, 1.2, 1.3, we didn't change anything about pre-training. We just continued doing more RL on top of it. So, when you think about the early versions of a model of a new class, they're actually relatively less, like, the RL to pre-training ratio is going to be lower for RL than the later versions of that class.

• 43:39Speaker 01

  That is kind of what the point releases in models are.

• 43:42Speaker 00

  Well, I think every company has its own convention. For us, that's what it is meant.

• 43:45Speaker 01

  But for a lot of companies, like, the big number is the pre-train, and then the...

• 43:49Speaker 02

  I think it kind of lost that semantic value. Yeah, I think now you actually can't...

• 43:54Speaker 01

  Okay, but it's true at Meta, at least.

• 43:55Speaker 00

  Well, we haven't even done it, we've kind of decided to not even give Muse...

• 43:58Speaker 01

  It's true of MuseSpark, at least.

• 44:00Speaker 02

  It's true of MuseSpark and MuseWonder. It is true that MuseSpark and MuseWonder are different pre-trains.

• 44:03Speaker 00

  And we decided to kick the can down the road on this question for MuseWonder by not giving it a number yet. But when we have our first dot release, we will have to decide.

• 44:11Speaker 01

  Yeah, yeah, yeah.

• 44:12Speaker 00

  But, yeah.

• 44:14Speaker 02

  But, you know, I think the RL stage, you know, part of what has been, you know, part of what's very interesting is, first is just producing the environments and producing the tasks. And I think that is one of the areas where there's a lot of intentionality in training these models. It's also one of the areas where, you know, you notice very funny things. Like, if you, you know, in a lot of cases, less is actually more when it comes to data in RL. And so, I think, I think the sort of like, the feel and the science of it, like, is pretty flipped, generally speaking, from pre-training. The other is that I think it can give very magnified kind of instruction to the model, if that makes sense.

• 44:54Speaker 02

  So, I think that, like, a lot of safety cases or alignment cases that we care a lot about are like, you know, and, you know, where you would see when you do large, high-compute RL runs on frontier models, are where you apply a lot of RL or optimization pressure in scenarios where the only way for the model to succeed is to do something misaligned, or do something unsafe. And so, that's another thing we need to be very careful about and thoughtful about in training MuseWonder, is ensuring that we curated the RL environments very thoughtfully to ensure that all of them were eliciting and driving towards more aligned behavior from the model.

• 45:30Speaker 00

  But the point that you're saying about quality, I think, is an important one, because I think a lot of the previous questions back and forth were about the quantity, and had the flavor of like, "Oh, you need to scale the quantity."

• 45:39Speaker 01

  Yes.

• 45:39Speaker 00

  But, I mean, I do think, like, a huge part of this is just, is the point that you just made of, like, the, even a smaller amount of high-quality.

• 45:47Speaker 02

  Yeah, I think, I think in RL, like, we, you know, as we've seen pretty big jumps in our MuseSpark line, I think that these have been, like, accompanied with, you know, real, meaningful improvements to the underlying recipe. And each of these improvements in the recipe have been like, you know, I think the improvement between MuseSpark 1 to MuseSpark 1.3 feels night and day. I think probably to most people, if you ask them, like, "Are these like, you know, do these feel like they came from different pre-trains?" I think they probably would say yes, because I think you can, you know, the, I think it's happening industry-wide, for what it's worth. Just like, we are learning to squeeze so much capability and so much,

• 46:30Speaker 02

  and elicit so much agentic capability from, you know, these existing base models. Now, that's not to say, like, we believe scaling is very important, that's why we have the MuseWonder line in the first place. But, you know, this sort of like, these two things are happening at the same time, which is also, by the way, for us, this is why we're seeing so much progress in general. Like, I think it's the fact that we're scaling both pre-training and scaling our recipes in RL, kind of like concurrently, at a nice clip.

• 46:57Speaker 00

  Yeah, and then there's these other scaling axes, too. I mean, like, making sure that the agent harness is good and that you can co-optimize these things is probably just another aspect of, or dimension for scaling.

• 47:08Speaker 01

  But if I'm understanding you guys correctly, are you saying that alignment is not about coming up with some alignment-specific RL steps, and instead about just making sure the entire RL process is kind of steering?

• 47:23Speaker 02

  You have to be thoughtful about it in the entire process. Yeah. So I think this is a, it's a holistic effort, I think.

• 47:30Speaker 00

  But also, I think different labs have very different theories. I mean, alignment is a very philosophical concept, right of like, who do you want it to be aligned to? And a lot of our theory is like, okay, we care more about it being aligned to, you know, we're trying to give people agents, right? And it needs to, if you ask it to do something, you need to have a high confidence that it's going to do what you ask it to do. You know, like, you want it to do that with high reliability. You don't want a moralizing lecture for why you're wrong for asking it to do the thing. Obviously, if you're trying to do something that's catastrophic, it needs to gracefully, without pissing you off, like, not do it. But, but in general, in a we view alignment as more of a question of an instruction and

• 48:13Speaker 00

  values alignment with the person who it's intending to serve. And I think that that's, that's kind of a different mode of thinking about it. It's very different from, I think, how some of the others do.

• 48:25Speaker 02

  But one of the more interesting things, I think, that is maybe a question related to your original question, is also just how long the process of training one of these models takes, like Muse Wonder, I mean, in general, like, we've planned Muse Wonder since, like, last, you know, August.

• 48:43Speaker 00

  Yeah, I remember the offsite when we were.

• 48:47Speaker 02

  So this is like.

• 48:48Speaker 00

  When they first presented this, yeah.

• 48:49Speaker 02

  So I think all of the stages that we just talked about are like, you know, you really work on them for quite a long time. And it's not that we, we didn't start working on Muse Wonder when we finished Muse Spark. It's that we've actually just been planning and working on this for quite a long time. And we really view Muse Spark, if anything, as like, I think we mentioned this in the first blog post where we wrote about it, that was, that was a stepping stone to get to Muse Wonder. And that really was how we thought.

• 49:17Speaker 01

  Even at the time, you thought about it.

• 49:18Speaker 00

  Yeah, and in terms of how long it takes, I mean, we're also even just talking about the research part. I mean, there's also the compute part where we've been investing and building up, first of all, a huge amount of compute overall, but a lot of, you know, part of our philosophy on this is that you can kind of measure the compute strength of a lab by how big its largest single computer is, supercomputer for being able to do these, these large training runs. So, I mean, we built this Prometheus cluster, we call it, in Ohio, that's, you know, scaling to larger than a gigawatt. And now we're building this Hyperion cluster in Louisiana that's going to scale to five gigawatts. And those actually predate MSL.

• 50:04Speaker 00

  Because like, building a whole, like, data center site from scratch, at least the plans for those, they're coming online now, but that was.

• 50:13Speaker 01

  You guys thought it might be the feed, right? That you just have used for a lot of compute generally?

• 50:17Speaker 00

  No, no, no. This is, this is kind of predating, even going back to the, to kind of the Llama program.

• 50:22Speaker 01

  I see.

• 50:24Speaker 00

  I mean, it was probably in 2023, or yeah, either 2023, 2024, was when we basically started making the plans to build those clusters. They're now kind of fully coming online. Like, the Louisiana one will come online over, over a couple of years. But, but if you want to go really end to end on, on doing this stuff, then, I mean, obviously, the, the kind of full stack, and I mean, Meta is a, you know, full stack tech company. I mean, we build, you know, we build our own data centers, we build our own chips, we build our own low level infrastructure. And now this is kind of the next component, which, I mean, arguably is probably the most important component in all of this for the next wave of technology is, is kind of building

• 51:07Speaker 00

  really good models and then delivering them to people. But, I mean, yeah, if you want to go like, fully end to end, I mean, this is a, it's a, yeah, long process.

• 51:16Speaker 02

  I do think this is like, a, I mean, maybe it's well appreciated by folks in the, in the tech industry, but like, it really is, you know, it feels like models are coming out constantly. So maybe you'd have the mental model that like, oh, a company like launches a model and then they figure out how they're going to make the next model and, or something like that. But it's actually these like, very long planning processes and these sort of like, very long work streams that kind of all come together at the very end and then result in these beautiful models.

• 51:41Speaker 00

  Yeah.

• 51:42Speaker 01

  How do you build good products when the feedback cycle is so long? Like, ultimately, you start planning the model in August and you don't know until more than a year later if it's good. And, you know, Silicon Valley famously likes to run on metrics, but like, ultimately, do you just need someone who is the esthete and says?

• 52:01Speaker 00

  It's nice when you have metrics, but I think the like, state of being an entrepreneur is like, how well do you make decisions in an information poor environment, right? So.

• 52:11Speaker 01

  But this is like bat on steroids, right?

• 52:13Speaker 02

  Mega steroids. Yeah, I think that this is, and I think Mark gets a huge amount of credit for this, like, the, because of how long everything takes, like, there, there were many points along over the past year since I've been here, but also, you know, as you mentioned, like, long before that, where you have to make very large investments or make very big decisions about, you know, how you're going to invest or develop the technology or, you know, what decisions you're going to make, long before there's proof points around, you know, whether or not even the thing is working. And so I think this is, this is like, you know, in some sense, like entrepreneurship on a hard mode, it's like the opposite of lean startup. Like, lean startup is like, yeah, you like, see if you get something working and then you double down and then you see if that works, then you double down.

• 52:58Speaker 02

  Like, you cannot do that in AI. You have to, like, we've, I mean, we've been investing into these data centers since, you know, for years and years and years, we've had to.

• 53:08Speaker 01

  And you'll be radically behind if you waste to see if things are working.

• 53:11Speaker 00

  So we have these conversations, we have these conversations where it's like, at the end of last year, we're like, all right, we are quite excited about the progress. We haven't seen the models yet. Like, it's like, I'm gonna like, you know, we're gonna write this check for like, I don't know, like a hundred billion dollars or two hundred billion dollars. I'm like, Alex, you sure this is gonna work?

• 53:32Speaker 02

  So. It was, it was so extreme that, you know, we had, we had like, the whole exec team.

• 53:37Speaker 00

  I think Susan scared you a bit.

• 53:38Speaker 02

  Yeah, well, yeah, no. Well, it's like, the, the joke, every time we talked about, we'd just be like, yeah, and we hope it's all gonna work and be good. But like, they were literally, like, the end of last year, other than me giving a vibe report or, or like, research was going and, you know, it could sort of show some plots and whatnot. But it was, it was like, I mean, I was feeling good about it, but it was like, certainly not clear at all that it was going to work.

• 54:02Speaker 01

  Yeah. You guys are saying that the essence of training a frontier model is you have to push your chips all in before you've been dealt the cards. You know, you just have to decide that you're going to develop a frontier model and you don't get metered.

• 54:19Speaker 00

  At this scale, I mean, if we, like, back a few years ago, the clusters didn't have to be this big, but now they, they do. So I think this is part of the, the excitement of, of like, booting up a frontier lab now, right? Is, is basically like, the ante is, is extremely high.

• 54:35Speaker 02

  And I think this is, and like, I kind of like viscerally remember this, we were in Hawaii at the end of last year, it was kind of like, the whole exec team got together. And, and yes, Susan pulled me aside many times and was like, you know, I just want to tell you, if, if this doesn't work, like, you know, this is gonna be really, really bad. It's gonna be very, very bad in financial terms.

• 54:58Speaker 01

  I really need to impress on you that this model has to work.

• 55:00Speaker 02

  It's like, it has to, it has to be good. If it's good, it's all good, we're great. But, but if it's bad, it's really, really, really bad. And Susan, who you know.

• 55:08Speaker 01

  And that was useful motivation.

• 55:09Speaker 02

  It was, well, it was like, it was one of these very funny moments where I was like, you know, the, obviously, the amounts of money are like, so, like, obscenely large.

• 55:20Speaker 01

  What are the amounts of money in training, order of matter, whatever you can share, but just like?

• 55:23Speaker 00

  Well, you can do the basic math. I mean, a gigawatt cluster, I mean, it's like, I don't know, I think what you'd see out there would be, you know, to operate a gigawatt a year, call it on the order of fifteen billion dollars.

• 55:36Speaker 01

  Okay, so it's roughly a fifteen billion dollar effort to train a frontier model?

• 55:40Speaker 00

  To operate, to operate a gigawatt cluster for a year, if you're talking about building the CapEx, it's probably closer to, you know, I mean, you pay for it upfront, so it's probably, you know, fifty or sixty billion dollars. So, but then, that's just kind of one cluster for one part of training.

• 55:57Speaker 01

  Yeah.

• 55:59Speaker 00

  Then you have all the other parts. Research isn't just doing the run, right? It's like, that's like, probably an equal amount of it is because for each change that you make, you're testing all the ladders, all the way up to the big, the big scale. And then they're serving the stuff. So, I mean, part of the stuff that, in some ways.

• 56:19Speaker 01

  Tens of billions of dollars to train a frontier model now?

• 56:22Speaker 02

  Yeah, definitely the whole, I mean, I've seen the financial rollups. Tens of billions of dollars is a good estimate for.

• 56:28Speaker 00

  Yeah, and then, but then you want to do something with it, right? So I actually think like, some of the more complicated questions that we had, we're like, all right, so we're training this model and we know, we know we want to train the model because this is like key, but like, how much product demand do we think we're gonna have? Right? Because, so now it's like, all right, well, the product doesn't exist. Like, and.

• 56:49Speaker 01

  As in, you're trying to build data centers to serve Muse for a product that doesn't exist yet?

• 56:52Speaker 00

  Yeah, and we have to make, in years in advance, right? So this is probably where, like, when we were talking about it last year, where it's like, huh, there was a moment where I think a bunch of people were like.

• 57:04Speaker 00

  Yeah, maybe we should just be a little more conservative. And it's like, no, we're doing this. It's like, it's like, we're gonna make it work.

• 57:10Speaker 01

  We're gonna make it work.

• 57:12Speaker 00

  This is gonna work. And.

• 57:15Speaker 02

  But it was, it was like, I mean, even, I mean, I too am an entrepreneur and founder and have, you know, at various points in my life, taken risk. I remember the visceral feeling at the time was like, this is a fuck ton of risk. Like, I, I don't know if I.

• 57:29Speaker 00

  Oh, it's only risky.

• 57:30Speaker 02

  Yeah, yeah, it's only, well, I do think this is like one of, um, one of the like, uh, things that I think Mark is very impressive about, which is, I think you have really internalized this concept of, you know, it's about how big it is when you win, not like, you know, failure is okay. It's just like, how big, how big is the win when you, when you do win? And this was, I think.

• 57:52Speaker 01

  Mark has fully internalized power laws in a way that many people have not.

• 57:55Speaker 02

  I think quite deeply, yeah.

• 57:56Speaker 00

  Well, I think the other piece is, I don't actually think it's quite as risky as this. I mean, the numbers are big, but I think.

• 58:02Speaker 01

  Meta has a backstop in terms of how it can use compute.

• 58:04Speaker 00

  Yeah, so I think there's two things. One is if AI works overall, plenty of other people want all our compute, right? So they're willing to pay like, I mean, if, if like, the cost is whatever, 15 billion a gigawatt.

• 58:14Speaker 01

  We can just be a neo.

• 58:15Speaker 00

  We could get like a lot more for that. So, so I think that you, in order for it not to work out, it's not just that we have to fail. AI overall has to fail.

• 58:23Speaker 01

  Yes.

• 58:24Speaker 00

  And then even in that happens, we still have all these other uses to productively use compute. So in the grand scheme of things, we were definitely building out this compute for inference ahead of knowing that the product was going to be ready.

• 58:41Speaker 01

  But your AI peeled enough that you said, if you use online empowered, will be useful to someone.

• 58:46Speaker 00

  Yeah, but also like, we believed in what we were building. I mean, it's like, I, I think, um, and you can look at the, kind of the trajectory where it's like, all right. First there's ChatGPT is the text based assistant. Then, you know, Anthropic got coding agents to work and no one had built the kind of personal consumer version of that. And it just seemed like this is the thing that Meta should build in the world. And I think that that was just like pretty clear to you and me and Nat, like from the beginning, right? That that's like, that's kind of what we're here to do. And so, I don't know. I mean, there's always like, the real question for me on the Muse launch even was, you never know when you launch something, how it's going to be received, right?

• 59:28Speaker 00

  So there's always the case where you get a homerun, which in this case it was, and it's just like extremely well received right off the bat. But the normal case would be you kind of, it's fine and you have to like iterate a few times to get product market fit before you really grow. In some ways, like I actually think the product, we maybe could have even grown it faster, but we didn't have all the growth stuff really ready to go when we shipped it. Because we are like, our median expectation was like, all right, we're probably gonna have to like iterate on this a bit and like, that'll be, that'll be fine. But I don't know. I, I think like, if it takes a few iterations, it's like, not the end of the world. That's normal.

• 1:00:05Speaker 01

  Yes.

• 1:00:06Speaker 00

  You can have launches where it's like, you get like, it's kind of catastrophic and then it takes a while to recover, but that's not good. But, but it's like, but I mean, even if it took a few iterations, it would have been fine. So I don't know. I guess it just seemed to me like, this was gonna be, yeah, the numbers were big, but it seemed like it was probably.

• 1:00:22Speaker 01

  Was it bonding for the executive team to be holding hands, taking this risk together? You're talking about Susan taking your side.

• 1:00:30Speaker 00

  Bonding or anti-bonding? It's like, I mean, it's stressful.

• 1:00:33Speaker 02

  I think it was.

• 1:00:34Speaker 01

  Both, both. Yeah, being in a stressful situation together is generally bonding.

• 1:00:38Speaker 02

  Well, one thing that I think is quite notable about Meta is the degree to which like, first of all, most of the exec team has been there quite a long time. Like Nat and I were really newbies and, um, and, and, you know, most people have been through crazy situations many times in the past as well. And I think, I think Meta as a company is like an extremely resilient company that has been through like, you know, many situations that also at the time would have seemed incredibly daunting. And so everyone's kind of like, knows the program, so to speak, in terms of, you know, this obviously on, on paper,

• 1:01:17Speaker 02

  and especially when like Susan Lee, our CFO, is sort of like pulling me aside in the meetings, like, does, you know, does seem like a particularly scary situation. But I think, I think most people were like, yeah, yeah, no, but we're Meta, we're gonna get through it. Like, I think there was like a, like a real camaraderie around that and a, and sort of a, like quite a nice, um, quite a nice energy around that. Which, which, which really I did appreciate because I think it would be very easy for, you know, it's the other executives of the company who are, um, operating and running.

• 1:01:48Speaker 01

  Puring the bills.

• 1:01:49Speaker 02

  Yeah, one of the greatest businesses of all time.

• 1:01:51Speaker 01

  Yes.

• 1:01:51Speaker 02

  And, uh, and it'd be very easy for them to be, uh, quite, uh, quite annoyed by this whole endeavor. But, but I think, I think everyone is also very excited about what the, what the opportunities for the company with AI are. And I think like, you know, taking a step back, I think one of the, um, one of Mark's jobs as like the founder and CEO of the company is to ensure that the company is always pushing towards the most ambitious and expansive version of what it can be. And I think that like, one of the great things about this entire MSL and AI push is that like, I think this is like one of the rawest expressions of that is like, you know,

• 1:02:33Speaker 02

  we, we knew that, you know, AI certainly is a very, a very important technology and it's, you know, one of the bigger technologies right now. But as a, you know, if it, if we were able to build it effectively and properly integrate it into everything that we do, as well as like, you know, as a full stack technology company, like build it into, you know, these incredible products that we know will exist in the future. This could be like immensely expansive for Meta. And I think this is true. Like, I think Muse is an incredibly exciting product. It has like, you know, it can go in a thousand different directions. I think is one of like, probably the most exciting products to just like, you know, when like late at night and you just think about all the ways it can go.

• 1:03:17Speaker 02

  Like, I think Nat, Mark and I just like, you know, shoot across messages late at night of just like, oh, we could do this and we can do this. And, you know, we can make this move and we can like build this product. And I think it's just like, um, you know, very, uh.

• 1:03:31Speaker 01

  It's a very exciting time in technology right now.

• 1:03:33Speaker 02

  Yeah, it's very blue ocean in that way.

• 1:03:34Speaker 01

  Yeah.

• 1:03:35Speaker 02

  Yeah.

• 1:03:36Speaker 01

  I'm struck by, as we talk about, uh, Muse and just how it clearly feels like the next platform, an agent operating on your behalf. You've talked for a long time about not wanting to be reliant on other platforms and how kind of the experience of building on Android and iOS kind of burned you on being dependent on other, and often when you would talk about, say, Reality Labs, is the importance of owning the platform of the future? And kind of many years working on Reality Labs with the intent of owning the platform of the future?

• 1:04:13Speaker 01

  And now this new modality comes along that feels pretty clearly like how you do a lot of productive work and is the platform of the future. And now Meta is really well set up.

• 1:04:24Speaker 00

  Yeah, no, I mean, it's less about wanting to own a platform for its own sake. It's about being able to build the best things. And at some level, you know, if you, you know, look at like other social apps, for example, I think part of the reason why Meta is so much better is because we don't just build the app, right? A lot of people like think about it, we're like this app company. We're not an app company, right? It's like, we're like a deep technology company and we do this because we are world class at machine learning and like all the recommendation systems are operating at a scale that basically few others are in the world. I think it's basically like just us and ByteDance on that. And, and then like, okay, we just kind of go all the way to the metal.

• 1:05:03Speaker 00

  And to me, being able to also deliver, go the last mile through the device, it does come from a while of having these experiences where we couldn't build the experiences that we wanted on phones. But I just think for the future, the, the models are going to be by far the most important part of delivering the technology experience. And I mean, this is something that I think we kind of bonded on and agreed on like very early, is this notion that like in every generation of technology companies, there's like, no, there's usually not one company that kind of is the, is like wins everything. I don't think that that's how this is going to go either.

• 1:05:43Speaker 00

  But with the internet, there were like, okay, five or six very big companies and then a whole ecosystem of other companies that are great. But the companies that did the best, um, kind of had the core technology to be able to, um, serve people the best there. And I think in this next era, there are going to probably be a small handful of companies that have the ability to build frontier superintelligence. And I think that those companies are going to be the ones that effectively drive the agenda and can build most of the interesting, the most interesting, um, experiences for consumers, for enterprises, hardware, these different kinds of things. So when I just look at it, it's okay, like, is it even really a choice, right? It's like, like, yeah, okay, Meta could, will probably exist into the future regardless of whether we do this.

• 1:06:28Speaker 00

  But if we want to be kind of an interesting company that is like inventing new things that matter to people in the future, I don't see how you can do that if you don't have the ability to build and to end your own superintelligence and build them into the products that people use. And I think that this is something that you just like believed from very early on, um, is basically it's like, all right, there are going to be like a handful of these. Like, it's like there's a kind of personal and consumer oriented version of this. And like this lane is open because everyone else seems to just be obsessed so much with just automating knowledge work, which, I mean, isn't even that charismatic version of the future, right? Because it's like, all right, like, I mean, a big part of our belief too is that the kind of highest use of superintelligence is invention and creation, not automation.

• 1:07:12Speaker 00

  And so, I don't know, it just seemed like there's like a big place in there, but you gotta, you kind of need to own the stack.

• 1:07:20Speaker 01

  How is the Muse mediated future going to shake out with businesses need to, okay, businesses variously need to build support for Muse, where it might try to computer use its way in, but that might be janky in certain ways. So you kind of want everyone to have first class support for agents talking to things.

• 1:07:49Speaker 00

  Yeah.

• 1:07:49Speaker 01

  Then the bigger question is they might actively try to block agents from interacting with them. And this might be because they have a misguided sense where like it's actually in their interest to support agents, but they don't want to. It might actually not be in their interest to support agents. You can imagine where maybe they have a very thin moat and it all comes from, you know, a business that only succeeds by consumers spending time on the site or only succeeds by overcharging consumers. Then like they should rationally not want to allow agents because the agents are like, why?

• 1:08:17Speaker 00

  But those aren't gonna, those are gonna get disrupted.

• 1:08:19Speaker 01

  Yeah. So talk to me just through the second order effects of like how the economy works.

• 1:08:24Speaker 00

  Well, I think that it starts off with, I mean, we trained into the models, very good browser use. Right? So that way we want to make it so that no one, no one has to do anything to integrate with Muse. And then, but then of course for, we're kind of just going down the list of the most important partners. And for people who want to work with us, we'll just get direct API access.

• 1:08:45Speaker 01

  Yeah.

• 1:08:45Speaker 00

  Right? And then it's like, okay, then it ends up being way more efficient and faster for people to use Muse. It's more efficient because now you don't have an agent crawling your website. It's just going, so anyone who wants to do that, we eventually want to get there. And then I think the fullness of time is that, you know, just like today, every business has a website and social media accounts and email addresses and all these things. Every business is going to have agents too. So, and we're also building that on the other side of it. And, and so of course Muse agents will talk to other types of agents, but they'll work really well with our agents, right? Because I mean, we'll, we'll be able to integrate that just, I mean.

• 1:09:25Speaker 01

  So Muse using websites is just a moment in time thing?

• 1:09:27Speaker 00

  And just to make it easy for everyone.

• 1:09:29Speaker 01

  You guys.

• 1:09:29Speaker 00

  Like you want to have it as a backstop. So it's reliable for people using it.

• 1:09:32Speaker 01

  Yes.

• 1:09:32Speaker 00

  And easy for businesses to get bootstrapped. But we have, we have way more businesses that are calling us, asking to integrate deeply. And very few really who have tried to block it. And my guess is that the ones that try to block it, it will end up not being the right strategic thing for them. But we'll see. This will, this will kind of play out over time.

• 1:09:53Speaker 02

  It saves them a lot of time. It saves them a lot of money. Like I think these are, these are like clearly good things for the consumer. So in the long arc, you'd clearly expect that demand will shift from current avenues more and more towards agentic demand. And you know, there always will be demand that lives elsewhere, just in the same way that like, there's still brick and mortar demand in commerce, but there's a lot of e-commerce demand. And kind of in the same way, like I think agentic commerce demand will grow as a portion of the pie. The others will still exist, but will grow as a portion of the pie. And I think there's sort of these questions for businesses in the long run. Like, do they want to participate in that slice or that portion of the economy over time? And I think it's an incredibly exciting area.

• 1:10:38Speaker 02

  I think it's like very, very exciting to imagine what will happen in this agentic commerce future. I actually think Patrick wrote a great.

• 1:10:45Speaker 00

  Yeah.

• 1:10:46Speaker 02

  Tweedical? Is that what we call it these days?

• 1:10:48Speaker 01

  I don't know. Patrick has his own thing going on with tweets. It's like they're buried in a.

• 1:10:52Speaker 02

  I noticed that Brother Karmazov was a, was a.

• 1:10:55Speaker 00

  Yeah, that was long.

• 1:10:56Speaker 02

  Quite a good literary analysis, but uh, but yeah, no, I think, I think like, I think clearly there's, this is a discontinuous event for how, you know, commerce will happen broadly speaking, or just generally speaking, how like the entire, how all the business pipes will work, maybe even in a broader sense. But I think that, you know, you just go back to, this is clearly a lot better for the consumers. It's a lot better for the customer. So there's a natural gravity towards that reality. Well, one of the exciting things, I'm curious what you guys see too. But I think, I think Muse is one of the first products where you're actually, people are fully delegating purchases to the agents. And this is like a, this is kind of like a, you know, a step change, right?

• 1:11:36Speaker 02

  Like I think, you know, we've been talking a lot about agentic shopping and like the possibility of it. But now it's like really happening and a lot of people are trusting the agents to shop for them. And I think the agents are finding them, you know, in my case, like great teas or, you know, who's ever case, like great products for them to purchase, which is quite exciting. And, and so I think that there's, there's clearly an exciting thing happening here. I think, I think part of the value prop, well, one, one part of the value prop for the user is just reduce friction. Like, hey, I don't feel like click on all these websites and do all this research to try to find the exact products, like that step is taken care of for me. And also like criterias can just be automatically applied. Like, hey, I need something by Wednesday. So, you know, the agent will only look for things that could get to me by Wednesday.

• 1:12:19Speaker 01

  But that's what I mean. Keyword search, it was insane that we ever relied on it. Cause that's a very reasonable constraint to want to shop by.

• 1:12:23Speaker 02

  Yeah, exactly. And, and, and so I think the like old way of like searching for stuff just feels so obviously primitive now, now that you can do this. Then there's the sort of like back part of the funnel, so to speak. But just like the, the ease of friction reduction of just, yeah, feeling it, just like getting it ordered is quite nice. And I think the long-term thing that we're quite excited about is, is greater personalization. Like ultimately, we think, I think one of the exciting things about Muse is that it is, has sort of like this holistic context into you as a person and all of the aspects of your life. So it's sort of like can holistically understand things about your family or things about your coworkers or things about your friends.

• 1:13:07Speaker 02

  And then like, you know, sort of like help you co-optimize across all of those to get you the exact right gift for your, you know, best friend in the context of your friend group, based on what it knows about your friend group. Like, I think there's just like a lot of exciting opportunities that, that are somewhat new and novel in, when it comes to like these personalization, these recommendations. Cause historically, a lot of these recommendation systems are quite siloed and quite isolated. And so they're sort of like, they don't see like a holistic view of you as a person. And Muse is, has an opportunity to offer much greater levels of personalization.

• 1:13:43Speaker 01

  Yeah. I think people are already seeing that where if they ask, like with Muse, if you ask it to buy something for you, quite frequently it'll add in some bit of context that it pulled in from a different, you know, conversation.

• 1:13:53Speaker 02

  You had a good example of this.

• 1:13:55Speaker 00

  Yeah. Well, so yeah, so my, my three-year-old daughter loves baking. She loves cooking. And so I'm like, all right, I'll start baking stuff. It's like, I'm normally into, you know, smoking meats, but like, but I'll, I'll do some baking in order to, to make, to make Aurelia happy. And so we've gotten into this really cute routine where, you know, we, we, I've been playing with Muse voice mode, which isn't fully rolled out yet, but we're, we're playing with it. And it does this great thing. It segments the, the voices. So it knows which one is me and it knows which one is not me based on that. And we just like, you know, throughout the week, we're talking about what we'll want to bake that weekend.

• 1:14:38Speaker 00

  And she'll like give it ideas or like she'll kind of come up with crazy things that are not real recipes, but like bounce it off with, with, with Muse. And, and then Muse will just make sure that we, every, you know, Saturday, when it's time to bake, that I get the ingredients from Instacart. And they, they kind of show up and they're on time, which I mean, the biggest thing is like, if I were just starting the activity on Saturday, would, you know, you'd have to get everything and it would take some time. But now everything's ready. Whenever we're going to go sit down and do it. This weekend it knew that watermelon was launching, so it threw a watermelon in the bag. It was very, very sweet. It was, it was very heartwarming.

• 1:15:21Speaker 00

  It deduced that, that I, that the recipe, it did not have enough, I, I, there was not enough, I guess, expense in it to get the delivery. So it needed something else. And it was like, you're gonna.

• 1:15:35Speaker 01

  And it topped it up with a watermelon.

• 1:15:35Speaker 00

  Yeah. It was like, do you want this watermelon? And I was like, yes. I'm like, yes. That's, that's very cute.

• 1:15:39Speaker 02

  Which I think you, uh, did you, is that the watermelon you?

• 1:15:41Speaker 01

  Yeah.

• 1:15:42Speaker 00

  Yeah. I ate it on the, on the.

• 1:15:43Speaker 01

  Okay. So, okay. But the watermelon in the live stream was.

• 1:15:46Speaker 00

  That was just like Muse just got it for me.

• 1:15:47Speaker 02

  Those of Muse, yeah.

• 1:15:49Speaker 00

  Just got it, just got it for me. But, but I think there's this.

• 1:15:53Speaker 02

  That was a very memeable moment of that very memeable situation. So we have, we have Muse to thank for it.

• 1:15:59Speaker 00

  So, but here's another interesting thing about the whole thing is I didn't really use Instacart that much before. You know, it's like every once in a while, maybe twice a year, I would like, like use Priscilla's account or something.

• 1:16:11Speaker 01

  Yes.

• 1:16:11Speaker 00

  I went and got an Instacart account to give Muse. And now I use it like once or twice a week because like I'm talking with Aurelia with Muse and we're coming up with these recipes and it's delivering them. And now it's like a core part of my workflow. So I actually think that the opportunity here isn't just, okay, it's going to be 20% more efficient on the conversion rate. Like I am a new Instacart like subscriber and I just wouldn't have done this before. So I do think that there's this question. And then there's this question, which I think is these sort of interesting. This is maybe the more interesting economic question. Is that Instacart account really for me or is it for the Muse? Because I think like over time, I think that the question is, is the total addressable market

• 1:16:54Speaker 00

  for all of commerce just increasing dramatically?

• 1:16:57Speaker 01

  Yes.

• 1:16:57Speaker 00

  Because it's not just the people anymore. Like now agents are effectively going to have accounts.

• 1:17:02Speaker 01

  Yes.

• 1:17:02Speaker 00

  And I think that that's where this starts to like become a really very exciting opportunity, where anyone who sits this out, I think is going to end up feeling sad over time.

• 1:17:10Speaker 01

  And why did you have to sign up for an Instacart account? Like, we're solving this at Stripe projects for developer stuff, where your coding agent

• 1:17:19Speaker 00

  The Stripe integration, by the way, is great. You guys did a great job with this on Muse. I think we're very happy.

• 1:17:26Speaker 01

  We've been very excited about agentic commerce for a long time, obviously, because again, it's felt to us like clearly how people are going to buy things in the in the future. And that was both apparent because no one needs all that friction in their life from the actual buying stage and how early product research with AI had product market fit, like even from kind of the earliest AIs. It really feels like computer use was like the last big unlock that we needed to make it happen, because now you can ask your Muse to buy anything, more or less, and it'll just figure it out with computers, right? And so it's like the ultimate backwards compatibility layer.

• 1:18:00Speaker 00

  Yeah, this was a set of things that Nat really obsessed over.

• 1:18:02Speaker 02

  Yeah, yeah, I think I think there was a lot of like, I think a lot of magic of Muse is the fact that like, a lot of this stuff just works. And so like, clearly, if everything was just like a little bit less reliable, it would feel like a total pain in the ass and nobody would use it and people would just throw away the app. And so I think the fact that so much of this has gotten to the point of reliability where, you know, it just works is like a huge part of the magic.

• 1:18:25Speaker 00

  Yeah.

• 1:18:25Speaker 01

  What were you saying Nat was obsessed with?

• 1:18:26Speaker 00

  Oh, well, I mean, well, Nat's not kind of.

• 1:18:29Speaker 02

  It's obsessed with everything, yeah.

• 1:18:30Speaker 01

  Generally obsessive character.

• 1:18:31Speaker 00

  Yeah, no, it's great, it's great. But this is, I mean, but yeah, I mean, this getting this experience to be good, I think, is something that that he and a core group of people on that team really focused on.

• 1:18:40Speaker 02

  It's kind of funny. I think I had a dinner with Patrick, like, I want to say it was 2023, where I was, he was like, what do you think we should be doing AI? I was like, oh, you're like, you're already way behind the eight ball on this agentic commerce stuff. Like, you got to figure this stuff out. Like, agents are going to be buying things and like, you're not prepared for it.

• 1:18:58Speaker 01

  So you were thinking this to Patrick or Patrick saying this to you?

• 1:19:00Speaker 02

  I was saying this to Patrick, well, because Patrick was very politely asking like, what do you think we should be doing? And I sort of had this like very like urgent spiel.

• 1:19:07Speaker 01

  Yeah.

• 1:19:07Speaker 02

  And I think it's very, it's funny how that was like.

• 1:19:10Speaker 01

  Thank you for your provocation.

• 1:19:11Speaker 02

  Well, it was directionally correct, but but like.

• 1:19:13Speaker 00

  When was it?

• 1:19:14Speaker 02

  Three years early, like 2023, I think. And so.

• 1:19:17Speaker 01

  It's very interesting to look at the agentic commerce stats at Stripe, because we started developing all this stuff and we had a whole bunch of announcements at sessions back in May, and then it only, as of September, basically started going vertical.

• 1:19:35Speaker 01

  Meta is building increasingly capable models, including Muse Wonder, and their models are available on OpenRouter alongside hundreds of others. Stripe has agreed to acquire OpenRouter, who's the leading model gateway. OpenRouter already routes more than 10 trillion tokens a day and handles much of the complexity of working with different models through a single API, making it easier to experiment, find competitive prices, and automatically route around provider outages. Try it for yourself at openrouter.ai.

• 1:20:07Speaker 00

  Hey, could you order a watermelon to Stripe's office in South San Francisco? I'm doing a podcast here, this would be funny.

• 1:20:17Speaker 03

  Funny podcast idea. I'll check that watermelon delivery now.

• 1:20:23Speaker 01

  Great, we'll see how we get on.

• 1:20:27Speaker 01

  Well, okay, this actually raises an interesting question. People have been trying to make the Hal 9000.

• 1:20:35Speaker 00

  It's working.

• 1:20:37Speaker 01

  I really like this. I think it's an awesome touch, where it's like clacking away. I really love that in the app.

• 1:20:44Speaker 00

  Just leave him there, we'll see what we got. It's tough to prop up.

• 1:20:49Speaker 01

  Yeah, there we go.

• 1:20:53Speaker 01

  People have been trying to make the speak to a computer AI vision happen for a really long time, where Apple launched Siri in 2014, 2015, something like that. And obviously, there's been voice mode in various AI products for a few years now. Why is it so hard? Like, it's still not the case. If you were driving your car and you wanted to check on the baking or move a meeting or something like that, you actually probably wouldn't do it with the voice. I mean, maybe almost now with your latest thing.

• 1:21:21Speaker 00

  Well, there's a lot. I mean, so the conversation that we're having right now is actually a good example, is we're talking over each other. And I mean, I don't know, when you're using different voice.

• 1:21:27Speaker 01

  Is it just technical limitations? That's why we haven't done it?

• 1:21:29Speaker 00

  Well, I think that there's a few things. And one is, you actually want it to go do stuff. So there's a question of when you speak with it, even if it understands.

• 1:21:37Speaker 01

  Yep.

• 1:21:37Speaker 00

  How's that going to work? Then I think that there's a lot of stuff about the way that people have conversations, it isn't typically, I will give a prompt, and now I will wait, and you will speak. And then, so it just goes back and forth. So how many of your sessions, when you're talking to an agent in voice mode, do you basically, oh, it starts saying something, but then like, I don't know, it's like, I kind of feel like if I pause for three seconds, it's going to think I'm done and want to speak. And then if it starts speaking, but then I cut back in, is it going to think that it's like a whole new question? I mean, there's a lot of the stuff about the conversational dynamics that you want to get right.

• 1:22:15Speaker 00

  But then there's this other issue, which is that for voice overall, it's quite high bandwidth for you to communicate with voice, but in terms of consuming information, having someone speak to you is very inefficient compared to having it be visual.

• 1:22:33Speaker 01

  Yeah.

• 1:22:33Speaker 00

  So I think like, that's why, you know, partially when we do these, these interfaces, I think it's kind of working on its task. It probably fired it off now. So I think the ideal interface is probably not just strictly voice, voice, although I think voice output from you is pretty useful.

• 1:22:50Speaker 02

  Yeah, I think I think there very clearly already is an explosion of voice input. Like, I think you even see this in coding, where like.

• 1:22:58Speaker 01

  People are speaking to the little.

• 1:22:59Speaker 02

  Yeah, most of the researchers just kind of like whisper into their into their keyboard. Because it is like, as Mark was mentioning, it's like the the highest bandwidth way to get your like thoughts out, or usually the lowest friction way to get your thoughts out.

• 1:23:11Speaker 01

  Yeah.

• 1:23:12Speaker 02

  I think the most extreme version of this, by the way, is like, I think how like college students now use AI is maybe the most interesting, where they just like, I think a very common modality of how to use AI is that they just will completely word vomit for like 10, 15 minutes. Like, the full, full sort of like stream of consciousness, shit, I thought of like all the things.

• 1:23:34Speaker 01

  And then have AI clean it up and infer.

• 1:23:35Speaker 02

  Yeah, and have AI effectively like, do all of this sort of like, organization of it and like pulling things together and and structuring everything. It's like, it's quite interesting. It's a very interesting, like, I think, modality of tech usage. But yeah, I think voice input has been big. I do think agents is like, that's a big step change, because I think the mental model is just very different. Like, if it's just like an informational thing that you're talking to, that like, you will clearly get into situations where it just feels useless, whereas like now that I can do things for you, it's like much closer to the experience of talking to a human or to a person. And so I think there's all these like, interesting uncanny valleys in like, interacting with

• 1:24:19Speaker 02

  AIs that until they've been able to take actions, like have felt kind of strange. And now that we have agents, like, I think one of the cool things about Muse is like this confluence of a number of things. Like, I think the fact that Muse is embodied and it like is this little guy that like you feel like is working on your behalf, is also only possible in a coherent, like, representation and embodiment of the product, because it has persistent memory and it has the ability to do things for you and the ability to take actions and has like a little bit of a personality. Like, is the confluence of all these things coming together that make this like embodiment concept even work in the first place?

• 1:24:59Speaker 00

  And it turns out, okay, actually, this is the real glasses, right? It's like, a lot of people want it, even with just like no display or a small display.

• 1:25:07Speaker 01

  Yes. The camera and audio is enough of a useful product.

• 1:25:11Speaker 00

  Yeah, yeah. And then we started shipping the display glasses that aren't the kind of wide field of view, full augmented reality, which I mean, I thought would be awesome too, but the even the small display is pretty powerful. So I think you'll start getting some interesting interactions there that are kind of fully multimodal, right? So it's voice in and out, and it'll be able to see what you see, and it'll be able to kind of paint the pixels.

• 1:25:39Speaker 02

  And one of the, I mean, the other exciting thing that we're, we've done here is Muse gadgets, which is really, I think, unleashing the ecosystem to try cool device formats and form factors that could be really exciting. Like, since releasing that, we've gone, actually, even a number of phone manufacturers, you know, talked to us about excited, you know, we're like, we're just going to integrate Muse gadgets into our phone and see how well it works. And like, we're really excited about that. Like, I think, I think part of what is cool about the current moment of technology is that, like, it's very clear agents are really exciting. It's really clear that like, this should unlock lots of things.

• 1:26:17Speaker 01

  Will you guys come back to the household devices? Like, this is obviously a personal device, but we actually had during COVID, the portrait?

• 1:26:25Speaker 02

  Portal?

• 1:26:26Speaker 01

  Portal, there we go.

• 1:26:26Speaker 00

  Yeah, I mean.

• 1:26:27Speaker 01

  Like, is it finally the portal time?

• 1:26:29Speaker 00

  One of the gadgets that we just shipped is the little home link. It's like the size of a little like nickel or something, but with a with a USB, right? And so you can basically connect that and then your Muse can connect to everything in your home. This is actually one of the, one of the most fun things too, is just having it.

• 1:26:47Speaker 01

  Yes.

• 1:26:47Speaker 00

  Connect to your Sonos and connect to all your lights. I mean, I have it basically, like, when it's time for me to start winding down in the evening, I just have it like dim all the lights across the house and it's, it's nice.

• 1:27:00Speaker 01

  How are we going to fund the AI build out? Everyone's worried about, you know, Google went free cash flow negative, you know, you were talking about one gigabyte, one gigawatts, and then five gigawatts and everything like this. Meta's not free cash flow negative yet, but there's always more CapEx to fund. And so what's the answer?

• 1:27:18Speaker 02

  Susan Lee.

• 1:27:19Speaker 01

  Exactly.

• 1:27:21Speaker 00

  No, I mean, look, I think.

• 1:27:24Speaker 00

  I think that there are going to be a lot of ways to create value from this. So what we're trying to do with Muse specifically is make it so that it can be widely used, because I think that there are going to be some valuable network effects at scale as people start being able to interact with each others. And that's going to be great. So I think there are all these reasons why you want it to get to large scale. Also, we were talking about partners. Partners are going to want to integrate more, the more people are using it. So it will get more valuable if we can get it to scale. The best way to do that is to basically have a model where it is affordable and free for people to use. But there are going to be a lot of ways that I think we can scale revenue. Subscriptions is going to be a big one.

• 1:28:06Speaker 00

  Basically, just because the base version of it is free and we want the free version to be excellent, doesn't mean that there are things that just naturally aren't going to be economical to include in that. So if you want the kind of, we talked about this earlier, if you want to start using Muse Wonder in Muse tomorrow, then the way that you'll do that is you'll pay for a subscription. So I think that that will be a big business. There are a bunch of other things around commerce that we're quite excited about, too, where we basically think, we believe that Muse is actually going to make people money and save people money by using it. And that over time, there are lots of different ways to effectively take a small cut of what you are saving or making for people, and that will end up being a very large revenue

• 1:28:51Speaker 00

  stream. And then there's all the other things, right? So

• 1:28:55Speaker 01

  You think Muse will generally just cost negative money for people because it helps them allocate their finances better?

• 1:28:59Speaker 00

  Well, there are already all these stories that I think are interesting, where people are like, go look through my emails, find a bunch of subscriptions that I don't actually use, cancel them, whatever. Yeah, so, yeah, I mean, that, I don't know if that's a thing that we would monetize, but you kind of get the point, and where it helps you make money. I mean, one of the ideas that we kind of talk about, and this is something that I think you've been kind of obsessed with, is making it like a business in a box for small businesses, so someone can just kind of, it can help them administer all these parts of what they're doing. And you can imagine that being very valuable and being very profitable, both for the business owner and for Meta. But I think one of the things that's interesting is like, okay, if we're building the models

• 1:29:42Speaker 00

  to build the personal agents and the business agent, there is this large surplus of value, because now the model, it's very general, right? So it doesn't just do those things. So we'll serve the API, that'll be profitable. A big part of why we want to start the Meta enterprise platform and have CJ decide, join us to build this, is we have this view, which is like, all right, yeah, sure. So traditionally, Meta hasn't been an enterprise company, but we believe that a huge part of the challenge is building the leading models in the first place, and they're very general. And once we do that, if we make a significant investment in building an enterprise group that can provide that in the way that enterprises need to consume it, then, I mean, I think a lot of enterprises are going to want to use Muse Wonder.

• 1:30:26Speaker 00

  It's going to be an excellent model and affordable. And so I think that that is going to end up being a very big business, too. And all of these things put together, coupled with the fact that we were already starting off as a very profitable business that can, that has good credit and can kind of underwrite these large investments, I think puts us in a pretty good position as a lab to be able to build out the compute that we need to build leading products, build up the network around this, and position the company to really lead in the area for decades to come.

• 1:31:00Speaker 01

  Okay, so you think ultimately there'll just be a lot of revenue from these products, and so there'll be self-funding in fairly short order?

• 1:31:06Speaker 00

  Yeah, I mean, that's a much shorter answer than mine.

• 1:31:08Speaker 01

  But you're saying people are underwriting?

• 1:31:11Speaker 00

  If the question was, are we going to be able to make money from this? The answer better be yes.

• 1:31:18Speaker 02

  I think the history

• 1:31:19Speaker 00

  Yes, the answer is yes.

• 1:31:20Speaker 01

  But the history is very front-loaded expense, right?

• 1:31:22Speaker 02

  Well, the history of AI as an industry, I think, to some degree, is like, you know, people have always sort of been hand-wringing about, you know, how are these businesses going to make money? And I think, you know, every year, people are sort of generally surprised at the upside

• 1:31:36Speaker 01

  of Turns out, superintelligence is useful and people pay money for it.

• 1:31:40Speaker 00

  This is kind of Meta's whole playbook, right? As we build something, we get it to a billion people, then we scale the revenue while growing our revenue primarily in the meantime by continuing to scale the things that are already kind of at scale and growing.

• 1:31:55Speaker 02

  I think this is also one of the dynamics, like, I think, you know, I think Dario's talked about this as well, but like, just in general, because it requires so much advanced planning to build these models, like, if you look at the R and D expense in any given time, that is like an overlapping R and D expense of multiple model generations at once. And, you know, they require a long time to work on, and then ultimately, you know, you see like immense payoff when you ultimately deliver them. And this is kind of what we saw with the Muse Spark family of models. Like, you know, again, we started working on these a long time ago and are seeing like incredible consumer adoption through the Muse product.

• 1:32:35Speaker 00

  The other thing is that we use the models to accelerate our core business, right? So I mean, you have this core business that's hundreds of billions of dollars and still growing extremely quickly. And now, like, if you can grow it a few percent faster by using these models to accelerate these things, then that ends up funding a huge amount of this, too, as you're scaling it out.

• 1:32:53Speaker 02

  The Meta core business is like eighth wonder of the world.

• 1:32:57Speaker 01

  It's an extremely good business.

• 1:32:58Speaker 02

  It really is, it's quite incredible and is growing at sort of, you know, astonishing rates.

• 1:33:05Speaker 00

  Yeah.

• 1:33:07Speaker 01

  You guys really cooked with that one.

• 1:33:11Speaker 01

  I think you're also getting at something, Alex, which is like, there's an interesting accounting artifact where if you go build a data center, you have CapEx and you get to depreciate that over a useful life of many years. And people debate, you know, what that useful life is, but the point is, you do get depreciators, whereas a lot of the AI costs are also just in researcher salaries, and those salaries are being spent on, you know, model training, thus also has a useful life of many years, but you do not get to capitalize generally.

• 1:33:39Speaker 00

  And that's also a financial advantage for us building out the data centers as opposed to a bunch of the other companies that are paying clouds, whereas they're paying the annual rate. But then at some point, the GPUs will be fully depreciated, but we're probably still going to be using them for a while, because they, I mean, the value of like, even like an H100 at this point is higher than it was last year, right? Just because if compute continues being constrained. So, yeah, I think it's, yeah, I think.

• 1:34:07Speaker 01

  Any advice for us with OpenRouter? We have agreed to acquire them, it has not closed yet.

• 1:34:12Speaker 02

  Yeah, I think, you know, part of the exciting, part of what we're starting to see in terms of the models, especially the frontier models that are being developed, like Muse Wonder versus, let's say, a Fable or an Astra, is that we actually are starting to focus quite, in quite different directions from the other models. And I think there's a lot of intentionality that goes in the model development process that we've discussed as part of this, you know, in this conversation, that is genuinely different from how other models are focusing. And so, you know, when these models kind of get collapsed to the same set of benchmarks, that aligns all of the

• 1:34:49Speaker 00

  This is a good point.

• 1:34:50Speaker 02

  The interesting sort of like texture that differs between the various models. And so I think one thing that is an exciting opportunity for OpenRouter, generally speaking, is like, you know, helping to, helping developers understand what these sort of subtle differences are, so that they can make the most informed decisions about what models ultimately they select. And then also, you can imagine the flip side of that, which is, you know, you have all these developers using these models, and so those developers are great people to ask about what are the interesting textual differences between these models that are like more, you know, more than just like tweet length, like, this model's good, this model's bad, like, you know, more than just that level of insight.

• 1:35:31Speaker 02

  And I think this is like a, you know, getting to the wine smellerie, you know, version or form of understanding the models and their sort of like relative strengths and weaknesses, I think would be quite exciting.

• 1:35:42Speaker 01

  Okay, so based on your experience, the OpenRouter view that there will be a diversity of models in the future, and they're optimized for different tasks, and

• 1:35:50Speaker 00

  I agree with that.

• 1:35:50Speaker 01

  Exactly, that's just your guys' experience based on.

• 1:35:53Speaker 02

  Yeah, like, we are focusing on different things. We care a lot more about personal and professional agents, my observation, than the other major labs, which is good, like, there should be differentiation in the economy. This is like the point of capitalism to some extent.

• 1:36:10Speaker 00

  Yeah.

• 1:36:10Speaker 01

  So you're going to say something?

• 1:36:11Speaker 00

  No, I just really agree with that point. I mean, I think that part of what I love about you guys working on OpenRouter and being able to scale it, is it just, the more infrastructure that can get developed, that supports a diversity of players in the ecosystem, I think is good overall. I also, for the same reason, am quite a fan of some of the recent Nvidia moves to support the open source ecosystem. But this just kind of goes with our whole theory and philosophy about like, what does a good future look like? We think that a good future is one where you achieve a, like, you basically get prosperity by empowering people. So that's kind of the personal superintelligence thing that we're doing, but there's the whole business version.

• 1:36:53Speaker 00

  But for a lot of the important safety topics, like, our view is that the way to get there is having a diversity and having checks and balances by kind of having a lot of different things out there and accessible to people, not like just having a couple of good models that are only in a couple of companies' hands. I think that that would be, from my perspective, I think that is probably a riskier configuration for the future than lots of people having access to different things.

• 1:37:21Speaker 01

  Do you guys have a theory as to why that is? Where I feel like maybe people predicted more of a winner takes all dynamic in AI, or one person clearly, like, a story that people told back in 2023 is that one AI lab will start to develop an advantage, and that will allow them to get recursive self-improvement going, and then they'll just pull further and further away from everyone else. Whereas, it has remained extremely competitive, where there is kind of multiple people at the frontier. You guys with Wonder today are, you know, coming along with your frontier class model. Open source models have done an impressive job of keeping up, all told. And so we had this story a few years ago, that someone was just going to run away with the prize.

• 1:38:06Speaker 01

  Why hasn't that happened?

• 1:38:07Speaker 02

  I think there's a lot of underlying reasons and mechanics that drive this. Like, one certainly is just execution. Like, I think, you know, for any given frontier lab, there are periods of strong execution and weak execution, and I think, you know, that is true across all the labs, and so that sort of like, you know, starts to average itself out over time. I think another

• 1:38:29Speaker 00

  Very diplomatic.

• 1:38:31Speaker 01

  So yeah, is it that like some of the players in the lead screwed up?

• 1:38:35Speaker 02

  I think, well, I think if all players, when they were in the lead, executed perfectly, they would probably be able to stay in the lead. So I think that's one factor. I think the power and strength of distillation is certainly another factor that's certainly benefiting the open source ecosystem quite a bit. And I think this is just like a

• 1:38:52Speaker 01

  There's an anti hegemonic force in the phenomenon of distilling a model.

• 1:38:56Speaker 02

  Yeah, and this is just a quality of the technology. Like, I don't think nobody, it's not like anyone planned for distillation to work super duper well, but it just turns out that you can, you know, you can get a lot of capabilities naturally through some form of distillation. And, and this is, you know, broadly speaking, distillation is quite a good force for the entire ecosystem, because it means that you can pack like very strong intelligence capabilities into smaller and smaller models, which enables us to serve them at much wider scale. And that's certainly like one of the very exciting stories is just like really small models today are like, you know, much more powerful than the, than the biggest models a year ago or two years ago.

• 1:39:37Speaker 02

  And then I think the last is that there's just sort of this, I think there's like, I actually think a lot of people in the ecosystem want this future where there, it is not just this like, you know,

• 1:39:51Speaker 00

  Definitely.

• 1:39:52Speaker 02

  one final company in all of capitalism kind of, kind of end outcome. Like, I think this is one of the reasons

• 1:39:59Speaker 01

  Not just individuals, but companies putting dollars against things.

• 1:40:02Speaker 02

  Yeah. Yeah, and I, and I think like, you know, this is actually one of the reasons why a lot of people were excited to join Meta and Meta Superintelligence Labs in the first place was that they were just sort of like, you know, I want to work at a place that is focused on giving this technology to billions of people. That is like not the stated goal necessarily of all the other frontier labs. And so I think there's like a lot of sort of just like natural forces where many people in the industry are sort of like grappling with, okay, what I, what is the actual world state that I want to live in in the future? And there's lots of sort of like decisions that people make at all levels in terms of where they work or who they do business with or what business deals they do that sort of like drive towards a more equalized outcome.

• 1:40:46Speaker 01

  You're getting to the, some of the MSL startup, which I also want to ask about, which is, it just seems fundamentally very hard still to, again, we talked about what goes into developing a frontier model, but also from a standing start. Like, when was the standing start for MSL?

• 1:41:06Speaker 00

  Well, we launched Llama4 on April 4th, I think it was 2025, right? And then, I think, what was it, April 20th or 21st was, I think, when I was like, hey, when I, when I, when I started calling people for advice on what to, on what to do.

• 1:41:30Speaker 02

  So I remember I was, I was in a staff meeting and I got like the WhatsApp notification, like, "Have time for a call?" I was like, huh, interesting.

• 1:41:39Speaker 01

  Okay, so MSL is a little over two years old.

• 1:41:42Speaker 02

  No, no, one year, one year.

• 1:41:44Speaker 01

  You said 25?

• 1:41:45Speaker 02

  Yeah.

• 1:41:45Speaker 01

  Or 26? Wow, sorry, I'm living in the future.

• 1:41:47Speaker 00

  Yeah, you are in the future.

• 1:41:49Speaker 01

  MSL is a little over one year old. And what are your reflections a little over a year in on what was load-bearing, as the eyes would say, in how it came together?

• 1:42:01Speaker 00

  Well, I mean, I think that there was this whole thing early on, which is just like, first, we had to pull together this core group, right? So, so Nat and Alex and I, we were like talking a bit upfront.

• 1:42:16Speaker 00

  And, you know, at first we were brainstorming, like I was talking to Nat about, could we, is there a way that we can like structure this that way it's like a win for Alex's shareholders? And like, we can like get him over here? And then, so then, but we were talking about, like, what would the plan be? And, and then basically I was just, I was in this mode where I was just like asking a lot of different people what we should do. And like every time that, that we would talk, I'd come away with like three more things of like, oh yeah, this is what we should do. It's like very like high signal, very like a lot of strategic clarity. And, so we were brainstorming and working on this for a bit.

• 1:42:58Speaker 00

  And then Nat and Daniel joined and we pulled together a few of the, the initial researchers. But like, in the beginning, it was sort of like the supernova thing where we just kind of like jammed a bunch of, of people together. And I don't know, it was like, like, I don't think either of us slut that much for the first couple of months. I like, don't even, I like, I kind of have this mental block almost where I like, I kind of almost cleared my calendar for like a month or two and was just meeting researchers and talking to them. And I remember even before we announced the deal where you were going to come over, we were basically, like, you were recruiting people and people like, what is going on? Like, that's like, what is this? But, but then there was the whole question, which was

• 1:43:43Speaker 02

  It was a really hands-on data labeling.

• 1:43:47Speaker 00

  But, but that was, that was this whole question too, where it's like, all right, well, like, okay, so if we're not happy with the trajectory of the current program, like who were the right set of people to lead the program going forward? And, you know, Alex and I had spent a bunch of time talking over the years and maybe like, whatever, once every few months or something, we'd talk. And I was always just very impressed by how he thought about things strategically. And, and I kind of had this view of like, all right, yeah, currently he's doing this, this very operational data business. But it always felt to me like he was the sharpest entrepreneur of his generation of the ones who I had had a chance to, to meet.

• 1:44:31Speaker 00

  And, and like, it always just struck me, it's like, all right, like, this is a guy who, yeah, he's like running this like kind of very grindy operational business, but like

• 1:44:41Speaker 02

  Lovely business still, by the way.

• 1:44:42Speaker 00

  What's up? We're big investors. It's not, it's, there was more commentary on the, on the difficulty of running the business.

• 1:44:50Speaker 01

  You run the eighth wonder of the world. Alex, a lot of businesses look like a grindy operational business.

• 1:44:56Speaker 00

  It's just like my, my view. It's like, grindy, grindy.

• 1:45:03Speaker 00

  No, no, no. But, no, so it's, so I just kind of had this view, I'm like, all right, like, like, even though this is sort of, I, I think maybe when we got started, a lot of people were like, okay, like, how is this going to work? It's like the, kind of like the social media guy is working with the like data guy. Like, is the, like, are we going to build a frontier lab here or what? But, but I mean, but my, but I mean, my like really strong conviction, because I talked to like all the leaders of all the labs, like all the heads of research, like all this stuff. And like, I was like, no, Alex is the guy. Like, he's the fucking guy. And like, and we're gonna, and basically, like, we're gonna basically be able to, to,

• 1:45:44Speaker 00

  to go pull together a team and recruit the, the people that we need to go do this. And like, we need this kind of very, this kind of like strategic clarity that he has around the, the different ways that we want to approach this, the kind of extreme focus on talent density, on, on kind of like the clarity of communicating the different vision, which we kind of fashioned together over this period.

• 1:46:11Speaker 00

  And a bunch of things that I think were kind of extreme, but have basically worked, like having this, this like very flat culture was something that you were kind of really obsessed with, right? It's, I mean, you have like, was it almost 200 direct reports? It's like, we, it's not, it's not quite that extreme in practice. I think that there's like, like team leads and all, and all this stuff.

• 1:46:36Speaker 02

  My muse helps me manage them.

• 1:46:37Speaker 00

  Yeah.

• 1:46:39Speaker 00

  But like, but there were these series of things that I think quite early, we, you kind of articulated, and I was like, okay, this is like kind of extreme, but it seems like directionally right, and like, we should go do this. And, you know, I went and like talked to other people or more like the people you'd conventionally think of. And it's like, it was just very much like, I don't know, what you'd expect like a big company to operate. And I was like, no, this is, we got it, we want to do something that's

• 1:47:07Speaker 01

  As you describe this, on those four things that I think even other opinionated, effective founder CEOs would not have done. One, you just called the recent prior efforts by a bunch of people at the company a failure and not good enough. I mean, you were more diplomatic than that, but the act of standing up

• 1:47:29Speaker 02

  Off the trajectory.

• 1:47:30Speaker 01

  Yeah, but the act of standing up by MSL, you know, people can read between the lines.

• 1:47:35Speaker 00

  So sometimes, yeah, sometimes you have to reboot a team.

• 1:47:37Speaker 01

  That's one. Thing two is you were willing to pay ungodly amounts of money for effectively talent. Like, you're paying a lot of money to bring Alex and some other people, but, you know, a lot of money for, for Alex in particular, over. And 49% of this very good grindy business.

• 1:47:57Speaker 00

  Yeah, exactly.

• 1:48:00Speaker 02

  Let's keep the story straight.

• 1:48:04Speaker 01

  So there's that. Thing three is, Alex had never trained a frontier model before. And so you were taking some amount of a bet on Alex's potential to do something where he had like done the stuff in AI. But like, no offense, you just had not done the specific thing that you were being hired to do. And then fourth, you were saying you cleared your calendar and just only did this for two months. I think a lot of CEOs would have had the business review and would have had the board thing. And they would have had the, you know, customer stuff and things like that. Those four seem to me pretty interesting where I think a lot of CEOs would not have done those.

• 1:48:39Speaker 00

  Yeah, that's interesting.

• 1:48:40Speaker 02

  I definitely think, I mean, one thing, you may not appreciate this by yourself, but I think one thing that is quite special about Mark is the degree to which like, once he concludes that something is the right thing or something makes sense, he just like fully shifts everything to be consistent with that belief set. And it is quite like, to your point, I think like the inertia of most, even six very successful founders who like are opinionated and edgy or whatnot, would just be to like, oh, I still have like my whole calendar and like, so I'll just like talk about doing this thing and hope that somebody does it, kind of, kind of situation. Whereas I think, no, I think when Mark concludes that something is the thing he has to do or makes a lot of sense,

• 1:49:25Speaker 02

  then he really does like, he has like maybe a very high learning rate in that sense. Like, he's very willing to like pivot entire things to be consistent with it. And, and I think that like, you know, one very underappreciated thing is the degree to which so much of the company had to pivot to support MSL as well. Like, I think that while it was a kind of like this more startup effort within a larger company, you know, the, the needs of MSL were still quite large. Like, we needed a huge amount of help on infrastructure. We needed these large scale data centers to be built. We needed like lots of support from many other parts of the company.

• 1:50:06Speaker 02

  And especially as Muse has been successful, we've also needed lots of collaborate, deep collaboration with the rest of the company. And this is one thing that I think like, kind of only is possible because Mark is the founder, CEO, like, and he can ensure that we have kind of like a global optimization towards this, this sort of like AI maximus version of Meta versus like the natural inertia of an organization towards like, you know, just local optimization ad infinitum.

• 1:50:38Speaker 00

  But it's also a lot of how we work together is it's important to me to be in the details of this work so that we can get everything around the company to go do what we need to do, right? And so it's like, I mean, yeah, I'll have opinions about this stuff, but I'm not like, I'm not the lead researcher, right? It's like, it's, you know, it's, but I, I think that's kind of a lot of, I think, how we work together well is that, yeah, it's, I mean, we're, we're kind of like quite in the details of all of this stuff, but, you know, running the lab is a big thing. Running the rest of the company in order to support this is like, the effort is a big thing. You almost like, you can't really get this done, I think, without both of those pieces.

• 1:51:19Speaker 02

  And, and one thing that I think was quite important about when we set the holding up is, you know, we, Meta, the company, invested a huge amount to stand up MSL. And then we, we very clearly told everyone like, hey, there's no deadlines. We are going to take things at the right pace, which I think on the one hand could be like, you know, absolutely terrifying. It's like you've just have like set up this extremely expensive unit and you're telling everyone like, hey.

• 1:51:45Speaker 01

  And you're worried inside a larger company that like, okay, seven years from now, maybe something will ship, right?

• 1:51:50Speaker 02

  Yeah, totally. And, and so, but I think that like, first, I think one of the cool things about, you know, focusing on talent density and hiring all these very brilliant people have done so much already. That's very impressive is, you know, everybody had like a lot of motivation to ship and, and everybody wanted to sort of like leave their mark, so to speak, on the field. And so everybody was like heavily, highly self motivated to, to move as quickly as possible. And one of the things that's, that's been funny and I've had to continue reminding the team of is like, you know, it is a team of, of a lot of insecure overachievers. So like, at any one point, if I ask them like, hey, how good do you think, where, where do you think we'll be in three months? I always get the most like.

• 1:52:35Speaker 00

  Everyone always.

• 1:52:35Speaker 01

  It's awful.

• 1:52:37Speaker 02

  These like very pessimistic answers. And then I record it and I like, three months later, I'm like, hey, you know, we just shipped this model. Let me just show you what all of you said, where all of you said we would be in three, three months ago. And, and this is.

• 1:52:52Speaker 01

  And has that caused them to get any more optimistic or no?

• 1:52:54Speaker 02

  No, they're still.

• 1:52:55Speaker 00

  They're very self critical. I think it's, it's not, it's not that they're pessimistic about, I think that the, the average person seems quite optimistic about MSL and very self critical about their own project. Which I think is actually quite a healthy, a healthy dynamic.

• 1:53:12Speaker 02

  But, but the trend continues. Like, I think, I think, and, and this is, I mean, in many ways Muse Wander is like this incredible milestone for everyone because it is a.

• 1:53:22Speaker 00

  The team was very, very critical.

• 1:53:23Speaker 02

  Yeah.

• 1:53:24Speaker 00

  That was working on it along the way.

• 1:53:26Speaker 01

  When did you guys know that Muse Wander was good?

• 1:53:32Speaker 02

  That's a good question. I think that we, so there's all these intermediate.

• 1:53:37Speaker 01

  You're kicking in the oven and it's like, I don't know, it seems kind of gooey still and yeah.

• 1:53:41Speaker 02

  Yeah, going back to it, I mean, there are all these intermediate signals that you can get along the way that things are good or bad.

• 1:53:48Speaker 01

  And what are the intermediate signals?

• 1:53:50Speaker 02

  So in, in pre-training, you have a lot of perplexity based evaluation. So you have a, it's sort of a different regime of evals that can help you in pre-training. And then you have sort of the more traditional benchmarks that can help you towards RL. But, but no, I mean, it's, I mean, this model, kind of as you mentioned, has been cooking for quite a while. And, you know, a lot of, I think one of the funniest things about AI research is that, you know, there's a lot of, there's a lot of times where people just sort of like are kind of squinting at a bunch of dots and being like, yeah, it looks like it's good or it looks a little bit off.

• 1:54:28Speaker 02

  Like, one of the funniest things is the, the, people have protractors, like, like analog protractors that they put against, you know, a set of lines on their monitor to like measure an angle and then they.

• 1:54:44Speaker 01

  Yeah, I have not come for this job yet.

• 1:54:46Speaker 02

  So I think there's, I think there's like a huge amount of like, like it's very, there's more vibes driving like.

• 1:54:53Speaker 00

  Well, I remember we had this conversation where.

• 1:54:55Speaker 01

  When did you know it was good?

• 1:54:57Speaker 00

  Well, we had this debate after we've done pre-training where it was, okay, how long will it take in RL before it is better than Muse Spark? And I remember the, the folks on the pre-training team were like, just trying to set expectations. They're like, look, like sometimes like this takes a while. And then it was like immediately. And so we're like, okay, I think, I think we're like on, on a good path, right? That to me was, was sort of, I think when, when people got, yeah.

• 1:55:30Speaker 02

  We've been, I think we've been pretty excited about it for a few months now. Yeah.

• 1:55:34Speaker 00

  Yeah, but there's a lot to do there. But going back to the beginning, I think one of the things that was very interesting and, and challenging in standing the thing up was like, you really just wanted this like very high talent density effort. And we, we were just kind of pulling some of the best people from Meta in and recruiting a lot of the best people. And I mean, you, you were asking, it's like, all right, well, like, okay, for people who like, if you're asking before, it's like, well, Alex hadn't necessarily trained models like this, but to me, one of the big things was like, all of the best people were very excited to work with him. And I think like, it's, I mean, you, I'm sure see this in all of, in a lot of the things that you do, which is like,

• 1:56:18Speaker 00

  you know, it's like you, most people are not kind of one dimensional, right? And yeah, you like, you, maybe you do a specific thing, but, but like, your ability to grow and expand is kind of largely based on how effectively can you like just do people who are excellent in different domains want to, want to come work with you on different things.

• 1:56:41Speaker 01

  Yeah, and the great people have followership.

• 1:56:43Speaker 00

  Yeah, but, and so I think that there was, there was a little bit of that. But also just it's even for the people that don't know you yet or like, or aren't, you know, don't know what you're about, like, are you like an impressive person and a compelling person that people want to come work with? Do people feel comfortable that you're going to create this environment? And I think you were like very thoughtful about this early on. We were kind of, some of the stuff I think you kind of had fully formed as beliefs of how you thought we should do this. But I also remember when we were having the initial recruiting conversations,

• 1:57:23Speaker 00

  we were sort of in real time shaping, it's like, okay, well, this construct that we're talking about with this person isn't quite gonna work, but like they seem good and like they have this thought. And it was just like, like really kind of coalescing this vision about what the shape of it should be in real time. I mean, I don't know, I mean, I'm sure you remember this, where it was like, there were, there were some of the, the initial concepts of like.

• 1:57:47Speaker 00

  I mean, I think probably the most extreme thing was I think you had the concept of the talent density and that kind of piece, but I think we weren't sure upfront that it was going to have to be a full reboot, right? It's like that that basically started when we were out talking to a lot of the kind of best people and we realized that, hey, like in order to make this as attractive as possible, it should feel like, okay, we're starting this thing again from scratch. And yeah, we'll pull in a lot of the great people from from the Lama effort and fair and all of that, too. But I think that was a very intentional thing that we basically figured out and that you figured out as we were out talking to the initial set of people of like, what was going to make this compelling? And it ended

• 1:58:32Speaker 00

  up being really smart and like making a huge difference in the culture of the lab and how it got and how it got started. But I don't know, it's like being able to be that dynamic about figuring out like, what is the shape of this need to be? It's not like that. I think that's a very uncommon trait. It's a very entrepreneurial trait, right? So when I was thinking about like, all right, like, this field is going to keep on evolving, right? So what is the thing that's going to be important for someone who's going to run a lab? Well, it's not like, are they an expert in whatever the latest modality is of like, or kind of of training technique of for a lab? It's, is the person a clear

• 1:59:12Speaker 00

  thinker and able to kind of dynamically update and move in like as kind of an entrepreneur would? I think this is just like an inherently very entrepreneurial space.

• 1:59:22Speaker 01

  Has this experience changed your view on the value of going external, where both your reboot, a bunch of the people you brought in were external, like Alex and Nat and Daniel and folks, but then also you're describing deciding whether you needed a reboot based on going to the outside intelligence of, you know, the aggregate views of the researchers in the industry and getting a sense for the just, do you feel like you will carry forward that approach for future unrelated projects? Because obviously the risk of a large company is that everyone's like, oh, we got a great approach. Yeah,

• 2:00:06Speaker 01

  let me show you our 100 page document on it.

• 2:00:08Speaker 00

  Not, I mean, obviously, you can't, a lot of what we had to do here was rotate a lot of the rest of the company and like for things like the infrastructure team that now, you know, it's like building these huge clusters to support this. I mean, that's obviously a huge thing. So huge parts of the company kind of had to pivot to do this. And obviously, you can't do too many of those at the same time, right? Only one thing can be the top priority.

• 2:00:34Speaker 01

  Yes.

• 2:00:36Speaker 00

  But I think on balance, it has made me somewhat more optimistic. And we brought in this guy, Kunal Shah, who, you know, it's, I think he's like one of the top entrepreneurs in India, and he was building this great company, Kred, and now he's running WhatsApp, which I always thought should do more in commerce. And we were sort of underserved there. And India is the biggest country that we're in. And but, I mean, that's, it's a bit of a different situation, because we're not like pivoting the whole company towards WhatsApp. We just wanted to bring more entrepreneurial energy there. This was like a level 10 out of 10 difficulty thing. I don't want to talk about in the past tense as if it's done, because it's still a level 10 difficulty thing. But, but I think the

• 2:01:19Speaker 00

  number of pieces that kind of had to work, it was, yeah, it was, it was quite all encompassing. And I think that that it's both difficult for the people doing it. But I do think that that also puts some level of strain on the rest of the company, on the financial model of the company. I think like you, so I wouldn't want to conflate those two pieces, right? It's like, I think there's, yeah, it's good to have new blood and it's good to have people who understand the context. And, you know, you were talking about when people have been around Meta for a while and understand the ups and downs. And it's like, I don't know, if you've worked at Meta for 10 years, you've seen some shit, right? It's like, it's like we've gone through a lot of hard times. And, and like, I think that's kind of important

• 2:02:02Speaker 00

  context, too. But, but no, I think a lot of this is just trying to, you're, whenever you're building a team, you're, you inherently, you know, it's like Alex had a theory for what the team shape needed to be. And like, I guess I had a theory for what like the shape of the leaders needed to be for it. And, and, you know, it's like a year plus in, it like seems, okay, like, it's like, first of all, it was just good to try something different, right? And then different than what everyone else is doing.

• 2:02:35Speaker 01

  Yes.

• 2:02:36Speaker 00

  And, and, yeah, I mean, that's.

• 2:02:41Speaker 02

  And I think there's some other dimensions here that are that are worth mentioning, too. Like, one of the, like, when deciding we had to do the full reboot, like, you know, a lot of it was, you know, one of the main inputs was external, but also one of the inputs, I don't think I've ever, actually, I can't remember I told you this story, but one of the first meetings I had when I joined, I was in this, it was in this, you know, meeting with some of the leads of the existing effort. And I was like, okay, I think the first thing we need to do is train a really good reasoning model by, you know, by some timeframe, I don't know when, but this is like clearly like the most important milestone. And one

• 2:03:21Speaker 02

  of the the key leads in the group, who no longer works at Meta, but was like, oh, we don't, we don't need a reasoning model. Does any, are any of our, do our consumers need a reasoning model? Like, I feel like we don't need to do a reasoning model. We could just do a normal model. And I was, I had like this heart sinking moment.

• 2:03:40Speaker 00

  Oh, no, I've joined the wrong company.

• 2:03:41Speaker 02

  I was like, oh, we, I've, I gotta take like 10 steps back. And I think, I think this is, you know, this was a kind of a funny anecdote. I remember calling Nat after this meeting. I was like, Nat, like, this is what they just said in the meeting. But I think it was, it was, it was specifically an indicator of something that I think, like, Mark, when he talked to people externally, also started to realize, which is like, we needed leadership in the company to be much more AI pilled and much more sort of like forward leaning and front leaning on like where the, how quickly the technology is moving, where it's going to go to, what the possibilities are, and like, you know, effectively, like dramatically

• 2:04:21Speaker 02

  more AI pilled. And I think as Mark started pivoting and shifting the whole company towards this direction, and it was very clear that this is like, you know, I think Mark got AI pilled, and then, you know, everyone saw that he was AI pilled and realized like, oh, this is like where the company is going. Then I think you had like, a lot of the existing leaders, like, very quickly pick it up and change. And like, Adam Mosseri, for example, who runs Instagram, has one of the most like, aesthetic, but also like effective, kind of like, personal vibe code and personal mini app setup of anyone that I've seen. Like, I think it's like, he's like, really gone quite far in the direction. And I think a

• 2:05:06Speaker 02

  lot of the leaders at Meta, who have been at Meta a long time, are like, all very adaptive. Like, I think, you know, kind of to the prior point, they've been in

• 2:05:15Speaker 00

  Yeah.

• 2:05:15Speaker 02

  a highly turbulent, like, period of technology. Now, AI is like, maybe, you know, five or 10 times more turbulent and faster moving than those pathways. But I think they're like, they have adapted to new trends many times. And so I think there's like, I've been very impressed, actually, by seeing how quickly a lot of Meta leadership. You know, if I compare the kind of conversations I would have with them when I first joined versus the kind of conversation I have now.

• 2:05:42Speaker 01

  They're now super AI pilled.

• 2:05:43Speaker 02

  Yeah. And it's, it's very impressive, honestly, for that to happen at like, the large scale of, you know, Meta, which is one of the largest companies in the world.

• 2:05:52Speaker 00

  Yeah, we got them there pretty quickly. It was, yeah. But, but I mean, like you're saying, it's like, if, you know, you start, you start coding, then it's helpful. But I think a big part of what, a big part of the culture that we were trying to create is also, you know, when we were talking about, you have all these direct reports. The point of it isn't that you have the direct reports, it's that there aren't, it's the absence of non-technical managers, right? So, so I don't think you would say that you're like, you know, that you're providing a ton of managerial support to 200 people at once. I mean, that's not a feature of the thing.

• 2:06:32Speaker 01

  Alex, well known for all his career chats and emotional, how are you doing, checking?

• 2:06:37Speaker 00

  I think it's just well known among researchers that, you know, there's this common pattern. It's like you go from being very technical to you go to a role where you're, you're not like in the details anymore. And then you very quickly become not technical or don't kind of understand it much faster than you probably realize. And it's quite dangerous.

• 2:06:56Speaker 01

  It's such a fast moving domain, especially.

• 2:06:58Speaker 00

  Yeah. So I think that like, this whole thing of just, like, you have to stay technical, people have to use the agents, people should be writing code.

• 2:07:06Speaker 01

  Yeah.

• 2:07:07Speaker 00

  I think that that's, that has been a really important part of this whole thing.

• 2:07:11Speaker 01

  Have you had any very funny disagreements, I should mention to you guys?

• 2:07:15Speaker 00

  We have a lot of disagreements.

• 2:07:16Speaker 02

  Yeah, I mean, we're working on naming up until very recently.

• 2:07:20Speaker 01

  Name is a good bike.

• 2:07:21Speaker 00

  It's one of the things I appreciate about Alex. It's like one of his, I mean, what this is like, I think it goes back to your, your philosophy. You're talking about this for recruiting back at scale is like, hire people who really care, right? I think there was some expletives in it. But, but, but I.

• 2:07:38Speaker 01

  But Alex really cares, it turns out. I have discovered.

• 2:07:40Speaker 00

  No, well, that's like one of the things that I thought, you know, I think that that's like really important, right? It's, so I think part of what you, you kind of get when you like have the supernova of like, all these different talented, but very strong willed people together is like, you're gonna have like fun fights. And not fun fights, but like, but it's like, to some degree, I'd rather have people push for things that they believe in and like, argue over things than, you know, just have people think that they're like blindly executing something. You know, it's like people need to feel ownership. So, yeah.

• 2:08:12Speaker 01

  How does Mark a different boss than you expected was?

• 2:08:15Speaker 02

  Well, I don't have a huge comparison set.

• 2:08:17Speaker 00

  I can leave now.

• 2:08:21Speaker 02

  No, I think that like the, I think it has been very, one of the very cool things about like, working with Mark over the past year, has just been watching, like, the true degree of his boldness and like, and this trait that we were talking about before of like, you know, once he internalizes certain beliefs, like, the speed and magnitude of like, willing to like, change, you know, prior actions and pivot towards the like worldview that is consistent with those beliefs. Like, I think these are. Pretty unusual traits, generally speaking. Like even among the space of founders.

• 2:09:05Speaker 01

  Mark actually ingests new information and fully updates his worldview and then acts in accordance with the newly installed worldview.

• 2:09:11Speaker 02

  Very Bayesian and then willing to take.

• 2:09:13Speaker 00

  Very Bayesian. I've never gotten such a scuffle before.

• 2:09:19Speaker 01

  What lovely priors you have.

• 2:09:22Speaker 02

  But the posteriors. It's the posteriors.

• 2:09:24Speaker 01

  You're not allowed to talk about posteriors anymore after your whole incident.

• 2:09:28Speaker 02

  No, but then I think also just the degree of boldness. And I think that like the, this is, it was like actually quite puzzling to me at first, the degree to which you were like willing to make these massive bets. It's something actually Nat and I would often like talk about. I was like, wow, we're really going big here. And both of us would be like, would you go this big? Would you go this big? And so I think that this, I think it's like, I mean, that is, that is like quite a notable trait. And something I really admire. And I think, I think is, I think is very, I actually like, if I look back at the history of Meta and I imagine what the like scenario would have been in all these like pivotal moments,

• 2:10:12Speaker 02

  I can see how like this is kind of the movie, so to speak. Or like this is kind of how a lot of those situations go where it's like, there's, I feel like probably, the way I imagine is like there are clearly these moments where like something very, very big is happening. And it may not even really be showing in the metrics or it may not be impacting user counts or impacting revenue or whatever it might be. Like it's, it's not really impacting things. But clearly, if you believe that thing is going to happen, then it has like massive implications for the company and the product. And, and I think, you know, my, you know, post hoc retelling of Meta's history is that like, I think Meta is a company where every time that happens,

• 2:10:56Speaker 02

  Meta takes it extremely, extremely seriously. And that has enabled the company to innovate and continue evolving with a very fast moving and fast changing tech landscape.

• 2:11:07Speaker 01

  Mark is very Bayesian.

• 2:11:08Speaker 02

  Yeah, very Bayesian.

• 2:11:09Speaker 00

  So Bayesian.

• 2:11:11Speaker 00

  Life is short, just do the right thing.

• 2:11:14Speaker 02

  And I think, I think AI is, I mean, I think AI is maybe that like, you know, times 100 or whatnot. But like, I think the, you know, a lot of CEOs would have told you last, you know, last summer, oh yeah, I think AI is the most important thing for my business. Like I think maybe every CEO would say that. But then you look at like what they're actually doing and how consistent are their actions with that belief? And I think this is like a, like quite, quite special thing about Meta. And Mark.

• 2:11:43Speaker 01

  Last question, because we're past all of our bedtime.

• 2:11:45Speaker 00

  I'm so flattered.

• 2:11:47Speaker 01

  What do you make of all of Alex's deranged posting on Twitter?

• 2:11:51Speaker 00

  I mean, I think most of it is great. But, but I guess my main view.

• 2:11:57Speaker 02

  But it doesn't matter how, how much you fail. It matters how win, how big you win when you win.

• 2:12:02Speaker 00

  I think with, so I think with, with communication, I think it's most important to just be authentic. And I think when you're trying to be authentic, some of the things don't come out the way you want. And I think that that is a, that is better to make the trade in that direction than it is to be overly scripted and not be authentic. But I mean, most of it, I just feel it's hilarious. So.

• 2:12:28Speaker 02

  Yeah. What do you think about my deranged tweets?

• 2:12:30Speaker 00

  I'm not a, I'm not a heavy X user, but.

• 2:12:34Speaker 01

  But you, you are back on X now?

• 2:12:36Speaker 00

  I'm back to posting. I mean, it's like a bunch of the researchers are there and we want to, you were there. But everything I post there, I also post on threads and it's all good. But every time I load X, I have like four Alex posts and it just like cracks me up. It's like some of these, like, it's like, where do you, we were just talking about this.

• 2:12:54Speaker 01

  Well, you're now a clearinghouse, I presume, where people send you good stuff.

• 2:12:57Speaker 00

  I was like, where do you come up with this stuff? It's like, yeah, this is, yeah.

• 2:13:00Speaker 01

  How does it work?

• 2:13:02Speaker 02

  Most of the, like, we were, we were talking about this with my partner yesterday and he was, you know, I think one observation was just, there was, actually probably still to this moment, my, my thought pattern oftentimes is, oh, that's funny. I wonder what Jolly would think about that. And then, and then, you know, you get good memes out of that. That being said, I have experienced the, the sort of like brutal metabolism of the internet and how quickly it like metabolizes memes. And so the sort of like, you know, you really have to stay, you have to stay on the, on the cutting edge of memeing.

• 2:13:40Speaker 01

  It's easy to fall behind.

• 2:13:40Speaker 02

  Yeah.

• 2:13:41Speaker 01

  Which is maybe a good summary of this conversation. It's easy to fall behind. You gotta stay parent, gotta stay at the frontier.

• 2:13:48Speaker 00

  Stay in the details.

• 2:13:49Speaker 01

  Yeah.

• 2:13:49Speaker 00

  Stay technical.

• 2:13:50Speaker 01

  And stay at the frontier.

• 2:13:51Speaker 00

  Yeah.

• 2:13:52Speaker 01

  Alrighty, we'll leave it there.

• 2:13:53Speaker 02

  Alrighty.

• 2:13:54Speaker 01

  Thanks, guys.

• 2:13:54Speaker 02

  Thank you.

• 2:13:55Speaker 01

  Okay, so what is everyone drinking?

• 2:13:58Speaker 00

  Well, that.

• 2:14:00Speaker 01

  What is this?

• 2:14:02Speaker 00

  That's a keg of Guinness that I brought you.

• 2:14:04Speaker 01

  This is Guinness from?

• 2:14:05Speaker 00

  Dublin.

• 2:14:06Speaker 01

  Oh, wow. You were just there?

• 2:14:07Speaker 00

  Yeah, I was just there this week.

• 2:14:08Speaker 01

  Oh, awesome. Well, thank you. We got the local stuff.

• 2:14:11Speaker 00

  Yeah, there you go. Yeah, no, I figured this would be a good occasion to get the Guinness going. Yeah.