#398 – Mark Zuckerberg: First Interview in the Metaverse

Mark Zuckerberg is CEO of Meta. Please support this podcast by checking out our sponsors: – LMNT: https://drinkLMNT.com/lex to get free sample pack – InsideTracker: https://insidetracker.com/lex to get 20% off – Eight Sleep: https://www.eightsleep.com/lex to get special savings – AG1: https://drinkag1.com/lex to get 1 month supply of fish oil – NetSuite: http://netsuite.com/lex to get free product tour Transcript: https://lexfridman.com/mark-zuckerberg-3-transcript EPISODE LINKS: Mark’s Facebook: https://facebook.com/zuck Mark’s Instagram: https://instagram.com/zuck Mark’s Threads: https://threads.net/@zuck Meta AI: https://ai.meta.com/ Meta Quest: https://www.meta.com/quest/ Meta Connect 2023: https://www.metaconnect.com PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ YouTube Full Episodes: https://youtube.com/lexfridman YouTube Clips: https://youtube.com/lexclips SUPPORT & CONNECT: – Check out the sponsors above, it’s the best way to support this podcast – Support on Patreon: https://www.patreon.com/lexfridman – Twitter: https://twitter.com/lexfridman – Instagram: https://www.instagram.com/lexfridman – LinkedIn: https://www.linkedin.com/in/lexfridman – Facebook: https://www.facebook.com/lexfridman – Medium: https://medium.com/@lexfridman OUTLINE: Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time. (00:00) – Introduction (08:38) – Metaverse (23:01) – Quest 3 (37:50) – Nature of reality (42:28) – AI in the Metaverse (59:26) – Large language models (1:05:23) – Future of humanity

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Lex Fridman?

  The following is a conversation with Mark Zuckerberg, inside the metaverse. Mark and I are hundreds of miles apart from each other in physical space, but it feels like we're in the same room, because we appear to each other as photorealistic codec avatars in 3D with spatial audio. This technology is incredible, and I think it's the future of how human beings connect to each other in a deeply meaningful way on the internet. These avatars can capture many of the nuances of facial expressions that we use, we humans use, to communicate emotion to each other. Now, I just need to work on upgrading

• 0:40Lex Fridman?

  my emotion-expressing capabilities of the underlying human. And now, a quick two-second mention of each sponsor. Check them out in the description. It's the best way to support this podcast. We've got Element for delicious electrolytes, Insight Tracker for biological data, AteSleep for delicious naps, AG1 for delicious health, and NetSuite for probably delicious, but I haven't tasted it, business management software. ChooseWise and my friends. Also, if you want to work with our team, we're always hiring. Go to lexfridman.com/hiring. And now, onto the full ad reads. As always, no ads in the middle. I try to make this interesting, but if you must skip them.

• 1:24Lex Fridman?

  Friends, please do check out the sponsors. I enjoy their stuff. Maybe you will too. You never know until you try. This episode is brought to you by the thing I consume many times a day. The thing I'm consuming currently as I am speaking the words I'm speaking. Element. Electrolyte drink, spelled L-M-N-T. I do a low-carb diet, sometimes a very low-carb diet, and I usually eat only once a day. So that's fasting and low carbs. And for that, you have to get your electrolytes right. Electrolytes include sodium, potassium, magnesium. The proportions are really important. The deliciousness of the proportions when combined with water is also important, it turns out.

• 2:08Lex Fridman?

  It's also just important to consume water and making water a little bit more delicious, which is what Element does. Is a win-win. That's what we call in the business a win-win, friends. Get a simple pack for free with any purchase. Try it. Join me and try it at drinkelement.com/lex. Get a simple pack for free with any purchase at drinkelement.com/lex. This show is also brought to you by InsideTracker, a service I use to listen to my body through the power of data science. So from my body comes signals through blood tests that give information about whatever's going on in my body.

• 2:51Lex Fridman?

  And that data is used with machine learning algorithms to make suggestions about what you should do in your life. Now, in this conversation about the metaverse, where Mark and I are incredibly in a whole other plane of existence. We have teleported into this place. While we're miles apart from each other in physical space, in the virtual space, we're right there together, shrouded in darkness. I wonder if those entities that are driven by the content of our minds, if they have biological-like signals, if they need to make lifestyle and diet decisions. I wonder, one day, we should be able to ask them.

• 3:34Lex Fridman?

  Actually, that's one of the things that Meta announced, which is really cool, which are the AI personalities. And I do think, just from a technical perspective, it's super exciting and difficult to encapsulate a particular personality like Snoop Dogg in an AI agent, where it reflects all the quirks, the weirdnesses, but also the beauty of the character of that particular human being. I love conversation, that task, doing that thing technically, super exciting, but I think it's super difficult, so I'm glad they're taking it on. Anyway, get special savings for a limited time when you go to insidetracker.com/lex. This episode is also brought to you by Eight Sleep and its Pod 3 mattress.

• 4:19Lex Fridman?

  It can cool down or heat up the two sides of the bed separately. This actually makes me think of that meme with the two kids that are scared in the corner and they're looking at a bunny. And.

• 4:35Lex Fridman?

  I am those two scared kids in the corner thinking about the bunny, where the bunny represents the people that want their bed heated up. I don't know who does this, but I want to meet you. Maybe you have an Asleep like in the Antarctic or somewhere in northern Canada. And but you also don't have heat and maybe you're out camping. I can't possibly construct the situation which you would use heat. I'm only half kidding, of course. Anyway, I prefer to cool down the bed. And it's an incredible experience, a cold bed with a warm blanket. It's it's a it's another place I teleport to in my mind when I go to the nap. It's not a metaverse, it's a napaverse.

• 5:16Lex Fridman?

  And there I find a deep peace from the chaos of life. And I return refreshed, ready to take the chaos on once again. Anyway, check it out and get special savings when you go to asleep.com/lex. This show is also brought to you by AG1, another delicious thing I consume every day. I just drank it like an hour ago. It's delicious, it's green and makes me feel like Popeye. I grew up, I don't know how many years ago, watching Popeye. There was a stretch of time where I wanted to be Popeye.

• 5:55Lex Fridman?

  I think I admired Popeye for being strong, maybe because I wanted to be strong. I always thought a man is supposed to be strong. And so if I just eat my spinach like Popeye did, I'll be strong. So hence the color green is forever associated with with strength. And I'm also playing Diablo now. And that's one of the things you can increase is strength, dexterity, willpower, intelligence, all that kind of stuff. But strength still has that sexy sound to it. And so every time I play Diablo, actually, and I drink AG1. And it just makes me feel stronger and better and like I got my nutritional bases covered.

• 6:37Lex Fridman?

  Anyway, they'll give you one month's supply of fish oil when you sign up at drinkag1.com/lex. This show is also brought to you by NetSuite, an all in one cloud business management system. I still hold in me a little bit of a flame that turns hopefully into a fire, which is a desire to contribute by building stuff, whether it's working at a company or starting a company. And I'm very cognizant of how difficult it is to run a company successfully, whether you're looking at a huge company like Meta or a tiny startup, all of it. And actually, yes, all of it is a source of challenge and complexities and fear and anxiety and uncertainty, all of that.

• 7:22Lex Fridman?

  But ultimately, the camaraderie of the people working together, that's a deeply fulfilling, deeply meaningful pursuit, especially when there's a big vision that you're reaching for. So I love people at companies, working at a thing, chasing that big impossible vision. Actually, the metaverse is one such thing. Just the technical complexity of everything involved is just really incredible. From the headset, software and the hardware, the cameras, the scanning for the avatar, all of it is just incredible. Anyway, if you want to figure out how to run a business, small or large, successfully, you should be using the best tools like NetSuite. You can start now with no payment or interest for six months.

• 8:05Lex Fridman?

  Go to netsuite.com/lex to access their one of a kind financing program. That's netsuite.com/lex. This is the Lex Fridman Podcast. And now, dear friends, here's Mark Zuckerberg.

• 8:42Lex Fridman?

  This is so great.

• 8:46Lex Fridman?

  Lighting change? Wow.

• 8:49Mark Zuckerberg?

  Yeah, we can put the light anywhere.

• 8:53Lex Fridman?

  And it doesn't feel awkward to be really close to you.

• 8:56Mark Zuckerberg?

  No, it does. I actually moved you, I moved you back a few feet before you got into the headset. You were like right here.

• 9:02Lex Fridman?

  I don't know if people can see this, but this is incredible. The realism here is just incredible.

• 9:10Lex Fridman?

  Where am I? Where are you, Mark? Where are we?

• 9:13Mark Zuckerberg?

  You're in Austin, right?

• 9:15Lex Fridman?

  No, I'm in this place. We're shrouded by darkness with ultra-realistic face and it just feels like we're in the same room. This is really the most incredible thing I've ever seen. And sorry to be in your personal space.

• 9:30Mark Zuckerberg?

  I mean.

• 9:31Lex Fridman?

  We have done jujitsu before.

• 9:32Mark Zuckerberg?

  Yeah, no, I was commenting to the team before that. Even that I feel like we've choked each other from further distances than it feels like we are right now.

• 9:42Lex Fridman?

  I mean, this is just really incredible. I don't know how to describe it with words. It really feels like

• 9:49Lex Fridman?

  it feels like we're in the same room.

• 9:51Mark Zuckerberg?

  Yeah.

• 9:51Lex Fridman?

  It feels like the future.

• 9:54Lex Fridman?

  This is truly, truly incredible. I just wanted to take it in. I'm still getting used to it. It's like, it's you. It's really you. But you're not here with me, right? You're there wearing a headset and I'm wearing a headset. It's really, really incredible. So what, um, can you describe what it takes currently for us to appear so photorealistic to each other?

• 10:18Mark Zuckerberg?

  Yeah, so I mean, for background, we both did these scans for this research project that we have at Meta called Kodak Avatars. And the idea is that instead of actually, instead of our avatars being cartoony, and instead of actually transmitting a video, what it does is we've sort of scanned ourselves in a lot of different expressions. And we've built a computer model of sort of each of our faces and bodies and the different expressions that we make and collapse that into a Kodak that then, when you have the headset on your head, it can, it sees your face, it sees your expression,

• 11:02Mark Zuckerberg?

  and it can basically send an encoded version of what you're supposed to look like over the wire. So in addition to being photorealistic, it's also actually much more bandwidth efficient than transmitting a full video or especially a 3D immersive video of a whole scene like this.

• 11:21Lex Fridman?

  And it captures everything, like the flaws. Like to me, the subtleties of the human face, like even the flaws, that's like, that's all amazing. It makes you, uh. It makes it so much more immersive. It makes you realize that like perfection isn't the thing that leads to immersion. It's like the little subtle flaws, like freckles and like variations in color and just

• 11:45Mark Zuckerberg?

  Yeah, wrinkles.

• 11:46Lex Fridman?

  All stuff about noses. Yeah, asymmetry and just the different like the corners of the eyes, like what your eyes do when you smile, all that kind of stuff.

• 11:54Mark Zuckerberg?

  Yeah, eyes are a huge part of it. Yeah, I mean, there's all the studies that most of communication, even when people are speaking, is not actually the words that they're saying, right? It's it's kind of the expression and all that. So and we try to capture that with the kind of classical

• 12:11Mark Zuckerberg?

  expressive avatar system that we have. That's the kind of more cartoon designed one. You can you can kind of put those kind of expressions on those faces as well. But there's obviously a certain realism that comes with delivering kind of this photorealistic experience that I I don't know, I just think it's really magical. And this gets to kind of the core of what the vision around virtual and augmented reality is of like delivering a sense of presence as if you're there together, no matter where you actually are in the world. And I mean, this this experience, I think, is a good embodiment of that, where it's like, I mean, we're in two completely different states halfway across the country. And it just like, you know, it looks like you're just sitting right in front of me. It's it's pretty wild.

• 12:51Lex Fridman?

  Yeah, I can't. It's I'm almost getting emotional. It's like it feels like a totally

• 12:57Lex Fridman?

  fundamentally new experience. Like for me to have this kind of conversations with loved ones. It would just change everything. Maybe just to elaborate. So the I went to Pittsburgh and went through the whole scanning procedure, which has so much incredible technology, so software and hardware going on. But it is a lengthy process. So what's your vision for the future of this? In terms of making this more accessible to people?

• 13:24Mark Zuckerberg?

  You know, it starts off with a small number of people doing these very detailed scans, right, which is that's the version that you did and that I did. And, you know, before there are a lot of people who who've done this kind of a scan for, we probably need to kind of over-collect expressions when we're doing the scanning, because we haven't figured out how much we can reduce that down to a really streamlined process and extrapolate from the the scans that have already been done. But, you know, the goal, and we have a project that's working on this already, is just to do a very quick scan with your cellphone, where you just take your phone, kind of wave it in front of your face for

• 14:06Mark Zuckerberg?

  a couple of minutes, you know, say a few sentences, make a bunch of expressions, but overall, have the whole process just be two to three minutes, and then produce something that's of the quality of what we have right now. So I think that that's one of the big challenges that remains. And right now, we have the ability to do the scans if you, you know, have hours to sit for one. And with today's technology, I mean, you're using a meta headset that exists, it's a product that's kind of for sale now, you can drive these with that. But the production of of these scans in a very efficient way is one of the last pieces that we still need to really nail. And then, obviously, there's all the experiences around it. I mean, right now, we're kind of sitting in a dark room,

• 14:51Mark Zuckerberg?

  which, you know, is familiar for for your podcast. But I think part of the the vision for this over time is is, you know, not just having this be like a video call. I mean, that's fine. It's cool. It feels like it's immersive. But, you know, you can do a video call on your phone. The thing that you can do in the metaverse that is different from what you can do on a phone is like, Doing stuff where you're physically there together and participating in things together. And we could play games like this. We could have meetings like this in the future. Once you mix, once you get mixed reality and augmented reality, we could have codec avatars like this and go into a meeting and have some people physically there and have some people show up in this photorealistic form,

• 15:36Mark Zuckerberg?

  superimposed on the, on the physical environment. I think that stuff like that is going to be super powerful. So we got to still build out all those kind of applications and the use cases around it. But I don't know, I think it's going to be a pretty wild next few years around this.

• 15:50Lex Fridman?

  I mean, I just, I'm actually almost at a loss of words. This is just so incredible. This is truly incredible. I, I hope that people like watching this can get a glimpse of like how incredible it is. It really feels like we're in the same room. Like there is that, I guess there's an uncanny valley that seems to have been crossed here. Like it looks like you.

• 16:10Mark Zuckerberg?

  Yeah. I mean, I think there's still a bunch of tuning that I think we'll want to do where different people emote to different extents, right? So I think one of the big questions is, you know, like when you smile, how wide is your smile and how wide do you want your smile to be? And I think getting that to be tuned on a per person basis is, is going to be one of the things that we, that we're going to need to figure out. You know, it's like, to what extent do you want to give people control over that? You know, some people might try to, you know, might, might prefer a version of themselves that's more emotive in their avatar than their actual faces.

• 16:51Mark Zuckerberg?

  You know, so for example, you know, I, I always get a lot of critique and shit for, for, for having like a relatively stiff expression. But you know, I mean, I, I might, I might feel pretty happy, but just make a pretty small smile. So I mean, maybe, you know, for, for me, I would, it's actually, you know, it's like, I'd want to have my avatar really be able to better express like how I'm feeling than, than what, than how I can do physically. So I think that there's a question about how you want to tune that. But, but overall, yeah, I mean, you, we, we want to start from the baseline of capturing how people actually emote and express themselves. And I mean, I think the, the initial version of this is, has been pretty impressive. And like you said, I do think we're, we're kind of beyond the, the uncanny valley here

• 17:34Mark Zuckerberg?

  where it, and it does feel like you, it doesn't feel, it doesn't feel weird or anything like that.

• 17:39Lex Fridman?

  I mean, that's gonna be the meme that the two most monotone people are in a metaverse together. But I think that actually makes it more difficult. The amazing thing here is that the subtleties of the expression of the eyes, you know, people say I'm monotone and emotionless, but I'm not. It's just maybe my expression of emotion is more subtle usually like with the eyes. And that's one of the things I've noticed is just how expressive the subtle movement of the corners of the eyes are in terms of displaying happiness or boredom or all that kind of stuff.

• 18:13Mark Zuckerberg?

  I am curious to see, just because I've never done one of these before, I've never done a podcast as one of these codec avatars. And I'm curious to see what people think of it because, you know, one of the issues that we've had in some of the VR and mixed reality work is it tends to feel a lot more profound when you're in it than the 2D videos capturing the experience. So I think that this one, because it's photorealistic, may look kind of as amazing in 2D for people watching it as it feels, I think, to be in it. But we've certainly had this issue where a lot of the other things, just it's like you feel the sense of immersion when you're in it that that doesn't quite translate to a 2D screen. But I don't know, I'm curious to see what people think.

• 18:55Lex Fridman?

  Yeah, I'm curious to see if people could see that, like, my heart is actually beating fast now. This is super interesting, like that such intimacy of conversation can be achieved remotely. There's been, you know, I don't do remote podcasts for this reason. And this is like breaks all of that. This feels like just an incredible transition to something else, to the different kind of communication, breaks all barriers, like geographic, physical barriers. You mentioned, do you have a sense of timeline in terms of how many difficult things have to be solved? And make this more accessible, like scanning with a smartphone?

• 19:36Mark Zuckerberg?

  Yeah, I mean, I think we'll probably roll this out progressively over time. So it's not going to be like we roll it out and one day everyone has a codec avatar. We want to get more people scanned and into the system. And then we want to start integrating it into each one of our apps, right? Making it so that, you know, I think that for a lot of the work style things, productivity, I think that this is going to make a ton of sense. In a lot of game environments, I mean, this could be fine, but games tend to have their own style, right? Where you almost want to fit more with the aesthetic style of the of the game. But I think for doing meetings, and one of the things that we get a lot of feedback on workrooms, where people are pretty blown away by the experience and this feeling that you can like

• 20:18Mark Zuckerberg?

  be remote, but feel like you're physically there around a table with people. But then, you know, we get some feedback that people have a hard time with the fact that the avatars are so expressive and and don't feel as realistic in that environment. So I think something like this could make a very big difference for those remote meetings. And especially with Quest 3 coming out, which is going to be the first mainstream mixed reality product, right? Where you're really taking digital,

• 20:44Mark Zuckerberg?

  you know, expressions of either a person or or objects and overlaying them on the physical world. I think the ability to do kind of remote meetings and and things like that where you're

• 20:56Mark Zuckerberg?

  just remote hang sessions with friends. I think that that's going to be very exciting. So yeah, rolling it out over the next over the next few years, it's not ready to be like a kind of mainstream product yet. But we just want to, we'll keep tuning it and keep getting more scans in there and keep, you know, and kind of rolling it out into more of the features. But yeah, I mean, definitely in the next, in the next few years, you'll be seeing a bunch more experiences like this.

• 21:18Lex Fridman?

  Yeah, I would love to see some celebrities scanned and some non-celebrities.

• 21:24Lex Fridman?

  I just, just more people to experience this. I would love to see that. This is something that, I mean, on my mind, I'm literally at a loss for words. Because it's very difficult to just. Convey how incredible this is, how like, how I feel the emotion, how I feel the presence, how I feel like the subtleties of the emotion in terms of like work meetings or any kind of, in terms of podcasts, this is like, this is awesome. And I don't even need your arms or legs.

• 21:51Mark Zuckerberg?

  Well, we gotta, we gotta, we gotta get that. I mean, that's, that's its own challenge. And, and part of the question is also, so you have the scan, then it takes a certain amount of compute to go drive that, both for the sensors on the headset and, and then rendering it. So one of the things that we're working through is what is the level of fidelity that is optimal, right? You could do the full body in, in kind of a codec, and that can be quite intensive. But, but one of the things that we're, we're thinking about is like, all right, maybe you can kind of stitch a somewhat lower fidelity version of your body, but still, still have the main kind of, the major movements. But, but, but

• 22:34Mark Zuckerberg?

  your face is really the thing that we have the most resolution on, right? In terms of being able to read and express emotions. I mean, like you said, if you move your eyebrows like a millimeter, I mean, that really changes the expression and what you're, you're emoting. Whereas, you know, I mean, moving your, your arm like a, an inch probably doesn't matter quite as much. So, so yes, I think that we'll, we do want to get all of that into here and that'll be some of the work over the next period as well.

• 23:01Lex Fridman?

  So you mentioned Quest 3, that's coming out. I've gotten a chance to try that too. That's awesome. So the, how'd you pull off the mixture? So it's not just virtual reality, it's mixed reality?

• 23:11Mark Zuckerberg?

  Yeah, I mean, I think it's gonna be, it's gonna be the first mainstream mixed reality device. And obviously, we shipped Quest Pro last year, but it was $1,500. And well, part of what I'm super proud of is, you know, we try to innovate, not just on pushing the state of the art and delivering new capabilities, but making it so it can be available to everyone. And, you know, we have this and it's, it's coming out and it's $500. And in, in some ways, I think the mixed reality is actually better in Quest 3 than it was, than, than what we're using right now in Quest Pro. So, and I'm really proud of the team for being able to deliver that kind of an innovation and get it out. But, you know, some of this is just,

• 23:51Mark Zuckerberg?

  Software you tune over time and get to be better. Part of it is you put together a product and you figure out what are the bottlenecks in terms of making it a good experience. So we got the resolution for the mixed reality cameras and sensors to be multiple times better in Quest 3. And we just figured that that that made a very big difference when we saw the experience that we were able to put together for Quest Pro. And part of it is also that Qualcomm just came out with their next generation chipset for VR and MR. And that we worked with them on a kind of custom version of it. But that was available this year for Quest 3 and it wasn't available in Quest Pro. So in a way, I'm Quest 3, even though it's not the Pro product, actually has a stronger chipset in it than the Pro line at a third of the cost.

• 24:36Mark Zuckerberg?

  So I'm really excited to get this in people's hands. It does all the VR stuff that Quest 2 and the others have done too. It does it better because the display is better and the chip is better. So you'll get better graphics. It's 40% thinner, so it's just more comfortable as well. But the MR is really the big capability shift. And part of what's exciting about the whole space right now is this isn't like smartphones where companies put out a new smartphone every year. And you can almost barely tell the difference between that and the one the year before it. Now, for this, each time we put out a new headset, it has like a major new capability. And the big one now is mixed reality.

• 25:18Mark Zuckerberg?

  The ability to basically take digital representations of people or objects and superimpose them on the world. And basically, I mean, there's a one version of this is you're going to kind of have these augments or holograms and experiences that you can kind of bring into your living room or a meeting space or office. Another thing that I just think is going to be a much kind of simpler innovation is that there are a lot of VR experiences today that don't need to be fully immersive. And if you're playing a shooter game or you're doing a fitness experience, sometimes people get worried about swinging their arms around. Like, am I going to hit a lamp or something? You know, it's and am I going to run into something?

• 26:02Mark Zuckerberg?

  So having that in mixed reality actually is just a lot more comfortable for people, right? You you kind of still get the immersion and the 3D experience. And you can you can have an experience that just wouldn't be possible in the physical world alone. But by being anchored to and being able to see the physical world around you, it's like it just feels so much safer and more secure. And I think a lot of people are really going to enjoy that, too. So, yeah, I'm really excited to see how people use it. But yeah, Quest 3 coming out later this fall.

• 26:28Lex Fridman?

  Yeah, and I got to experience it with other people sitting around and there's a lot of furniture. And so you get to see that furniture, you get to see those people and you get to see those people like enjoy the ridiculousness of you like swinging your arms. I mean, presumably they're friends of yours, even if they make fun of you, they uh, there's a lot of love behind that. And that you got to experience that's that's a really fundamentally different experience than just pure VR. With like with zombies coming out of walls and gee.

• 26:54Mark Zuckerberg?

  Yeah, it's like someone shooting at you and you hide behind your real couch in order to to duck the the fire. Yeah.

• 27:00Lex Fridman?

  It's incredible how it's all integrated, but also like subtle stuff like in a room with no windows, you can add windows to it and you can look outside as the zombies run towards you. But like, it's still a nice view outside, you know?

• 27:12Mark Zuckerberg?

  Yeah.

• 27:12Lex Fridman?

  It's just, it's really, and so that's pulled off by having cameras on the outside of the headset that do the pass through. And that technology is incredible to do that on a small headset.

• 27:24Mark Zuckerberg?

  Yeah, and it's not just the cameras. You basically need to, you need multiple cameras to capture the different angles and and sort of the three dimensional space. And then it's a pretty complex compute problem and AI problem to map that to your perspective, right? Because the cameras aren't exactly where your eyes are, because no two people's eyes are are you're not going to be in exactly the same place. You kind of need to to to get that to to to line up. And then do that basically in real time and then generate something that looks that kind of feels natural.

• 27:55Mark Zuckerberg?

  And then superimpose whatever digital objects you want to put there. So it's, yeah, it's a very interesting technical challenge. And I think we'll continue tuning this for, for the years to come as well. But, but I'm pretty excited to, to get this out because I think Quest 3 is going to be the first device like this with, that millions of people are going to get. That's mixed reality. And it's only when you have millions of people using something that you start getting the whole developer community really starting to experiment and build stuff. Cause now there are going to be people who actually use it. So I think we'll get, you know, we got some of that flywheel going with Quest Pro, but I think it'll really get accelerated once Quest 3 gets out there. So yeah, I'm, I'm pretty excited about this one.

• 28:35Lex Fridman?

  Plus there's hand tracking without, so you don't need to have a control. So this camera, the cameras aren't just doing the pass through of the entire physical reality around you. It's also tracking the details of your hands in order to use that for like gesture recognition, this kind of stuff.

• 28:51Mark Zuckerberg?

  Yeah, we've been able to get way further on hand recognition in a shorter period of time than I expected. So that's been pretty cool. I don't know. Did you see the, the demo experience that we built around?

• 29:03Lex Fridman?

  Piano?

• 29:03Mark Zuckerberg?

  Like, yeah, the piano, learning to play piano.

• 29:06Lex Fridman?

  Yeah, it's incredible. You're basically playing piano on a table and it's, that's without any controller. And like how well it matches physical reality with no latency. And it's, it's tracking your hands with no latency and it's tracking all the people around you with no latency, integrating physical reality and digital reality. Obviously that connects exactly to this Kodak Avatar, which is in parallel, allows us to have ultra realistic copies of ourselves in this mixed reality. It's, uh, so like it's all converging towards like an incredible digital experience in the metaverse. To me, obviously I love the intimacy of conversations. So even this is awesome. But do you have other?

• 29:52Lex Fridman?

  Ideas of what this unlocks, of like something like Kodak Avatar unlocks in terms of applications, in terms of things we're able to do?

• 30:02Mark Zuckerberg?

  Well, there's what you can do with avatars overall in terms of superimposing digital objects on the physical world. And then there's kind of psychologically, what does having photorealistic do? So I think we're moving towards a world where we're going to have something that looks like normal glasses, where you can just see, you see the physical world, but you will see holograms. And in that world, I think that they're going to be, you know, not too far off, maybe, you know, by the end of this decade, we'll be living in a world where there are kind of as many holograms when you walk into a room as there are physical objects. And it really raises this interesting

• 30:43Mark Zuckerberg?

  question about what are, about, you know, a lot of people have this phrase where they call the physical world, the real world. And, you know, I kind of think increasingly, you know, the physical world is super important, but I actually think the real world is the combination of the physical world and the digital worlds coming together. But until this technology, they were sort of separate, right? It's like you access the digital world through a screen, right? And, you know, maybe it's a small screen that you carry around, or it's a bigger screen when you sit down at your desk and strap in for a long session. But, but they're kind of fundamentally divorced and disconnected. And I think part of what this technology is going to do is bring those together into a single coherent experience

• 31:27Mark Zuckerberg?

  of what the modern real world is, which is, it's got to be physical because that we're physical beings. So the physical world is always going to be super important. But, but increasingly, I think a lot of the things that we kind of think of can be digital holograms. I mean, any screen that you have can be a hologram, you know, any media, you know, any book, art, you know, can basically be just as effective as a hologram as a physical object, any game that you're playing, a board game, or, or any kind of physical game, cards, you know, ping pong, things like that, they're often a lot better as holograms because you could just kind of snap your fingers and instantiate them and have them.

• 32:08Mark Zuckerberg?

  Show up, you know, it's like you have a ping pong table show up in your living room, but then you can snap your fingers and have it be gone. So that's super powerful. So I think that it's actually an amazing thought experiment of like, how many physical things we have today that could actually be better as interactive holograms. But then beyond that, I think the, the most important thing obviously is people. So the ability to have these mixed hangouts, whether they're social or meetings where you show up to a conference room, you're wearing glasses or a headset in the very near term, but you know, hopefully by, you know, over the next five years, glasses or so. And, and you know, you're there physically, some people are there physically, but other people are just

• 32:53Mark Zuckerberg?

  there as holograms and it feels like it's them who are right there. And also, by the way, another thing that I think is going to be fascinating about being able to blend together the digital and physical worlds in this way is we're also going to be able to embody AIs as well. So I think you'll also have meetings in the future where you're basically, you know, maybe you're sitting there physically and then you have, you know, a couple of other people who are there as holograms. And then you have like Bob, the AI, who's an engineer on your team who's helping with things and he can now be embodied as a, you know, as, as a realistic avatar as well and just join the meeting in that way. So I think that that, that's going to be pretty compelling as well. So then, okay.

• 33:38Mark Zuckerberg?

  So what can you do with photorealistic avatars compared to kind of the more expressive ones that we have today? Well, I think a lot of this actually comes down to acceptance of the technology. And because all of the stuff that we're doing, I mean, the, the motion of your eyebrows, the motion of your eyes, the cheeks and all of that, there's actually no reason why you couldn't do that on an expressive avatar too. I mean, it wouldn't look exactly like you, but you can make a cartoon version of yourself and still have it be almost as expressive. But I do think that there's this bridge between the current state of, of most of our interactions in the physical world and where we're getting

• 34:21Mark Zuckerberg?

  in the future with this kind of hybrid, physical and digital world where I think it's going to be a lot easier for people to. Kind of take some of these experiences seriously with the photorealistic avatars to start. And then I'm actually really curious to see where it goes longer term. I could see a world where people stick to the photorealistic and maybe they modify them to make them a little bit more interesting, but maybe fundamentally we like photorealistic things. But I can also see a world that once people get used to the photorealistic avatars and they get used to these experiences, that I actually think that there could be a world where people actually

• 35:00Mark Zuckerberg?

  prefer being able to express themselves in kind of non, you know, ways that aren't so tied to their physical reality. And so that's one of the things that I'm really curious about. And I don't know, in a bunch of our internal experiments on this, one of the things that has, I thought was psychologically pretty interesting is people have no issues blending photorealistic stuff and not. So, you know, we could have a, you know, for this specific scene that we're in now, we, we happen to sort of be in a dark room. I think part of that aesthetic decision, I think, was based on the way you like to do your podcast. But we've done experiences like this where you have like a cartoony background,

• 35:45Mark Zuckerberg?

  but photorealistic people who you're talking to. And we seem to, like, people just seem to just think that that is completely normal, right? It doesn't bother you. It doesn't feel like it's weird. Another thing that, that we've experienced with is, is basically you have a photorealistic avatar that you're talking to. And then right next to them, you have an expressive kind of cartoon avatar. And that actually is pretty normal too, right? It's like, it's not that weird, right, to, to basically being, interacting with, with different people in different modes like that. So I'm not sure. I think it'll be an interesting question. To what extent these photorealistic avatars are like a key part of just transitioning from being comfortable in the

• 36:26Mark Zuckerberg?

  physical world to this kind of new, modern real world that, that kind of includes both the digital and physical? Or if this is like the long-term way that it stays? That's, that's a, I mean, I think that there are going to be uses for both the expressive and the photorealistic over time. I just don't know what the balance is going to be.

• 36:42Lex Fridman?

  Yeah, it's a really good, interesting philosophical question. But to me, in the short term, the photorealistic is amazing. To where I would prefer, like you said, the workroom, but like on a beach with a beer, to see a buddy of mine remotely on a chair next to me, drinking a beer. I mean, that as realistic as possible is an incredible experience. So I don't want any fake hats on him. I don't want any, just chilling with a, with a friend, drinking beer, looking at the ocean, while not being in the same place together. I mean, that.

• 37:14Mark Zuckerberg?

  Yeah.

• 37:15Lex Fridman?

  That experience is just, it's, it's, it's a fundamentally, uh, it's just a high quality experience of friendship. Whatever we seek in friendship, it seems to be present there in the same kind of realism I'm seeing right now. This is, this is totally a game changer. So to me, this is, I can see myself sticking with this for a long time.

• 37:35Mark Zuckerberg?

  Yeah, and, and I mean, it's also, it's novel and it's also a technological feat, right? It's like be, being able to pull this off is like, it's a, it's like a pretty impressive. And I, I think to some degree, it's just this kind of like awesome experience.

• 37:47Lex Fridman?

  Yeah.

• 37:48Mark Zuckerberg?

  Um.

• 37:48Lex Fridman?

  But I'm already, sorry to interrupt, I'm already forgetting that you're not real. Like this really, so it's novel.

• 37:57Mark Zuckerberg?

  It's, this is just a, an avatar version of me.

• 38:00Lex Fridman?

  That's a deep philosophical question, yes.

• 38:03Mark Zuckerberg?

  But I mean, but here's some of the, so I put this on this morning and I was like, all right, like, uh, it's like, okay, so this, my hair is a little shorter in this than my physical hair is right now. I probably need to go get a haircut. Um, and like, and I actually, I did happen to shave this morning, but, but if I hadn't, you know, I could still have this photorealistic avatar that is, that is more cleanly shaven, right? Even if I'm, you know, a few days in, um, physically. So I do think that there are going to start to be these subtle questions that seep in where the, the avatar is realistic, um, in, in the sense of this is kind of what you looked like at the time of capture, but it's not necessarily temporally accurate to exactly what you look like in this moment.

• 38:47Mark Zuckerberg?

  And I think that there are going to end up being, um, a bunch of questions that come from that over time that I think are going to be fascinating, too.

• 38:56Lex Fridman?

  You mean, just like the nature of identity of who we are? Are we the?

• 38:59Mark Zuckerberg?

  Yeah.

• 39:00Lex Fridman?

  People, you know how people do like, like summer beach body? Would people be for the scan, they'll try to lose some weight and look their best and sexiest with the nice hair and everything like that. I mean, it, it does, um, it does raise the question of.

• 39:16Lex Fridman?

  You know, if a lot of people interacting with the digital version of ourselves, who are we really? Are we the the entity driving the avatar? Or are we the avatar?

• 39:26Mark Zuckerberg?

  Well, I mean, I think our physical bodies also fluctuate and change over time, too. So I think there's a similar question of like, which version of that are we? Right? There's there's like the, I mean, it's and it's interesting identity question because, all right, it's like, I don't know, it's like, weight fluctuates or things like that. It's like, I think most people don't tend to think of themselves as the, I don't know, it's it's an interesting psychological question. Some, maybe some people, maybe a lot of people do think about themselves as the kind of worst version. But, you know, but I think a lot of people probably think about themselves as the best version. And and I and then it's like, what you are on a day to day basis doesn't necessarily map to to to either of those.

• 40:07Mark Zuckerberg?

  I know that's, yeah, it is. There will definitely be a bunch of, a bunch of social scientists and folks will have to, you know, and psychologists are really, there's going to be a lot to understand about how our perception of ourselves and others shifted from this.

• 40:25Lex Fridman?

  Well, this might be a bit of a complicated and a dark question, but one of the first feelings I had experiencing this is I would love to talk to loved ones. And the next question I have is, I would love to talk to people who are no longer here that are loved ones. So like, if you look into the future, is that something you think about when people pass away, but they can still exist in the metaverse? You could still have, you know, talk to your father, talk to your grandfather and grandmother and a mother once they pass away. The power of that experience is one of the first things my mind jumped to because it's like, this is so real.

• 41:04Mark Zuckerberg?

  Yeah, I think that. There are a lot of norms and things that people have to figure out around that. There's probably some balance where, you know, if someone has lost a loved one and is grieving, there may be ways in which, you know, being able to interact or relive certain memories could be helpful. But then there's also probably an extent to which it could become unhealthy. And I mean, I'm not an expert in that, so I think we'd have to study that and understand it in more detail. We have, you know, a fair amount of experience with how to handle death and identity and people's digital content through social media already, unfortunately, right?

• 41:46Mark Zuckerberg?

  Where there, you know, there's, you know, unfortunately, you know, people who use our services die every day and their families, you know, often want to have access to their profiles. And we have whole protocols that we go through where, you know, there are certain parts of it that, um, that we try to memorialize so that way the, the family can, can get access to it. So the way the account doesn't just go away immediately. But then there are other things that are, you know, important kind of private things that that person has. Like we're not going to give the family access to someone's messages, you know, for example. So, um, so yeah, I, I think that there's, there's some best practices, I think, from the current digital world that will carry over, but, um, but yeah, I think that this will enable some different things.

• 42:29Mark Zuckerberg?

  Another version of this is, is how this intersects with AIs, right? Because, and one of the, the things that, that we're really focused on is, you know, we, we want there to, we want the world to evolve in a way where there isn't like a single AI superintelligence, but where, you know, a lot of people are empowered by having AI tools to, to do their jobs and, you know, make their lives better. And if you're a creator, right, and if you run a, you know, podcast like you do, then, you know, you have a big community of people who are super interested to talk to you. I know you'd love to, you know, cultivate that community and you interact with them online outside of the, the podcast as well.

• 43:10Mark Zuckerberg?

  But, I mean, there's way more demand, both to interact with you and I'm sure you'd love to interact with the community more, but you just are limited by the number of hours in the day. So, you know, at some point, I think, making it so that You could build an AI version of yourself that could interact with people, you know, not if you die, but but while you're here to help, you know, help help people kind of fulfill this desire to to interact with you and your desire to build a community. And there's a a lot of interesting questions around that. And, you know, that's obviously, it's not just in in the metaverse. I think, you know, we'd want to make that work, you know, across all the messaging platforms, you know, WhatsApp and

• 43:53Mark Zuckerberg?

  Messenger and Instagram Direct. But, you know, there's certainly, you know, a version of that where if you could have an avatar version of yourself in the metaverse that people can interact with, and you could define that sort of an AI version where, you know, people would know that they're interacting with an AI, that it's not, you know, the the kind of physical version of you. But maybe that AI, even if they know it's an AI, is the next best thing, because they're probably not going to, you know, necessarily all get to interact with you directly. I I think that that could be a really compelling experience. There's a lot of things that we need to get right about it. That, you know, it's we're not ready to release the the version that a creator can can kind of build a version of themselves yet. But we're starting to experiment with it in terms of

• 44:35Mark Zuckerberg?

  releasing a number of AIs that people can interact with in different ways. And I I think that that is is also just going to be a very powerful, you know, set of capabilities that people have over time.

• 44:47Lex Fridman?

  So you've made major strides in developing these early AI personalities with the idea where you can talk to them across the meta apps and have like interesting, unique kind of conversations. What can you describe your vision there in these early strides? And what are some technical challenges there?

• 45:08Mark Zuckerberg?

  Yeah. So I mean, a lot of the vision comes from this idea that, you know, I I don't I don't think we necessarily want there to be like one big super intelligence. We want to empower everyone to both, you know, have more fun, accomplish their business goals, you know, it's just everything that that they're trying to do. And, you know, we don't tend to have, you know, one person that we work with on everything. And I don't think in the future we're going to have, you know, one AI that we work with. I think you're going to want

• 45:36Mark Zuckerberg?

  A variety of these. So there are a bunch of different uses. If some will be kind of more assistant oriented, there's a sort of the kind of plain and simple one that we that we're building is called just meta AI. It's simple. You can chat with it in any of your threads. It doesn't have a face, right? It's just it's it's just kind of more vanilla and neutral and kind of factual, but it could help you with a bunch of stuff. Then there are a bunch of cases that are more kind of business oriented. Let's say you want to contact a small business. You know, similarly, that business probably doesn't want to have to staff someone to man

• 46:18Mark Zuckerberg?

  the phones and you probably don't want to wait on the phone to talk to someone, but having someone who you can just like talk to in a natural way, who can help you if you're having an issue with a product, or if you want to make a reservation, or if you want to buy something online, having the ability to do that and have a natural conversation rather than having to get some website or have to call someone and wait on hold, things can be really good, both for the businesses and for for normal people who want to interact with businesses. So I think stuff like that makes sense. Then there are going to be a bunch of use cases that I think are just fun, right? So I think people are gonna, I think that there will be AIs that can tell jokes. You can put them into chat thread with friends. And I think a lot of this, because we're like a

• 47:01Mark Zuckerberg?

  social company, right? I mean, we're fundamentally around helping people connect in different ways. And part of what I'm excited about is how do you enable these kind of AIs to facilitate connection between two people or more, you know, put them in a group chat, you know, make the group chat more interesting around whatever your interests are, sports, fashion, trivia.

• 47:25Lex Fridman?

  Video games. I love the idea of of playing, I think you mentioned Baldur's Gate, an incredible game, just having an AI that you play together with. I mean, that could, that seems like a small thing, but it could deeply enrich the like gaming experience.

• 47:42Mark Zuckerberg?

  I do think that AIs will be, will make the NPC is a lot better in games too. So that's a separate thing that I'm pretty excited about. But But yeah, I mean, one of the one of the AIs that we've built that just in our internal testing, people have loved the most is like a like an adventure text based, like a dungeon master.

• 48:03Lex Fridman?

  Yeah, nice.

• 48:04Mark Zuckerberg?

  And and I think, you know, part of what has been fun, and we talked about this a bit, but we've gotten some like real kind of cultural figures to play a bunch of these folks and be the embodiment and the avatar of them. So Snoop Dogg is the dungeon master, which I think is just hilarious.

• 48:22Lex Fridman?

  In terms of the next steps of, you know, if you mentioned you mentioned Snoop to create a Snoop AI, so basically AI personality replica, a copy, or not a copy, maybe inspired by Snoop, what are the some of the technical challenges of that? What does that experience look like for Snoop to be able to create that AI?

• 48:45Mark Zuckerberg?

  So starting off, creating new personas is easier because it doesn't need to stick exactly to what, you know, that physical person would want, how they'd want to be represented, right? It's like, it's just a new character that we created. So even though Snoop in that case is, you know, he's, you know, he's basically an actor, right? He's playing the dungeon master, but it's not Snoop Dogg, right? It's whoever the dungeon master is. If you want to actually make it so that you have an AI embodying a real creator, there's a whole set of things that you need to do to make sure that that AI is not going

• 49:25Mark Zuckerberg?

  to say things that the creator doesn't want, right? And and that the AI is going to know things and be able to represent things in the way that the creator would want, the way that the creator would know. So I think that it's less of a it's less of a question around like having the avatar express them. I mean, that that I think we're, you know, it's like, well, we have our kind of V one of that that will release soon is after Connect, but you know, that'll get better over time. But a lot of this is really just about continuing to make the the models for these AIs so that they're just more and more, I don't know,

• 50:06Mark Zuckerberg?

  you could say like reliable or predictable in terms of what they'll communicate. It's that way, you know, when you want to create the Lex assistant AI that that your community can talk to, you can You know, you don't program them like normal computers. You're training them. They're AI models, not kind of normal computer programs, but you want to get it to be predictable enough so that way you can set some parameters for it. And even if it isn't perfect all the time, you want it to generally be able to stay within those bounds. So that's a lot of what I think we need to nail for the creators. And that's why that one's actually

• 50:48Mark Zuckerberg?

  a much harder problem, I think, than starting with new characters that you're creating from scratch. So that one, I think, will probably start releasing sometime next year, not this year, but experimenting with existing characters and the assistant and games and a bunch of different personalities and experimenting with some small businesses. I think that that stuff will be ready to do this year and we're rolling it out, you know, basically right after Connect.

• 51:16Lex Fridman?

  Yeah, I'm deeply entertained by the possibility of me sitting down with myself and saying, hey, man, like, you need to stop the dad jokes, or whatever.

• 51:25Mark Zuckerberg?

  I think the idea of a podcast between you and AI assistant Lex podcast.

• 51:33Lex Fridman?

  I mean, there is a, just even the experience of a Kodak Avatar, being able to freeze yourself, like, to basically first mimic yourself. So everything you do, you get to see yourself do it. That's a surreal experience. That feels like if I was like an ape looking in a mirror for the first time, realizing, like, oh, that's you. But then freezing that and being able to look around like I'm looking at you. It's a, I don't know how to put it into words, but it just feels like a fundamentally new experience. Like, I'm seeing maybe color for the first time. I'm seeing, I'm experiencing a new way of seeing the world for the first time, because it's physical reality, but it's digital. Like, and realizing that that's

• 52:18Lex Fridman?

  possible is just, it's blowing my mind. This is really exciting. Because I lived most of my life, you know, before the internet and experiencing the internet, experiencing voice communication, video communication. You think like, well, there's a ceiling to this, but this is making me feel like, oh, there might not be, there might be that blend of physical reality and digital reality. It's actually what the future is.

• 52:46Mark Zuckerberg?

  Yeah, I think so.

• 52:46Lex Fridman?

  It's a weird experience. It's a, it feels like the early days of, of like a totally new way of living. And like, there's a lot of people that kind of complain, well, you know, the internet is not, that's not reality. You need to turn all that off and go, you know, in nature. But this feels like this will make those people happy, I feel like, because it feels real. The flaws and everything.

• 53:11Mark Zuckerberg?

  Yeah, well, I mean, a big part of how we're trying to design this, these new computing products is that they should be physical. But I think part, that's a big part of the issue with computers and TVs and even phones is like, yeah, I mean, maybe you can interact with them in different places, but they're, they're fundamentally like you're sitting, you're, you're still. And I mean, people are just not meant to be that way. I mean, I think you, you and I have this shared passion for, for sports and martial arts and doing stuff like that, where you're just moving around. It's like so much of what makes us people is like, you, you move around. You're not, we're not just like a brain in a tank, right? It's the, where, you know, the, the human experience is a physical one. And, um, so it's, it's not just about having the immersive expression of the digital world.

• 53:55Mark Zuckerberg?

  It's about being able to really natively bring that together. And, and it, it, I do really think that the, the real world is this mix of the physical and the digital, right? The digital is, there's too much digital at this point for it to just be solid to a small screen, but the physical is too important. So you, you don't want to just sit down all day long, um, at a desk. So I, I think that this is, uh, uh, yeah, I do think that this is the future. This is, I, I think the kind of philosophical way that I would want the world to work in the future is a much more coherently blended physical and digital world.

• 54:30Lex Fridman?

  There may be some difficult philosophical and ethical questions we have to figure out as a society. Uh, maybe you can comment on this. So the, the metaverse seems to enable.

• 54:43Lex Fridman?

  Sort of unlock a lot of experiences that we don't have in the physical world. And the question is, like, what is and isn't allowed in the metaverse? You know, in video games, we allow all kinds of crazy stuff. And in physical reality, you know, a lot of that is illegal. So where's that line? Where's that gray area between video game and physical reality? Do you have a sense of that?

• 55:10Mark Zuckerberg?

  Well, I think, I mean, there are, there are content policies and things like that, right? In terms of what, what people are allowed to create. But I mean, a lot of the rules around physical, I think we try to have a society that is as free as possible, meaning that people can do as much of what they want unless you're going to do damage to other people and infringe on, on their rights. And the idea of damage is somewhat different in a, in a digital environment. I mean, when I get into, you know, some world with my friends, the, the first thing we start doing is shooting each other, which obviously we would not do in the physical world because you'd hurt each other. But in, in a game, that's like just a, it's almost, you know, it's like just fun. And, um, and even in like the lobby of a game, right?

• 55:54Mark Zuckerberg?

  It's like, it just, you, it's not even bearing on the game. It's just kind of like a funny, um, sort of humorous thing to do. So it's like, is that, is that problematic? I don't think so, because it's, it's fundamentally, it's not, you're not causing harm in that world. So I think that the, um, part of the question that I think we need to figure out is, what are the ways where things could have been harmful in the physical world that will now be freed from that? And therefore there should be fewer restrictions in the digital world. And then there might be new ways in which there could be harm in the digital world that there weren't the case before. So there's more anonymity, right? It's, um, you know, when you, when you show up to a, you know, a restaurant or something, it's like all the norms where you pay the bill at the end.

• 56:36Mark Zuckerberg?

  It's because, you know, you, you, you have one identity and, you know, the, you know, if you, if you stiff them, then like, you know, life is a repeat game and that's not going to work out well for you. But, you know, in a digital world where you can be anonymous and show up in different ways, I think the incentive to act like a good citizen can be a lot less. And that causes a lot of issues and toxic behavior. So that, that needs to get sorted out.

• 57:02Mark Zuckerberg?

  So I think in terms of what is allowed, I think you want to just look at what are the damages. But then there's also other things that are not related to harm, less about what should be allowed and more about what will be possible, that are more about the laws of physics. It's like if you wanted to travel to see me in person, you'd have to get on a plane and that would take a few hours to get here, whereas we could just jump in a conference room and put on these headsets and we're basically teleported into a space where it feels like we're together. So that's a very novel experience, that it breaks down some things that previously

• 57:45Mark Zuckerberg?

  would have defied the laws of physics for what it would take to get together. And I think that that will create a lot of new opportunities. So one of the things that I'm curious about is, there are all these debates right now about remote work or people being together. And I think this gets us a lot closer to being able to work physically in different places, but actually have it feel like we're together. So I think the dream is that people will one day be able to just work wherever they want, but will have all the same opportunities because you'll be able to feel like you're physically together. I think we're not there today with just video conferencing and the basic technologies that we have. But I think part of the idea is that with something like this, over time, you could get closer to that. And that would open up a lot of opportunities, right?

• 58:29Mark Zuckerberg?

  Because then people could live physically where they want while still being able to get the benefits of being physically or kind of feeling like you're together with people at work, all the ways that that helps to build more culture and build better relationships and build trust, which I think are real issues that if you're not seeing people in person ever. So yeah, I don't know. I think it's going to be, it's very hard from first principles to think about all the implications of a technology like this and all the good and the things that you need to mitigate. So you try to do your best to envision what things are going to be like and accentuate the things that they're going to be awesome and hopefully mitigate some of the downside things.

• 59:14Mark Zuckerberg?

  But the reality is that we're going to be building this out one year at a time. It's going to take a while. So we're going to just get to see how it evolves and what developers and different folks do with it.

• 59:26Lex Fridman?

  If you could comment, this might be a bit of a very specific technical question, but Llama 2 is incredible. You released it recently. There's already been a lot of exciting developments around it. What's your sense about its release? And is there a Llama 3 in the future?

• 59:48Mark Zuckerberg?

  Yeah, I think on the last podcast that we did together, we were talking about the debate that we were having around open sourcing Llama 2. And I'm glad that we did. I think at this point, the value of open sourcing a foundation model like Llama 2 is significantly greater than the risks, in my view. We spent a lot of time to give a very rigorous assessment of that and red teaming it. But I'm very glad that we released Llama 2. I think the reception has been... It's just been really exciting to see how excited people have been about it. And it's gotten way more downloads and usage than I would have even expected.

• 1:00:32Mark Zuckerberg?

  And I was pretty optimistic about it. That's been great. Llama 3? I mean, there's always another model that we're training. So for right now, we trained Llama 2, and we released it as an open source model. And right now, the priority is building that into a bunch of the consumer products, all the different AIs and a bunch of different products that we're basically building as consumer products. Because Llama 2 by itself, it's not a consumer product, right? It's more of a piece of infrastructure that people could build things with. So that's been the big priority, is continuing to fine tune

• 1:01:16Mark Zuckerberg?

  and just get Llama 2 and its little branches that we built off of it, ready for consumer products that hopefully hundreds of millions of people will enjoy using those products in billions one day. But yeah, we're also working on the future foundation models. And I don't have anything...

• 1:01:41Mark Zuckerberg?

  New or news on that, I don't know. I don't know exactly when it's going to be ready. I think just like we had a debate around Lama 2 and open sourcing it. I think we'll need to have a similar debate and process to red team this and make sure that this is safe. But and my hope is that we'll be able to to open source this next version when it's ready too. But but that's not that we're not. We're not, you know, close to doing that this month. I mean, this is that's just it's a thing that we're we're still somewhat early in working on.

• 1:02:12Lex Fridman?

  Well, in general, thank you so much for open sourcing Lama 2 and for being transparent about all the exciting developments around AI. I feel like that's contributing to a really awesome conversation about where we go with AI. And obviously, it's really interesting to see all the same kind of technology integrated into these personalized AI systems with the with the AI personas, which I think when you put it in people's hands and they get to have conversations with these AI personas, you get to see like interesting failure cases, like where the things are dumb or they go into weird directions. Or and we get to learn as a society together what's what's too far, what's interesting, what's fun,

• 1:02:55Lex Fridman?

  how much personalization is good, how much generic is good, and we get to learn all of this. And you probably don't know this yourself, like we have to all figure it out by using it, right?

• 1:03:05Mark Zuckerberg?

  Yeah, I mean, part of what we're trying to do with the initial AI's launch is having a diversity of different use cases, just so that people can try different things because I don't know what's going to work. I mean, are people going to like playing in the text based adventure games? Are they going to like having a comedian who can add jokes to threads? Or they can want to interact with historical figures? You know, we made we made one of Jane Austen and one of Marcus Aurelius, and I'm curious to see how that goes.

• 1:03:38Lex Fridman?

  I'm excited for both. I'm a big fan. I'm excited for both. I have a conversation with them. I mean, yeah, that's, you know, and I am also excited to see, you know, the internet. I don't know if you heard can get kind of weird. And I applaud them for it.

• 1:03:52Mark Zuckerberg?

  I've heard that, yeah.

• 1:03:53Lex Fridman?

  Yeah, it'd be, it'd be nice to see how weird they take it. What kind of memes are generated from this? And I think all of it is, especially in these early stages of development. As we progress towards AGI, it's good to learn by, by playing with those systems and interacting with them at like a large scale, like you said.

• 1:04:12Mark Zuckerberg?

  Yeah, totally. I mean, that's why, well, we're starting out with a set. And then we're also working on this platform that we call AI Studio. That's going to make it so that, you know, over time, anyone will be able to create one of these AIs, almost like they create any other UGC content across the platform. So I'm excited about that. I think that to some degree, we're not going to see the full potential of this until you just have the full creativity of the whole community being able to build stuff. But there's a lot of, a lot of stuff that we need to get right. So, so I'm excited to take this in stages. I don't, I don't think anyone out there is really doing what we're doing here.

• 1:04:54Mark Zuckerberg?

  I think that there are, there are people who are, who are doing kind of like fictional or consumer oriented character type stuff. But the extent to which we're building it out with the avatars and expressiveness and, and making it so that they can interact across, you know, all of the different apps. And they'll have profiles, you know, we'll be able to engage people on Instagram and Facebook. I, I, I think it's, it's just, it's, it's going to be really fun.

• 1:05:23Lex Fridman?

  Well, I'm still, so we're talking about AI, but I'm still blown away this entire time that I'm talking to Mark Zuckerberg and you're not here, but you feel like you're here. I've done. Quite a few intimate conversations with people alone in a room. And this feels like that. So I keep forgetting for long stretches of time that like we're not in the same room. And for me to imagine a future where I can with a snap of a finger do that with anyone in my life. The way we can just call right now and have this kind of shallow 2D experience.

• 1:05:57Lex Fridman?

  To have this experience like we're sitting next to each other is like, I don't, I don't think I can. I don't think we can even imagine what, how that changes things. Where you can immediately have intimate one-on-one conversations with anyone. That's, that might like, in a way, we might not even predict changed civilization.

• 1:06:18Mark Zuckerberg?

  Well, I mean, this is a lot of the thesis behind the whole metaverse is giving people the ability to feel like you're present with someone. I mean, this is like the main thing I talk about all the time, but I, I do think that there's a lot to, to process about it. I mean, from my perspective, I mean, I, I'm definitely here. We're just not, we're, we're not physically in the same place. It's not like you're, you know, you're not talking to an AI, right? You're, you know, this is, um, so I, I think the, the thing that's novel is the ability to convey through technology. A sense of almost physical presence. So the, the thing that is not physically real is, um, is us being in the same physical place. But, uh, but, but kind of everything else is.

• 1:07:01Mark Zuckerberg?

  And I think that that gets to this somewhat philosophical question about what is the nature of kind of the modern real world? And I, I just think that that's, it really is this combination of a physical world and the presence that we feel. But also being able to combine that with this increasingly rich and powerful and capable digital world that we have. And, and, and all of the, the innovation that's getting created there. So, um, I, I think it's super exciting because I mean, the, the digital world is just, is just increasing. And in its capability and our ability to do awesome things. But the physical world is so profound and that's a lot of what makes us human is, is that we're, we're, we're physical beings. So I don't think we want to run away from that and just spend all day on a screen.

• 1:07:44Mark Zuckerberg?

  And that's like, you know, it's one of the reasons why I care so much about, about helping to shape and accelerate the, these future computing platforms. I just think this is so powerful and it's, it's, you know, even though. The current version of this is like you're wearing a headset. I just think this is gonna be by far the most human and social computing platform that has ever existed. And that's what makes me excited.

• 1:08:10Lex Fridman?

  Yeah, I think just to linger on this kind of changing nature of reality, like of what is real, maybe shifting it towards the sort of consciousness of what is real is the subjective experience of a thing that makes it feel real versus necessarily being in the same physical space. Because it feels like we're in the same physical space.

• 1:08:34Mark Zuckerberg?

  Yeah.

• 1:08:35Lex Fridman?

  And that the conscious experience of it, that's probably what is real, not like that the space-time, like the physics of it, like you're basically breaking physics and focusing on the consciousness, that's what's real. It's whatever's going on inside my head.

• 1:08:51Mark Zuckerberg?

  But there are a lot of social and psychological things that go along with that experience that was previously only physical presence, right? I think that there's like an intimacy, a trust, you know, there's a level of communication because so much of communication is non-verbal and is based on expressions that you're kind of, you're sharing with someone when you're in this kind of environment. And before those things would have only been possible, you know, had I gotten on a plane and flown to Austin and sat physically with you in the same place. So I think we're basically short cutting those laws of physics and delivering the social and psychological benefits of being

• 1:09:36Mark Zuckerberg?

  able to be present and feel like you're there with another person, which I think could real benefits to anyone in the world. And I think that that, like you said, I mean, I think that is gonna be a very profound thing. And that, a lot of that is, that's the promise of the metaverse and what, you know, why, you know, I just. Well, I think that that's the next frontier for what we're working on. You know, I started working on social networks when they were primarily text. Where the first version of Facebook, your profile, you know, you had one photo and the rest of it was like lists of things that you were interested in. And then we kind of went through the period where we were doing photos. And now we're kind of in the period where most of the content is video.

• 1:10:17Mark Zuckerberg?

  But there's a clear trend where, you know, over time, the way that we want to express ourselves and and kind of get insight and content about the world around us gets increasingly just richer and more vivid. And I think the ability to be immersed and feel present with the people around you or the people who you care about is, from my perspective, clearly the next frontier. It just so happens that it's incredibly technologically difficult, right? It requires building up these new computing platforms and completely new software stacks to deliver that. But I mean, I kind of feel like that's what we're here to do as a company.

• 1:10:55Lex Fridman?

  Well, I really love the connection you have through conversation. And so for me, this photorealism is really, really exciting. I'm really excited for this future and thank you for building it. Thanks to you and thanks to the amazing meta teams that I've met, the engineers and just everybody I've met here. Thank you for helping to build this future. And thank you, Mark, for talking to me inside the Metaverse. This is blowing my mind. I can't quite express. I would love to measure my heart rate this whole time. Would be hilarious if you're actually like sitting on a beach right now.

• 1:11:34Mark Zuckerberg?

  I'm not. I'm in a conference room.

• 1:11:36Lex Fridman?

  Okay, well, I'm at a beach and not wearing any pants. I'm really sorry about that for anyone else who's watching me in physical space. Anyway, thank you so much for talking today. This is, this really blew my mind. It's one of the most incredible experiences in my life, so thank you for giving that to me.

• 1:11:51Mark Zuckerberg?

  Awesome, awesome. Glad you got to check it out and it's always fun to talk. All right, I'll catch you soon.

• 1:11:56Lex Fridman?

  See you later. This is so, so amazing, man.

• 1:12:02Lex Fridman?

  This is so amazing.