Meta's CTO On Its Augmented Reality Dreams — With Andrew Bosworth

Andrew Bosworth is the Chief Technology Officer at Meta. Bosworth joins Big Technology Podcast to discuss Meta's new Orion augmented reality glasses and the future of human-computer interaction. Tune in to hear his insights on the technical capabilities of AR, the potential of artificial intelligence, and how these technologies could reshape the way we interact with computers and each other in the coming years. We also cover the evolution of computing paradigms, Meta's strategy in the AI and AR/VR space, and the philosophical implications of advanced AI. Hit play for a fascinating conversation with one of the top technology executives at the forefront of the AR and AI revolution. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/ Want a discount for Big Technology on Substack? Here’s 40% off for the first year: https://tinyurl.com/bigtechnology Questions? Feedback? Write to: [email protected] Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  Meta's Chief Technology Officer joins us from Menlo Park to talk about Orion, the company's new augmented reality glasses and what they mean for the future of computing. That's coming up right after this. This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis Chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, Head of Enterprise Business Solutions at Voya Financial, to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs, and lessons along the way.

• 0:42Alex Kantrowitz?

  You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay? Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference costs by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 1:14Alex Kantrowitz?

  Welcome to Big Technology Podcast, a show for cool-headed nuanced conversation of the tech world and beyond. And we're here in Menlo Park with Andrew Bosworth, Boz as he's known, the Chief Technology Officer of Meta. Boz, great to see you. Welcome to the show.

• 1:28Speaker 01

  Thanks for having me. Yeah, great to be here.

• 1:29Alex Kantrowitz?

  So I am just off a demo of Orion, your new holographic augmented reality glasses. First of all, they're crazy.

• 1:38Speaker 01

  That's what we're going for. First and foremost, they're crazy. I love that in Mark's, the Supercut Mark show, the kind of sizzle reel. It was just like people say, like 10 people in a row just saying, oh, this is crazy.

• 1:47Alex Kantrowitz?

  It's amazing because you expect it to be an experience you're having in a VR goggles. I'll tell people, you know, just for our listeners, so these are glasses. They don't feel extremely heavy. You wear them just on your face like you would normal glasses, but they're augmented reality. So you can make calls on them, you can watch videos on them, you can play games on them. And you have to like remind yourself like, oh, wow, like these are, these are glasses.

• 2:11Speaker 01

  That's right.

• 2:12Alex Kantrowitz?

  I'm not wearing goggles. So you've had the, and this is a 10-year project within Meta, but it looks like from what I saw with your team, that this is something that has really come a long way, even in the past year.

• 2:22Speaker 01

  That's right.

• 2:23Alex Kantrowitz?

  And you've had an opportunity to use them. What have you been using them for?

• 2:26Speaker 01

  Yeah, the hardware came up last year at this time. And so the software bring up really only came together a few months ago. And they just made incredible progress since then already. So for us, there's a couple of different things I'd call out. The first one is actually very relatable, I suspect, for those of us who worked during the pandemic. During the pandemic, you're on these, you know, obviously doing a lot of video calling. And one positive silver lining from all of that was the development of the back channel. So every, you know, meeting would have a chat alongside the meeting. And somebody could ask a question, hey, I missed that. Is there a link to that? Okay, cool. And so you're not interrupting the meeting to get this extra context. And what's been fun with these glasses is these glasses kind of serves like the in-meeting back channel. So now I'm in an in-person meeting, in a live meeting.

• 3:08Speaker 01

  And somebody can add context to a conversation that's happening in the glasses. So I'm like, oh, okay, somebody references a prototype. And then I, the next thing I know, up pops an image of the prototype that they were referencing. So now I'm fully in context again in the meetings. So these things are kind of wild, like these were, before this, it would have been very rude to me in the middle of this meeting with Mark Zuckerberg. Take my phone out, like, go look up the reference of the thing. So that's been kind of interesting, this real life back channel when you're in a live scenario. But let's also not overthink it. It's also a great way to scroll Instagram and like enjoy the reels and tap on those things. And to watch videos. The surprise hit, I would say, from the demos has been video calling. And there's two parts to that, right? One of which is the experience of having someone video call you while you're just walking around is really cool.

• 3:53Speaker 01

  And you can see them in full fidelity, but you can also see and navigate the world really effectively. You've got your hands free. And then conversely, what they would see is your codec avatar. And we've really fooled some people with the codec avatars. When they've seen the codec avatar call in, they don't realize it's codec avatar. They think it's a real, an image.

• 4:10Alex Kantrowitz?

  So let's just go right away to like what this does to real world experience. You mentioned it, right? So like, It's so hard to be in the moment now, right? And a lot of times you're speaking with someone, oh, there's a buzz on their phone. Hey, one second, let me check this out. And then we went, a lot of people went to smartwatches. I see you're not wearing one.

• 4:29Speaker 01

  Not currently, no.

• 4:30Alex Kantrowitz?

  And that also, like, you're like, sitting at a table with someone and they're like, keep on looking at their, their watch. So, does this take people out of the real world in a way that, you know, might be detrimental? I mean, what do you think about the real world implications here?

• 4:44Speaker 01

  No, tremendously important question. And we, we've tried to use the tagline, more present and connected. The idea of kind of reducing the dichotomy we often feel. And actually, one of my anti-patterns that I've been telling people about for these glasses is just having to be notification surfaces. Because notification services are an invitation to leave the present moment and instead go elsewhere. Like, that's what a notification is intended to do. It's to let you know, hey, if you want to, you could leave this physical plane and go dive into this digital surface over there. I want people to be able to do that, if that's what they want to do. But it can't just be that. Instead, though, if you think of the examples I've given, it's about adding context. And so, the real difference maker between these and a phone in your pocket, and all the things that are attached to a phone in your pocket,

• 5:27Speaker 01

  is they have the ability to be aware of the world at all times. And this is where AI has been such a tremendous tailwind for our vision for these. You know, we really thought, early on, when we were developing these, even until maybe a year ago, it was going to be holograms, and then, at some point later, AI. And in fact, it's going to be the opposite. The AI is going to come first. It's already come to, to the Meta Ray-Ban glasses. And so, for me at least, we've got this idea where, okay, what if these glasses are contextually aware? They know I'm talking to someone face to face. Hey, I don't need that notification about my cat is moving on the camera at home. I don't need that notification right now. We can hold off on that. Oh, someone's calling me, it's my kid's school.

• 6:06Speaker 01

  Okay, even though you're talking to someone's face, like, we can probably pipe that through and let you know, hey, your kid's school is calling. And so, that is the real missing piece for allowing these to both be more present and more connected when you're using them, is the contextual awareness that they have the potential to have about the world around you and how you're engaging in it.

• 6:26Alex Kantrowitz?

  And there's also just like a cool part of it that I saw in the demo is imagine you're just like at a restaurant waiting for somebody instead of like sitting and scrolling your phone, you could basically have a computer with you, which you just brought in with your glasses, you can do some work, you can watch a video, do whatever you want, and it's there with you.

• 6:44Speaker 01

  Yeah, one of the fun demos that we we have in prototyping, it's not even in the demo that you tried yet, is this cool thing where you you have your whole workstation set up. So you've got like a three panel workstation set up and then you like just get up and walk around and it goes away and it minimizes and now you've got like you're walking around UI if you need it. And then if you sit down again, you can just recall it immediately. So you're like all your context comes immediately back whenever you want it there. So not just a mobile and portable workstation, but one that's literally, you know, at the, you know, twitch of a finger, ready for you to use.

• 7:17Alex Kantrowitz?

  A lot of computing, it solves a problem, right? And another question I have for you is why do we need this? Like what type of problem does this solve? Can't we just get along fine with our phones? And I know that's going to sound stupid because every computing phase, they have been like, well, can't we just use desktops? Why do we need this in our pocket? But so what problem do you think this solves?

• 7:38Speaker 01

  No, it is. It's exactly the right question. It speaks to how we as a society so quickly normalize everything that's come before as completely normal and everything that's new is like kind of this unnecessary thing. Douglas Adams, a famous quote on this comes to mind. I'll leave that as an exercise to the to the listener. The so for me at least that, yeah, when people, I remember when cell phones came out and everyone was like, who is so important that they need a phone everywhere they go? And it's like, all of us are. It turns out all of us are that important. And I feel a little bit the same way. And some of these things will sound trivial, but having Meta AI is actually a really good test point. Having I've had Meta AI on my phone for a while as an employee of the company. And it's pretty useful. And I like asking it questions. Now I have Meta

• 8:19Speaker 01

  AI on the Ray-Ban Meta glasses. And it's like, oh, I use it 10 times more often. Because just the friction of like getting my phone out of my pocket and like pulling it up and like pulling the app is not trivial. To me being able to just like ask a question. Um, and now what if instead that AI didn't need to even be asked a question? It's anticipating, it sees me, I'm like checking my pockets, I'm checking, it's like, oh, your keys are over in the desk where you left them. Um, it, so these things start to come together where because the glasses are on your face, they're not in your pocket. And they have the ability over time, not immediately, to become aware of the world and to understand what you're trying to do in the world. Uh, they can actually be very proactively helpful. Just to you privately, you're in control.

• 9:02Speaker 01

  Um, but that is a thing that phones can't do. Uh, and I do think it is a real step forward in our relationship with digital technologies from going very transactional. Like I have gone to the technology and then I come back with my answer to being very integrated.

• 9:17Alex Kantrowitz?

  So is this basically a way to just deliver the AI platform more seamlessly?

• 9:22Speaker 01

  It's not just the AI platform. I think the AI platform is what changes the scope of how we interact with it. Um, from being completely reactive, which is what computers have been from time immemorial to being proactive. Um, I think it also of course is a super more convenient way to go ahead and manage communication and manager, like manage all the different things that phones do. Listen, use video calling again as an example. You know, this is just a terrible way to, oh, I'm holding my phone out, my hand out in front of my face. Like my arm is literally tired. I've been doing this for eight seconds and I'm like, I work out, man. Like, it's like, I can't. So there's a few things that are just fundamentally better on this platform. Um, and then there's a few things that I think, yeah, you can do it on the phone or you can do it here. But if this was already on your face, you wouldn't use your phone.

• 10:03Alex Kantrowitz?

  Right. I mean, I've definitely had those moments where I'm like video calling in New York and just running into people.

• 10:09Speaker 01

  Absolutely. I've walked into a post before, like I'm not proud.

• 10:14Alex Kantrowitz?

  Why is it important for Meta to do this? So Meta's mission long has been, all right, let's, you know, connect people. Obviously it was the name change. Now it's Metaverse or, you know, Meta. Why is it important for this company to build this device?

• 10:26Speaker 01

  Yeah, there's two different answers to this. One that I think people will recognize and understand strategically, we have just been at the mercy of platforms for since the start of the mobile era. And we were a web company, right? We were born of the web. And the web was this like amazing, unowned platform that you just like really had a lot of control over what you built and you had a direct relationship with consumers. And we absolutely feel that in our application development, we're stifled by the gatekeepers, especially Apple, in terms of the policies, what they allow, what they don't allow. And there's a real value that we could have delivered to humans who love our products and would be happy to have that value, but we weren't allowed to. We're just not allowed to.

• 11:11Speaker 01

  Well, anything that has to do with, you know, sort of, keep in mind, on the web, Facebook was a, had a major games platform, you know, which we were not allowed to bring to the iOS. Because of app store fees and app store rules. And gosh, I mean, and they've gone even further than that and kind of, you know, finally, we built an app install business and then Apple came for that with ATT, you know, a tremendous amount of kind of self-preferencing for themselves there. And in doing so, they really put, you know, the brakes on what was a really rapidly growing part of the games industry. And so anyway, so I think like, for us, like, platform control is important to us. That's a strategic reason. That's what everyone jumps on. That's a fair one. If I could be honest with you, that's like a very nice story. And I'm sure it's a part of it, but it's not the real story. The real story is that we're a technology company

• 11:54Speaker 01

  and Mark is an enthusiast who wants to continue to advance what is the state of the art and how we connect people with each other and with technology. And we've known for a long time that these are the next thing. As an industry, I think we've had a sense, AR especially, is like, we don't have to convince people that someday, AR is probably a thing that will exist and will be great. And this is where Michael Abrash, our chief scientist, has this kind of state, he's like, you know, this is the myth of technological eventualism, this idea that eventually the technology will exist. It's like, it's, it doesn't, that's not how it works. Someone has to stop. Sit down and actually build it. And it's a tremendous credit to Mark. I'm gonna sound like a kiss ass. I don't mean to. I'm very critical of Mark in many, many areas. But on this, he deserves his credit.

• 12:38Speaker 01

  It's a tremendous vision and commitment that he's made to this technology in the face of incredible scrutiny the last several years to stay with these investments and deliver Orion, which I think is just a watershed moment for us to say like this is, this technology is coming. It's here in some form. It's not evenly distributed, but it's here and, and we, it's like within, it's within our lifetime. Certainly within the next decade, it's in our view, we're going to be able to use and enjoy this technology. And I think he just wanted to do it.

• 13:07Alex Kantrowitz?

  And we should say that these glasses are internal. So over time, they might be released. They're not released yet.

• 13:13Speaker 01

  That's right.

• 13:13Alex Kantrowitz?

  But I do think we should talk a little bit more about the platform side. So, okay, for listeners.

• 13:17Speaker 01

  For listeners, I'm sorry. I just put on, I put on a pair of, of clear and transparent ones. The, the ones that we actually use are, have opaque magnesium shells. The clear transparent ones are super fun to have. They don't, they only work for about half an hour at a time before the thermal shut down because this plastic does not dissipate heat fast enough. But the real ones, the real ones, they don't get, they don't thermally exhaust themselves. They go till the battery is exhausted, two to three hours. But yeah, so it's pretty fun. It's just, I'm sorry, if I have these cool demo props, what fun is it if I can't put them on?

• 13:46Alex Kantrowitz?

  But let's talk a little bit about the, the platform risk and, and the operating system. I mean, do you have to build your own operating system for these?

• 13:53Speaker 01

  Yeah, so a couple of different stories to this. I mean, when we talk about what is an operating system? One of my former colleagues, Fikas Patrick, had this great saying. It was like, operating systems are a set of promises made to developers and to consumers. So you can think of in iOS, the swipe gesture will get you home. Like no matter what's happening on the app, that's a promise that you're making to a consumer and the apps can't control it. That's a promise that's being made. You know, you promise to developers like this memory is yours. You're allowed to use this memory. This API will cause this thing to happen. And so, you know, we're, we're all, we're using Android at the most underlying low level as, as the operating system. But of course, the operating system isn't Android. The, the thing that people are interacting with on, whether it be in mixed reality or in augmented reality, it doesn't look or feel like Android. And there's a bunch of reasons for that.

• 14:38Speaker 01

  Number one is the interaction paradigms are different. And so for augmented reality, to stay there for the moment, you're not going to have a direct manipulation interface. You don't have a keyboard. You don't have a mouse. You don't have a touchscreen. Well, it's worth noting, this is really the first computing platform that we've had. Ever that didn't have a direct manipulation interface. I mean, you're going back to Stanford Research Institute, Xerox PARC, the keyboard and mouse. Before that was, it was all mainframes, the keyboard and mouse. Then you had this tremendous innovation in touchscreens and multi-touchscreens, direct manipulation, very akin to the mouse. Every single application you've ever used, you've used via a combination of direct manipulation. We don't have any of that because you're not going to walk around with those things. You're not going to have touch controllers with you as you go about your day. So what are we using? We're using hands, which can sometimes be modeled into direct manipulation with a little

• 15:23Speaker 01

  laser pointers like we do in mixed reality. But ideally, you're using voice, which is not always appropriate, and eye tracking and EMG, which is, again, approximating what you had with direct manipulation with your hands. And so for me at least, like, when you change the interaction paradigm, that is a half of what an operating system is. It's like, how do you, the consumer, affect the machine? The other half is, how does the machine bring information back to you? And here, we're more similar. You've got a display, you've got audio, except for the fact that the display is additive light. So there's light coming through from the world around you. So the display has to be somewhat aware of what is happening in the world around you, ideally, to be effective. And so anytime you're building a new interaction paradigm, especially when pairing with novel

• 16:07Speaker 01

  display technology, you're basically building a new platform. There's not really another way to do it.

• 16:12Alex Kantrowitz?

  And so then, how do you think about, like, who else might build this? You were just speaking with Ben Thompson. You talked about, I don't know if you were saying the potential for Apple to build something like this, but you talked about, I'm just going to quote from you, you look at what are Orion glasses, they're full AR glasses, you have custom silicon in the glasses, custom silicon in the puck, but Apple could build all this and just be like, oh, it only works with us. So does it fully insulate yourself from the platform risk?

• 16:39Speaker 01

  No, nothing fully insulates us, especially in a world where mobile is the dominant platform.

• 16:43Alex Kantrowitz?

  Do you expect them to build something like this as well?

• 16:45Speaker 01

  I have no idea. You know, they keep it pretty tight down in Cupertino. They're a pretty tight ship, and so I don't have great information on that. I think they would love to build something like this. I think we've been investing longer. I think we've believed in it more. And as a consequence of frankly Mark Zuckerberg's faith and tremendous execution by a team of thousands of people over a long period of years, I think we have a meaningful lead in a bunch of the technologies here. But you'd be foolish to count out a great technology company, one of the great all-time companies in general at Apple. But my point is, I think the fact that they already have this phone that's in your pocket has some advantages because you've now got this device that has a bunch of compute, it has a bunch of silicon.

• 17:28Speaker 01

  They can do great custom silicon, they can add more to that. It's also a disadvantage, I think, in a way. Are they going to be willing to walk away from, you know, one tremendously outsized profit center in pursuit of something that is fundamentally disruptive to that profit center? And if you're centering more of the experiences on the phone as opposed to on the glasses, do you make the leap that you need to make in developer platform, in interaction design? Because a lot of those things that we've discovered, if you really were to center them on just the phone and the way phone apps want to work, they don't work as well. So I think there's like, it's going to be a dynamic landscape for a while. And I think we take some risk sharing this with the world early to demonstrate what's possible, but we do want to ignite the passion of the developer community, the engineering

• 18:12Speaker 01

  community around us. And I think it's also something that we kind of owe to people so they can see what this 10-year journey we've been on in Reality Labs is all about.

• 18:20Alex Kantrowitz?

  Okay, so let's talk a little bit about the reason to have an app. It's just the normal way that you use a phone. But you're talking about these glasses, they're sensing the world, you know, you now have AI voice that you can speak with. I think Mark yesterday said, or we're talking a week before this is coming out, but he said, I think that voice is going to be a more natural way of interacting with AI than text, which is basically interacting with computing if you think that's the way this is all moving. Do we need apps anymore?

• 18:50Speaker 01

  It's a great question. I love this. And actually, it goes back, you know, applications didn't always exist, right? In the beginning, it was just like, the system was the software it was running at the moment, and you would kind of program it. And I think often about JCR Lick lighter, sitting down for the first time at a terminal, which is reprogramming itself on the fly. It's really one of the first times in history that people are doing human in the loop computing, as opposed to what would have before that been through mainframes and time shares. Like, you put your job in, you come back later, get the results to your job. If it was wrong, you change your software and try again. This was human in the loop computing, and it was this tremendous watershed moment that ultimately led through, you know, his work at DARPA, the IPTO, with Doug Engelbart, with Xerox PARC, to the Alto.

• 19:33Speaker 01

  And that's where you start to see, oh, there's like a windowing, SRI, like there's a windowing model, and each application is contained. And the application was born as a consequence of us being in a human loop at the computing. And what's crazy to me is the analog that I draw to ChatGPT, in fact, which was this human in the loop AI. Whereas before this, you would like create an AI, and it would go do a job and come back. And if the job wasn't right, you'd like reprogram your AI. Whereas this is like, no, no, like, I can just modify my queries on the fly, and the AI is like adapting to it. And so one does wonder, like, does this get into a more, if not an app-free world, a more web-like world again, where things are deeply linked and interconnected through services to one another?

• 20:15Speaker 01

  I think it's one of the challenges that the app model is going to face, like, things like Apple Intelligence, for example, will face on the phone, is what I really want is for something that actually intermediates the apps, and can go to my Amazon app and grab my purchase history, and then go to, you know, a different app and drop that in there and add that to my spreadsheet for my budget, which is in this third app that I have. And that is like, that is not something that Apple wants some other developer to be able to do, because that would disintermediate their position, their privileged position on the phone. And so like, if the AI can't work across all the breadth of my applications, well, it's not the most helpful AI it could be. One thing that's really fun about the desktop and the web still is that it can still work that way.

• 20:59Speaker 01

  And I love those platforms for that, including OSX. And so I think often about, okay, if I, once I have this AI, I want it to work at basically user level. Like, the AI is actually my agent in the machine. I think you probably end up still having applications. There's a lot of really valuable constructs we've built up as an industry over time that have to do with the safety and security of data and the model of how these things interact with both the computer itself, with the network and with other users. So there's actually still quite a bit of value in having these apps and not to mention just how literally would we do it and have these systems not constantly walk on each other. So I think the application model doesn't go away, but I do think you end up injecting this potential new layer on top of it, which is an agentic layer.

• 21:40Alex Kantrowitz?

  So how do developers then think about building for this type of technology?

• 21:44Speaker 01

  It becomes much more about what APIs you're providing and the unique value that you provide through your, the software below.

• 21:51Alex Kantrowitz?

  And if it's about the API you're providing, does that mean the main interaction layer is going to be through some sort of voice AI or?

• 21:56Speaker 01

  It's hard to say at this point. I think that's, voice is one, I think voice has a great modality for some things. It also has a lot of vagueness to it, even amongst humans, a lot of context that's required even among humans that obviously even a really great agent wouldn't have for a long time if it was with you. And so I think voice is probably not the only piece of it. Voice and text are part of it. I'm sure there will still be a lot of very direct manipulation interfaces for a long time to come. But you can start to imagine, yes, as voice gets more developed, my ability as a developer to create value in how I manage processed data or how I manufacture new artifacts is actually the more important thing I'm doing rather than having to also extend that to the consumer through some kind of interface.

• 22:37Alex Kantrowitz?

  So in this world, do the AR glasses replace the phone or do they supersede it in some way? I can't imagine it goes away.

• 22:45Speaker 01

  Yeah, it's a good question. We think about this a lot. I don't, it doesn't go away for a long time, if it goes away at all. There is a real strong argument that when the AI glasses become super prevalent and that is your interface to a device and you have, especially if you have a display, whether it be on AR glasses or it be on your phone or your watch or whatever the thing is, that you can kind of, when you need to directly manipulate something or provide very precise entry, you can. You don't need a phone anymore. Like it becomes, it becomes what the phone is today. We're a long ways from that for a bunch of reasons, not the least of which being the technology's not ready yet, but also because the application ecosystems will take a long time to transfer and user behaviors take a long time to change. So I think we have a long timeline there.

• 23:29Speaker 01

  But like, I do think it's conceivable that for a lot of, at least, you know, high-end capabilities and consumers, that AR will replace the phone.

• 23:39Alex Kantrowitz?

  So what does like computing look like in that world?

• 23:42Speaker 01

  Uh, hopefully it's a lot more fluid. Um, hopefully it's a lot more self-aware. I mean, computing today is like, it's hilariously dumb, actually. Like, it's like, we all like, computers are amazing at what they can do. And we're impressed because like, they can do math really fast. Um, but like, if I like, type in my phone that I wanna watch InstaGrab, it like, it's like, sorry, there's no app called InstaGrab. It's like, I meant Instagram. It's like, I didn't mean InstaGrab. It's just a typo. But like, the machine's like, no, I'm sorry. You clearly wrote InstaGrab here, and I therefore will not get you the app that you want. Um, like, computers are dumb in that way. Um, they have no construct of what my intentionality is. Children, human children, can intuit the intentionality of adults, like, very near infancy.

• 24:24Speaker 01

  It's like, incre- like, um, you know, I was just talking to someone recently who told me about these studies that they do with, um, human toddlers, or actually, I think even, uh, maybe infants, who are watching their mother interact with somebody, and if the interaction is very kind, they start to build a model of the person the mother's interacting with. And if the interaction looks transactional, then they just ignore the other person exists at all. Like, there's like, early on, like, we're like, okay, there's some intentionality in the world that's happening, theory of mind of other people that exist, and like, what are they trying to accomplish? And what am I trying to accomplish? Computers have none of that. And it's the fundamental way that we, as people, interact with one another. Um, we are, we understand what each other is trying to accomplish, and we then react accordingly.

• 25:04Speaker 01

  Um, and so, I think that's the thing that, the thing to look forward to the most about the future of computing is, if you have a device that is, instead of being throttled on the bandwidth of how much information you can give it, it's actually throttled in the opposite way. It has total context on you, uh, what you've been doing, what you're trying to do right now, what's possible, what's, what tools are available to you. Um, and then has enough, um, intelligence to do something useful to that, to give you either visually through a display, or audibly to your ears, or through haptics, the hints that you need to then just go be successful at what you're trying to accomplish. It starts to feel, instead of, hey, I'm a human with a great tool,

• 25:49Speaker 01

  it starts to feel like superpowers. It starts to feel like, oh, like, I just have tremendous more capability than I used to.

• 25:56Alex Kantrowitz?

  Okay, I have some follow up to that, but I'm going to ask them on the other side of this break where I'm going to get a little weird. So we'll be back right after this. This episode is brought to you by Avpoint. Everyone's racing to roll out AI right now. Copilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data and most teams have no single way to see it, secure it, and prove it's under control. That's exactly what Avpoint does. For 25 years, they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate, your data, your cloud, and the agents acting on your behalf.

• 26:38Alex Kantrowitz?

  It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. Avpoint, the unifying trust layer for AI. Head to avpoint.com to see how enterprises deploy AI with confidence. Learn more at avpt.co/bigtechnologypodcast. That's avpt.co/bigtechnologypodcast. Violent threats against executives are growing at an alarming rate. When a company experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door. The team at Ironwell knows this better than anyone. They've protected some of the

• 27:21Alex Kantrowitz?

  most targeted executives and individuals on the planet for almost two decades. As a white glove enterprise security service, they protect your people with continuous personal data removal, proactive prevention tools, and human-led emergency support. So when someone goes looking for your executives, Ironwell ensures they hit a dead end. So here's what to do. Go to ironwall.com/bigtechnology, fill in the quick form, and request your free risk assessment. Their team will show you just how exposed your executives are and how to lock it down before a threat reaches their front door. That's ironwall.com/bigtechnology. Learn how to stop online threats before they become real world attacks. The link's in the show notes.

• 28:04Alex Kantrowitz?

  And we're back on Big Technology Podcast here with Boz, Andrew Bozworth, the Chief Technology Officer of Meta. Boz, great to see you. So I promised some weirdness before the break and let's do it. Is this, you talked about like humans and computing getting closer and being more context aware of each other. Is this AR move just a step toward where we eventually go, which is that we just eventually insert computing into our brains through brain-computer interfaces?

• 28:31Speaker 01

  It's a great question.

• 28:32Alex Kantrowitz?

  And you already have the neural control for the device.

• 28:34Speaker 01

  That's right. The neural interface on the wrist, which is already very successful. And we think that's just the beginning. We think we can do a lot more with that over time.

• 28:43Alex Kantrowitz?

  Right. And for listeners, so like you have this wristband around your wrist and you're pinching and it selects, you tap your middle finger, it goes home, you can scroll by just like putting your thumb and pushing it on your hand. Like it's nuts. But sorry, go ahead.

• 28:55Speaker 01

  And it's just the beginning. Over time, we think we can do that without having any gestures at all.

• 28:58Alex Kantrowitz?

  Just think it.

• 28:59Speaker 01

  Just having the intention, expressing the intention down your control system. So a couple of things, you know, eight years ago, we did an invasive neural implant in partnership with UCSF. And it was a patient who was non-verbal due to a traumatic brain injury. And we were able to provide them a 400-word lexicon by virtue of this implant that we gave them. And the reason we did that study was then when the, when the person was using the control system, we put super sensitive EMG sensors on their head to see if we could detect any pattern of the underlying neural activity without having to be invasive.

• 29:40Speaker 01

  We couldn't. It was actually your skull is an incredible insulator of electrical activity and it's kind of scrambled together. The challenge with neural implants, we've actually been doing this for even longer than that. You know, the first people to be able to control a computer mouse through a neural implant goes back into the 20, early 2010s, if not earlier than that. There's a couple of challenges, one of which is that the brain myelinates the things that are put there. So they become electrically inert over time. So they have to be removed and replaced. And if you leave them in long enough, they can prove fatal or at least have very severe complications. So you wouldn't be talking about with the technology that we have today, with the materials we have today and what we know about how the brain works. You're not just talking about a one time brain surgery, you're talking about annual

• 30:22Speaker 01

  brain surgery and annual brain surgery for me is like a nonstarter as a consumer product. I'm just going to go ahead and assume that's not going to be the way things are going to go.

• 30:29Alex Kantrowitz?

  I guess I'd rather wear the glasses than that.

• 30:31Speaker 01

  Yeah, right. I think I think the glasses are a popular option in the relative to that. And so we were tracking that space carefully, certainly, and thrilled to see people investing a lot to try to see if they can crack those problems. But one of the things that we keep finding is you don't necessarily need to go to that point. And the glasses are pretty spectacular and like it turns out, once you get to all the wearability with their lighter and they have full battery life, the ability to take them off is actually pretty nice and the ability to just like have, you know, to kind of be restored to that center. Does it get to implants at some point? Yeah, maybe we, I think that's probably probably like, I don't think it happens in my lifetime.

• 31:06Alex Kantrowitz?

  Really?

• 31:07Speaker 01

  It's like that feels like even distant by my standards. I think we like have in, and maybe AI is a key to unlocking this more. I mean, I studied, one of the things I studied as an undergraduate was a lot of neurobiology was kind of part of my interest in in AI and in computation. And like the the challenges that we have understanding the brain feel like an order of magnitude bigger in terms of just complexity than anything we've managed to actually map out so far.

• 31:36Alex Kantrowitz?

  So this idea that we're going to merge with AI, you're not buying it?

• 31:40Speaker 01

  No, I don't see that happening in the near future. No, I I'm not saying.

• 31:44Alex Kantrowitz?

  Would you do it if you could?

• 31:46Speaker 01

  We are getting weird. I appreciate you. Good as delivered.

• 31:49Alex Kantrowitz?

  I promised.

• 31:50Speaker 01

  I think, what I think, I really like, there's a book, Hofstadter's book, I Am A Strange Loop, which has this idea of consciousness as an emergent property of sufficiently self referential systems. And it's a beautiful book and a rumination on consciousness from, you know, one of the best to ever do it. And it also is kind of a beautiful hand to his wife, his late wife. And he has this idea that his wife, who had passed away unexpectedly. There's like a version of her consciousness running in his brain, right? Like it's been uploaded to his brain because he can imagine with high fidelity. And this gets back to what I said earlier about, you know, the idea of understanding someone else's intentionality and agency.

• 32:33Speaker 01

  He has like a pretty reasonable approximation of her consciousness in his consciousness.

• 32:37Alex Kantrowitz?

  It's it's true. Like if you've if you're close enough with someone, you can effectively like speak with them if they're not with you.

• 32:44Speaker 01

  Absolutely. And we have a version of their consciousness running through it. So I think if you got to the point where an AI was ever present in your life and helping you, guiding you, you're confiding in it. Maybe even your innermost thoughts that you wouldn't normally even share with a close friend or someone else. That it would have in its systems, a version of your consciousness, a queryable, like interactable, animated version of your consciousness. Is it you? That's a I'll leave that to the philosophers. But it would it would pass as you in many contexts. And and obviously, there's a lot of questions that raises in terms of privacy and security and how you control these things.

• 33:27Speaker 01

  But assuming we can conquer all those things, yeah, I'd be thrilled. I'd be thrilled for my kids if something were to happen to me to have access to, even if it's a simulacrum, a construct that approximated what I was for their use. And I'm sure there's going to be a whole new set of challenges that that introduces. But I think that's a version of the future that is likely.

• 33:50Alex Kantrowitz?

  Do you think that the current systems are conscious or approximate consciousness?

• 33:55Speaker 01

  No, I don't.

• 33:56Alex Kantrowitz?

  They're not getting close to it?

• 33:57Speaker 01

  No, I don't think so. I think it's a category error, to be honest with you. And I'm a huge I find this is a funny thing that happens with these AI conversations because I am both one of the most fanatic believers in the power of large language models. I talked about how I think it I think of it in the fundamental as a fundamental computing paradigm shift of going from human in the loop computing to human in the loop AI, where the AI is much more informed by modern sensors and availability of information. So I'm a tremendous believer in it all. And also, I do not believe it is even the kind of thing from which Consciousness exists. It doesn't exist in any, again, to my point about Doug Hofstetter. It doesn't exist in any self referential loop. In fact, it doesn't exist at all, except when I instantiate it to exist.

• 34:41Speaker 01

  It's a very, very cool associative database. It reminds me, actually, when I was an undergraduate, the reason I studied this computational neurobiology certificate that I got was this idea of what could, how could I, with a computer, emulate human associative databases. Human memory is this wonderful associative tool where when I say the word purple to you, this huge network of ideas pops into your brain. And it's like very rich and complicated and it's very personal to you and what your history of that, with that language is. And it's a really hard problem for computers to do that. Like it's to have that kind of loose and generative associations.

• 35:22Speaker 01

  And so I can say to AI today, I can say to it, like, imagine a purple elephant and it can do it. And that was kind of wild before, there was no way to do it before. So I love it, but I do see it as fundamentally an associative database. And it's an important, maybe a component of a total system. But I tend to think Jan LeCoon's critiques of it are spot on.

• 35:43Alex Kantrowitz?

  Are spot on. Okay, so let's rapid fire through the rest of my questions. We have a few minutes left.

• 35:48Speaker 01

  Let's do it.

• 35:49Alex Kantrowitz?

  What happens to VR or these mixed reality devices, the Quest and the Ray-Ban Metas? If Orion takes off?

• 35:55Speaker 01

  The analogy I make is to your phone and your laptop. I mean, the truth is, there's a, and this is what we found so much. And the reason they have different operating systems, though we have tons of shared technology layers and sensors and hand tracking and eye tracking and voice and AI, shared technology layers, different operating systems is because the interaction design is different. And the reason that's different is because the use cases are different. Just like I tell the story all the time internally, but most people who are in information work understand it. Sometimes I'm on my phone and responding to an email. My phone does email, fine. And I'm like, mm, not a phone email. I take my phone down, get my laptop, this is a laptop email. I need to have five browser windows open. I need to have, like, I need to, this is a real, I gotta write this thing. And so I do think of these things as like fundamentally different pieces. I certainly watch quite a bit of short form video on my phone.

• 36:38Speaker 01

  I also get the big screen at home with my wife and kids and watch a movie. And it's just like, they're different things. And I can't describe to you in human terms why they're different, but you and I both know that they are different and they feel different. And so I think there continue to be, even when you have these wonderful AR glasses, things that can be done in virtual and mixed reality that can be done no other way. Your augmented reality glasses will never convince you that gravity is off because you can literally see the physical world behind them. When you're in the headset, you can absolutely play as if gravity doesn't exist. And so I think they both serve tremendous value, and I think of it as being a phone and a laptop.

• 37:15Alex Kantrowitz?

  Again, speaking with developers, they have lots of options. Build on the phone, build for these devices that are starting to get a user base, but aren't anywhere near as big. It's tough to reach scale. What's the one-minute pitch to them if they're thinking about building for these devices, these VR mixed reality or AR devices? Or even AI voice?

• 37:41Speaker 01

  For sure. The market's super efficient, and so, yep, there's a lot of people on the phone, but it's also the hardest place to get noticed or stand out. It's the hardest place to do new things because there's nothing new that you can do there. It's all been done before. Phones haven't changed in 10 years. Whereas these new devices are risky because the audiences are smaller, but you have an enthusiastic audience who's deeply engaged and is absolutely desperate to see new things, to try new experiences. So if you have a great idea, it feels like a goldmine. Walking around Connect the last couple of days has been so energizing because I'm surrounded by people who have thought of ideas that we never considered when we build these platforms, and they're just powering ahead and pioneering those things. So I can't listen. Every developer should probably play to their strengths

• 38:25Speaker 01

  and understand there's these devices and AI in particular all live in very different contexts.

• 38:32Speaker 01

  But it's going to be hard to break out in the phone, and it's designed to make it hard for you.

• 38:38Alex Kantrowitz?

  Yeah, because I mean, without the applications from developers, it's going to be tough to find like great value in these devices.

• 38:45Speaker 01

  That's why every time we talk about Quest, we talk about the ecosystem.

• 38:47Alex Kantrowitz?

  Right.

• 38:47Speaker 01

  It's not just the devices. I'm not here to sell devices. I'm here to build an ecosystem, and that's about getting more developers to build great experiences to get more consumers, and more consumers which make it attractive for more developers.

• 38:57Alex Kantrowitz?

  Okay. Couple AI questions before we end. We have a debate often on the show. Is it going to be the development of the foundational models? Or is it going to be the application of them where the real value is found in artificial intelligence? Obviously, you guys are open sourcing.

• 39:12Speaker 01

  Yeah.

• 39:13Alex Kantrowitz?

  The foundational model.

• 39:14Speaker 01

  You're going to tip your hand on that answer.

• 39:15Alex Kantrowitz?

  You have an answer. So talk a little bit about, do you see promising applications being built? And what happens to the big foundational model companies if this stuff is all effectively commoditized?

• 39:27Speaker 01

  Yeah, so I was one of the loudest voices internally to channel encouraging us to open source Llama one, Llama two. Obviously, you see how bought in Mark is with his announcements around Llama three. And the reason I feel that for me is like, it's a tremendous benefit to our company. Whether we make the AI or someone else makes the AI good, I'm the only one who can use it in our products. And that's we have a direct relationship with our consumers. So we can make our products better for consumers, whether we make the AI or someone else makes the AI. So that makes this a compliment to us. And you want to always commoditize your compliments. You always want to make sure that the thing that you depend on is like, cheap and getting better by a lot of people, not just by your effort and money. And so strategically, it makes perfect sense. But more than that, we're a technology company, and we've always seen AI

• 40:12Speaker 01

  in particular, as the data we bring to it for our own applications, that's special to us. And the application itself is special to us. But we've always with PyTorch, we've always tried to make sure that the middle piece, that the the infrastructural pieces, those are for everybody. And that like, that makes our developers have better apps when they're on building for Quest. It makes sure that the partners that we're working with, you know, for Instagram are having a better experience. It's really good for us to have the entire consumer base, whether they're on our app or somebody else's, have a better experience. I think for us, though, to be honest with you, it's hard to say what happens to these companies. I would have been wrong about AWS if you'd asked me this question a long time ago, because we were building a

• 40:57Speaker 01

  Lamp stack, we were doing it all ourselves, and it was everyone was doing it, it was fine. And then it turns out, if you can abstract away at good enough economics, the complexity of managing a bunch of web servers and load balancing and scaling up and scaling down, there's a business there. So I think there probably will be a business there for some of these AI hyperscalers. It's probably, you know, potentially a big, AWS is a big business. But they've got to get to that point where the economics makes sense, and it's sufficiently well abstracted from from them. I think the bulk of the value, though, so there will be valuable companies there, but the bulk of the value will be in the application side.

• 41:30Alex Kantrowitz?

  So I was driving down 101 today, which I've been doing for, I was thinking about it nine years, coming down to visit you guys. It's an interesting moment, right? We are ahead of November 2024. We're not talking about politics. I was also like thinking about the different types of technology that I saw when I came down here. And I mean, you know, credit to your teams, but it was like stickers and Messenger and 360 video.

• 41:53Speaker 01

  Sure.

• 41:54Alex Kantrowitz?

  What does this technology moment look like to you? Because it, I mean, seriously, we're talking about artificial intelligence that can effectively get to know us, glasses, I'm looking through earlier glasses that you can like see through.

• 42:06Speaker 01

  That's right.

• 42:07Alex Kantrowitz?

  And

• 42:07Speaker 01

  Perfectly clear. Those are clear. That's right.

• 42:10Alex Kantrowitz?

  No, I'm like thinking about the lens.

• 42:11Speaker 01

  Yeah, totally.

• 42:12Alex Kantrowitz?

  Just like, uh, it looks like a VR device pretty much, but I mean, from the inside. So talk a little bit about this moment. And do you think we're going to continue the momentum? Or the tech industry really will continue the momentum because it seems like progress is nuts right now?

• 42:27Speaker 01

  One of the things that we talk about all the time as technologists is these overlapping S-curves, where a new technology is created and then has this period where it struggles and then it gains adoption and then it explodes and it's everywhere. And then there's still a tremendous amount of value at the top, but it starts to like the rate of change starts to slow down. And so, you know, I'll take an easy one. You talked about Messenger stickers. And it's easy to make fun of Messenger stickers in retrospect, except that they're still super popular and we wouldn't trivialize emojis. Emojis, this idea of expressing yourself through visual form as opposed to expressing yourself explicitly through text, was at first a laughable construct that like our generation got made fun of a lot by the older generations. And then now is like ubiquitous among all generations and arguably helps

• 43:11Speaker 01

  us express emotions through a medium that otherwise lacks the ability to do so so cleanly. And so I don't like to trivialize any of the things that have come before, but I do think it's fair to recognize that 10 years ago, we all felt like we were at the top of an S-curve of really what was the kind of tail end of the web. We're riding the mobile curve straight up the ramp at that point. So the web is kind of starting to trail off. Mobile is like still going gangbusters. And it's a tremendous, you know, foresight from Mark to think, what's the one behind that? Like, what's, what's the one?

• 43:41Alex Kantrowitz?

  It was crazy. When I was here, I was trying out your AI assistant in Messenger called M. That was actually contractors in the WhatsApp building that were approximating what it would look like.

• 43:50Speaker 01

  And we were all getting this wrong as an industry. Two years ago, a year ago, you know, I remember when we had Portal, still one of our most beloved devices, and I'm sorry to all of those for whom we had to shut that down.

• 44:00Alex Kantrowitz?

  Yeah, look into the camera until Katie Natopoulos.

• 44:02Speaker 01

  Katie, Katie, you know how much I apologize. It was, and our, my, knowing more than my, our own employees, our employees were devastating me to that tremendously popular product, small market, but tremendously popular product. And we had our assistant on that and we had an Alexa assistant as well. And when we got into building our own assistant, I could not believe the, how poorly they scale, because you have to build it for every language, for every device, and for every use case. Oh my gosh, that's not software. That's, that's, that's the worst scaling laws of all time. And so even back then, I knew this isn't it. Like, we didn't do it. And it wasn't until you get self-supervised learning confined with the breakthroughs that

• 44:43Speaker 01

  Google led in, in how they, they built these, these models, and then ultimately ChatGPT, that you see it, you're like, ah, that's what it is. Like, that's the thing. That's the one that we needed all along, and we didn't have it yet. You know what they say in the, in the industry, though, there's no bad ideas, just bad timing. And, and was, was probably one of those.

• 45:00Alex Kantrowitz?

  Okay. Andrew Bosworth, great to see you. Thanks for having me in.

• 45:03Speaker 01

  Thanks, man.

• 45:04Alex Kantrowitz?

  Thanks, everybody, for listening, and we'll see you next time on Big Technology Podcast.

• 45:08Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com, not authorized by any candidate or candidate's committee.

• 45:40Speaker 03

  Every mile matters. Whether you're driving your dream car or your daily commuter, the Mobile One team knows that. It's why we created Mobile One Advanced Clean Motor Oil. It cleans your engine in one oil change and helps extend engine life, delivering performance, protection, and cleanliness for vehicles of all ages. All to help keep your engine running like new. Whatever you're driving, visit mobileone.com/advancedclean to learn more.