Why Meta Wants To Build Artificial General Intelligence — With Joelle Pineau

Joelle Pineau is the head of Meta's AI Research division. She joins Big Technology Podcast to discuss the company's recent proclamation that it intends to build Artificial General Intelligence, digging into how we get there and why it feels it must. In a wide-ranging discussion, we cover the latest research trends, the company's open-source practices, what actual products developers have been built with AI, video generation, and the reasons why NVIDIA chips are so in demand. Tune in for a deep dive into the world's most crucial technology from someone directing one of its most important research labs. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/ Questions? Feedback? Write to: [email protected] Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  The head of Meta's AI Research Division joins us today to discuss the company's pursuit of human-level artificial intelligence, the cutting edge of AI, why it's open sourcing, its large language models, and plenty more in the only podcast interview the company is giving about its recent news. All that and more, coming up right after this. This episode is brought to you by Genesis. What does it actually take to put a Gentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with

• 0:41Alex Kantrowitz?

  Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get front to your computer. How do you know where to look, who to trust, or how much to pay? Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 1:25Alex Kantrowitz?

  Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. Boy, do we have a show for you today. We're recording to the minute on Wednesday here, right before we drop this episode, because there's breaking news coming out of Meta, all about the moves that they're making with their AI division, their pursuit of human-level intelligence. We have none other than Joel Pino here to talk to us about it. She's the head of Meta's AI Research Division, formerly called FAIR, now I guess it's called Mare.

• 1:54Speaker 01

  Still FAIR, Fundamental AI Research. We love the name.

• 1:59Alex Kantrowitz?

  Okay, we'll keep it fair, keep it running. We spoke actually in October 2022 before ChatGPT, so this is going to be a really cool moment to talk a little bit about where we've come from there and where we're going. Joel, welcome to the show. Great to see you.

• 2:13Speaker 01

  Thank you, Alex. Great to be here.

• 2:16Alex Kantrowitz?

  So if you recall, in October 2022, and we spoke a couple of times at the World Summit AI.

• 2:24Alex Kantrowitz?

  One of the, it's kind of funny because like the big storyline then was whether AI is sentient. And this was kind of a moment where like all the big research houses had big, large language model chatbots internal and they hadn't released it yet. And it's kind of interesting how society starts to talk about a breakthrough, right? It sometimes goes in a weird direction before we're actually refocused on what matters. And now I think we are refocused on what matters, right? There's been much more talk beyond sentience in terms of like the near term viability of this technology. I'm curious just to start, what has surprised you in the research since that discussion? Not necessarily about, okay, we all know that it's now taken off and it's been hyped,

• 3:06Alex Kantrowitz?

  but has there been anything that's made you sit back and be like, wow, we can actually do more than we thought we could, you know, a year or a year and a half ago?

• 3:15Speaker 01

  So many times this year, honestly, and it's great to think back to that, to that point in time. I hope you didn't ask me for any very specific predictions, even for someone who's deeply in the space of AI, just predicting how this is unfolding continues to be full of surprises. I will say, you know, it's also been interesting, it's the faster we progress, the more we have a sense of how much more is left to do. And so though, you know, back you mentioned back in October 2022, we were worried about sentience and we don't, we hardly talk about it now. And yet we are so much further along on the map in terms of our ability to have models

• 3:59Speaker 01

  that deeply understand information and process multimodal data. So we're getting further along and yet we worry about some of the more, more concrete problems. We've talked a lot this year about safety, for example, about how to make sure that we have models that are performing well, but also are aligning with the values of peoples and the needs of people, which I consider sort of a much more grounded problem that we can tackle with research. So that's, I think, the major change that I see. Significant progress, but that means we also have a much better view of what are the real problems we need to solve.

• 4:39Alex Kantrowitz?

  Yeah, it's funny because back then we also had a discussion about whether we should be focusing on like the short term or the long term problems. And obviously, those are both worthy of attention. And it's kind of wild that the focus on the long term problems, it seems like blew up OpenAI over a weekend and maybe it's been put back together now. But the talk from Meta now is actually focused on some of the more big ideas that people might have thought were more long term. But now it actually seems like, you know, it might be closer than we think, at least according to some of what we hear from OpenAI and others. So this is a quote from Mark Zuckerberg that just came out fairly recently. He says, as recently as last week, "We've come to view

• 5:21Alex Kantrowitz?

  that in order to build the products that we want to build, we need to build for general intelligence." So I mean, Yann LeCun, in our discussions, I've been speaking with him since 2015, one of your colleagues, he's always talked about how the goal is building for artificial general intelligence. So when I saw Mark come out with that last week, I was like, yeah, yeah, that's been the focus for Meta. But all of a sudden, it almost feels like there's a more pragmatic or it feels more real now than it did before. Am I reading that right? Like, what is leading us to now start to like, talk about this as something that's not pie in the sky, you know, 20, 30 years down the road, but something that might be achievable in the, you know, nearer term?

• 6:01Speaker 01

  Yeah, I mean, Yann and I and the team at FAIR have been talking in those terms for many years. It's been clear we've been putting in place sort of a portfolio of projects that are trying to build the building blocks towards general intelligence. And in the last year, Mark, as well as many others, has taken a deeper interest in what's going on in AI. I think he was always aware of a lot of the good work we were doing, but didn't dig in quite as deeply and in the last year definitely has. And through a lot of conversations, you know, I think has come to To see how in many ways, the the path even to bringing AI to to the products that that people use and and love from the company, the path to making those AI systems better goes by through

• 6:46Speaker 01

  building general intelligence, not narrow intelligence. And and we've done a ton of work on AI on the platform over the last few years. That was what I would call more narrow specialized models. And we can continue to do that, but the the bigger step change are going to come through the more general model, building foundation models, building up to world models that essentially can capture a much richer version of the information. And so I think that's that's what you're hearing from Mark. It's it's things that you've been hearing from from Jan, myself and others through the years. We're we're working together to to connect these pieces together, and both the the research roadmap as well as the the

• 7:26Speaker 01

  product roadmap and and make sure that we have the ability to to connect these together. So so the ability to have our research quickly diffuse in the best way possible through the product, and the ability to learn. The the thing about general intelligence is you have to solve many different problems to have, you know, the ability to claim general intelligence. And and fortunately, there are a lot of use cases across Meta, across our family of products. And so that's giving us wonderful material with which to work.

• 7:57Alex Kantrowitz?

  So why, so let's go back to, you know, that October 2022 discussion that we had before ChatGPT come out. Like, the idea of me asking you this question about, like, why is human level intelligence now in focus? I never would have asked it. It just didn't seem like it would be something that would be relevant to ask. But now it does seem more relevant, and we're hearing it more and more in the discussion. So you mentioned world models, foundational models, but what about AI research now is allowing us to ask those questions?

• 8:28Speaker 01

  I think it's because, you know, the the models are are getting increasingly general. If you look at a model like ChatGPT, the Llama family of models that we've been releasing, you know, they started just as word prediction models. All they would do is take in sentences and predict what comes next. And what we're seeing is we can use them now through through many other uses, whether it's to predict things that are not just words, but are actually code. And some of that code is actually executable. Or you can predict, you know, the components of an image. And then you can plug in a diffusion model or or other kind of synthesizer to to realize the information.

• 9:10Speaker 01

  So what started as just language model has become much more general on its own. It gives us a path. It may not be the path, but it gives us at least a path to move towards general intelligence. And it's an exciting one. It's one that that we're exploring. It doesn't mean that we've stopped exploring other paths towards general intelligence, but but that is definitely the one that has has proven to to make the fastest progress.

• 9:37Alex Kantrowitz?

  And what would you do? What would you say the path is?

• 9:40Speaker 01

  The path is to essentially capture a lot of human information through this this representation that we call language. And so the the hypothesis that, you know, even things that are not necessarily text based originally, if you describe them through these discrete tokens, and sometimes these discrete tokens are the words that we use to express. But sometimes discrete tokens are, for example, code, numbers, essentially like chunks of images. And these discrete tokens are a path to representing all of human information.

• 10:17Alex Kantrowitz?

  There was a study that came out last year, basically saying that these models can't generalize outside of their training set. And I think that was like a lot of the the hype around these models, where people saying that they were really able to to have these capabilities that you wouldn't expect, emergent capabilities. And the study basically pushed back on it and was like, listen, they're not going to generalize beyond their training set. And your evaluation of that? Study basically, you know, leads you to believe either A, if you believe that study, then you're a lot less optimistic about this wave. If you don't believe that study, you're, you can be, you can really use your imagination and believe that what we're hitting

• 11:00Alex Kantrowitz?

  on now, these foundational models that you talk about, can lead us in directions that we never could have dreamed of. So I'm curious what your evaluation of that study is and how we should be thinking about this.

• 11:11Speaker 01

  I tend to really like be quite balanced on a lot of these questions. I, I think it's very easy to kind of, you know, pull opinions to one side or another, but, but the truth is like, machine learning algorithms can generalize. That is a property of, of how we build these algorithms. Even the simplest, just linear models, they do linearize, they just linearize along a line. So, you know, the, the fact of the matter is though, when you project that into a very, very high dimension. So some of these models have hundreds of billions of parameters. You have to think of like you're, you're learning a function in that really high dimensional space. The directions in which you can generalize are so many that

• 11:54Speaker 01

  it's hard to know which are the good directions to generalize and which are the poor directions to generalize. The more data you have, the more that constrains that question. So I, I do believe they can generalize. I think they generalize relatively narrowly, or at least, you know, as long as you stay close, you get a good manifold of information. When you start to go really far afield from your data, because the dimensions are so large, you get all sorts of, all sorts of noise. So the advantage and one of the reasons, you know, a lot of the progress has been through better and bigger data sets, bigger but also cleaner data, is because that really defines which parts of this really high dimensional

• 12:38Speaker 01

  space are the most interesting one. And when there's not a lot of data to populate that space, then the models tend to regurgitate the things that they've been trained on.

• 12:48Alex Kantrowitz?

  So let's go back to that Zuckerberg quote that I read earlier. We've come to view that in order to build the products that we want to build, we need to build for general intelligence. Now, we talked a little bit about why that is now relevant, the path towards general intelligence. But now I'm kind of left with another question, which is, why does Meta need to build general intelligence in order to build the products that you want to build? I mean, yeah.

• 13:11Speaker 01

  I mean, just looking at like a couple of the AI products we've released this year. One of them is the Meta AI Assistant. People who are in the US have been able to try this out on some of our platforms, where you can essentially ask for questions and ask for assistance. In that case, you know, there's a sense that it has to understand a very large spectrum of information to be able to do well. And as we incorporate more data and as we perfect this assistant, the more it's going to have essentially world knowledge, the better it's going to be. Another example is for those who've been following our work on AR devices, the smart glasses that we released earlier this year

• 13:55Speaker 01

  also come now with an AI model, also accessible mostly in the US at this time. There, too, you know, you have essentially a more embodied version of this Meta AI Assistant that sees the world as you see it, that is able to take on some action. In this case, the actions are not just words. It can take pictures, it can provide information, it can record information. And so to be able to do well in a wide set of different tasks, with a wide set of different people, different environments, you need to have to move towards more general intelligence. That's really where that connects, you know, the research work we're doing and already what we're seeing in terms of the applications that Meta is putting out there.

• 14:42Alex Kantrowitz?

  Now, let's say you do achieve this and you open source it. Is that kind of like the the end? Like, is human, reaching human level intelligence or general intelligence kind of like the end of AI research? Or is there more to do after that happens?

• 14:56Speaker 01

  There is no end to this journey. I mean, I hope there's no end to this journey, right? Like, do we as adults sort of say, OK, I'm gonna keep on growing my knowledge? And at some point in time, I don't know, for some of us, maybe 25, some of us, maybe, you know, 75, you decide like, OK, now I'm done. Like, I have reached where I am in terms of human intelligence. I don't think that's how it works for humans. The world is always evolving. There is always more to be curious about. And so I think that's the path that we are on with our AI algorithms. Similarly, they need to stay curious about the world that they evolve in. And over time, they need to figure out, you know, how to integrate that that information

• 15:40Speaker 01

  and sort of rise to the challenge of the world that they're building. But because the environment is not static, I don't see us coming to an end.

• 15:49Alex Kantrowitz?

  That's so interesting, because it's always described as the finish line. And actually, there's people who would argue that there's no such thing as as human level intelligence, that the second you hit that, you're basically left with superintelligence and game over. But

• 16:03Speaker 01

  Yeah, I mean, I don't really ascribe to that that scenario, I have to say. And the other nuance I will add to this, you know, often this notion of general intelligence is articulated in in the context of like a single agent, a single uber intelligent agent. And I don't, I don't think that's really where we will move towards either. There's, there's clear evidence that that as a species, humans, animals, we learn so much more through interactions and so much of our culture and our intelligence is derived from our ability to interact, to collaborate.

• 16:44Speaker 01

  So I think that's also gonna be a super interesting door to open as we, as we are on this journey, to think about how do we build AI agents that are not just, you know, pushing for single entity intelligence, but are connected to a network of other intelligent agents, whether synthetic silicon agents or our human agents.

• 17:06Alex Kantrowitz?

  Well, it's so interesting because language, of course, like speaking of types of intelligence, language is only one type.

• 17:11Speaker 01

  Yes.

• 17:12Alex Kantrowitz?

  And Jan and I spoke about this on a recent show, not so recent anymore, but your interactions in the world teach you so much that you never learn with language. Your understanding of gravity, for instance, is not something that like you can implicitly understand from language. So are you doing research now to help Meta's research division to figure out stuff beyond words and images?

• 17:35Speaker 01

  Absolutely. And I would say, you know, that may be one of the distinguishing factors compared to other research group out there. There's a strong belief that having AI agents that are deployed in the physical world, where the notion of embodiment is important, is something we should be pursuing. We have a research team that's dedicated to this. They do some work in robotics in particular because that's the best agents we have to consider physical embodiment, spatial constraints. It's not necessarily because Meta intends to commercialize robots. It's because by going through these essentially devices, we have a lot to learn about how to build AI models that live in the physical world.

• 18:17Speaker 01

  In the work that we've done recently with the smart glasses, the models that proved to be useful for that use case came out of the work that this group was doing. People who were looking at robotics and devices, physical devices, living into the world and building AI models specialized for that was incredibly useful to inform the work going into the glasses. Of course, we also leverage the work we were doing on language and our Llama family of models, but Llama on its own doesn't make for the best assistant on glasses because it doesn't have enough of an understanding of the physical world of images and so on.

• 18:56Alex Kantrowitz?

  Now, there are some people saying that the reason why Meta is now speaking about AGI is because OpenAI is speaking so much about AGI and other research houses are. And getting the talent to work on these projects is really difficult. This is something that Mark actually said in that Verge interview. I think that it's important to convey because a lot of the best researchers want to work on the more ambitious problems. So I got to ask you straight up, like, is the talk about AGI more of a recruiting thing?

• 19:30Speaker 01

  No, I mean, like, of course, we love to have great talent. And of course, this is a competitive market for talent. But but we don't talk about anything just because someone else talks about it. Like, we genuinely are doing the work and we've been doing it for a number of years. Um, it's there's no major shift in terms of our ambition to solve AI. That's been inscribed in our mission and our goals for FAIR for many years now. And Mark is talking about it now. I think he's excited about the work. It's wonderful to have have some support to do it. But it doesn't necessarily fundamentally change the the the problems that we have to solve, the work that we're doing. I think there's also a sense that we are, you know, we are being more explicitly ambitious

• 20:12Speaker 01

  about this work, which goes along with some of our investments on the compute side, uh, which are necessary to to fuel that work. And so that's why it's coming out, um, maybe more from what you're hearing from Mark. But I think if you go back and and listen to what Jan or I or some of our other senior researchers have been saying for for a number of years, there is not a departure there.

• 20:33Alex Kantrowitz?

  Right. And briefly on the talent market, what does the talent market look like right now? Is there a real scarcity in the type of people that can do this type of work? And what is it like recruiting against fast growing? And especially in terms of valuation competitors like OpenAI, Anthropic, et cetera?

• 20:52Speaker 01

  Yeah, it's always been a very competitive market. I would say going back to about, um, twenty sixteen, twenty seventeen. Um, since then, I I don't really remember a year where where it was like an easy slow market in AI. And so it continues to be. One of the things that has changed in the last year or so is mostly on the startup scene, I would say, you know, three years ago, we didn't feel much competition with the startup scene. Now we do a lot more. I tend to view this as relatively positively, to be honest with you. And that's one of the reasons we open source our work. We genuinely believe that more people working on this is good.

• 21:33Speaker 01

  And so when we open source our work, we get to leverage the creativity of a greater number of people. And there's many more than we can hire. And so I think that the very, very top talent that can train these models continues to be incredibly valuable to Meta as well as to other organizations. Fortunately, there's also a good pipeline of students. You know, I do have an affiliation with Mila, the Montreal Institute for Learning Algorithms. There are hundreds of amazing grad students coming out of that institute as well as others. We've set up some joint PhD programs in some cases so that these students have an opportunity to come work at least part-time or through internships with us. And so we're both, you know, sharing with them the work that we do as well as having

• 22:16Speaker 01

  an ability for us to see whether they're a good fit for our work. So I feel like we have a great talent pipeline, but it continues to be a competitive market.

• 22:27Alex Kantrowitz?

  Got to ask you about open source. Brad Smith from Microsoft has talked about how OpenAI is the most open. And I'm kind of curious, from your position, are they living up to that open name? Is there real open sourcing there? And what is the state of open sourcing? I mean, why is Meta open sourcing outside of like, I mean, from like a, you know, Meta's a business, so from a business perspective, why open source?

• 22:52Speaker 01

  Yeah. There's different levels of open sourcing, right? I do think, you know, having an AI model where you provide an API is sort of one layer of that, which is something that OpenAI has done. But there's a lot more that goes on. And so, you know, from just providing an API, you can make available the code that was used to train it. You can make available the trained model weights, which enables someone to run the models. And then there's a number of other artifacts that come across from this. We've been focusing on making available like model cars that give a better understanding and transparency about our models, good use guides, tools for safety and so on. So there's like a whole ecosystem of artifacts.

• 23:36Speaker 01

  I think the purists would say like, everything has to be, to be out there in an open way. So we even have some, some people who are coming from the software open source community who feel like we're not, you know, living up to the, the full view. Again, there's a continuum on this. It is clear that Meta has taken a much more open view than other big players in this space. And in particular, we've been releasing some of our code and model weights for some of our larger models, including Lama. It, it comes from a lot of, of deep discussions in, in doing that.

• 24:14Speaker 01

  And so I think there, there may be a misperception that, that where, you know, we would do this without, you know, the, without any process or, or reflection and it's a little bit of a religion. That's really not the case. We have quite a thoughtful process that's been put in place. You, you have to remember, we've been doing open sourcing work for 10 years, since the first day of this organization. So we've built up a lot of muscle of how to do that in a, in a responsible way. We do it in consultation with a wide set of people who have deep understanding of safety, ethics and, and so on, who get brought into the, into the process. And what's been wonderful to see in the last year is as the conversation has been moving

• 24:59Speaker 01

  and as the models have been going better and getting bigger, we've invested a lot more into being thoughtful about our release process. And so I would say now we have a much more mature process than we did a year and a half ago. That involves a much more diverse group of stakeholders. We have a really rigorous process in terms of measuring the risks of these models across different, different categories of risk. And so it's been exciting to see how much our commitment to open sourcing has driven us to innovate. And we've open sourced a lot of those innovations on the, on the safety side. I think the, the purple Lama tools is an example of that, which we released in December. And so. It's been great to see that. Um, I, I do hear a lot of people who are concerned about open sourcing.

• 25:44Speaker 01

  Um, I, and I have many conversations with them, including other, other large organizations. My, my worry about closing the doors down now is that the models are only getting better. And so if we don't release them now, we really miss an opportunity to develop the muscle we need to make these models safer. Um, and I don't think today's model are the ones that are gonna, you know, bring to the front the hardest questions. These models are yet to be trained and built.

• 26:18Alex Kantrowitz?

  Is there anything, is there anything stand out that you've seen being built on top of, uh, the open source Llama model that Meta has put out there? Anything stand out in terms of like a cool product that you've seen and anything concerning that you can talk about?

• 26:32Speaker 01

  Um, there's definitely been dozens of, of, of product that are, that are coming out of that.

• 26:37Alex Kantrowitz?

  How about naming, yeah, do you wanna name one?

• 26:39Speaker 01

  Yeah, let, let me take an example, our segment anything model, which is a little bit different than our, than our Llama model, um, but, but I think has been the one that has been just incredibly impactful in terms of people quickly building on it. Our, our segment anything model is one where you take an image and it gives you a detailed segmentation of that, uh, of that. We released it back in April, including a lot of, um, tools and data to, to go along with it. And, and within days, we had people who had built up applications essentially for, um, conservation applications. So being able to track down some species who may be endangered, using that to, to follow them. We had people use it for, uh, the treatment of medical images, so segmenting cells from

• 27:23Speaker 01

  some, some of these images. And it's been wonderful to see that, that explosion, that explosion of work. Um, on the language side, we also saw many people build up all sorts of, um, different tools. And in particularly, the, the work that we're most excited about is, uh, the work on efficiency, to be honest with you. Um, there is so much that we can do to make these model more compact and, and efficient and. And running really, really fast with low energy. And I think that's one of the things that I've been most excited about seeing. There's lots of other applications too.

• 27:58Alex Kantrowitz?

  Anything that stood out and made you say, oh, that's not good. That's not what we want.

• 28:02Speaker 01

  There are definitely some that are getting flagged that we discuss internally. I'm probably not going to go into the details of them right now, but there are definitely a number of them that we are tracking. I will say in a number of the cases that we are most concerned, people are not respecting the terms of use of these models. So we release these models with very clear terms of use and people may not be respecting those terms of use.

• 28:25Alex Kantrowitz?

  And do you have recourse once they disrespect those terms?

• 28:28Speaker 01

  Yeah, I think that's, I'm not going to go into the details of that today, but this is, you know, this is definitely part of the conversation. We are thoughtful about the conditions under which we release and so we are thoughtful about the follow through as well.

• 28:42Alex Kantrowitz?

  Before we go to Brick, I want to ask you about this move toward getting these models to reason. There was like this momentary freak out around this Q-Star model thing that OpenAI apparently has developed internally, which gets people to reason, gets the model to reason. What's your perspective on this technology moving towards like the ability to reason? And how should, how should we think about it when we think, when we see stories like the one about Q-Star?

• 29:13Speaker 01

  I mean, I think the number one thing is just like, don't get too worked up about it. The amount of, you know, speculation probably far outweighed what was going on there. I don't have firsthand information on Q-Star. We have a lot of, you know, a lot of speculation of our own of what it is. What I will say though is to some degree, people shouldn't be too surprised. You know, a while ago, we shared a model that could play the game of diplomacy at the level of human player. I don't know what people thought that model is.

• 29:46Alex Kantrowitz?

  Is this Cicero?

• 29:46Speaker 01

  Cicero, exactly, right? Cicero was having conversations with other players and it was reasoning about the game strategy. And so this was an example of a model that had language and that could reason, arguably in the hardest game out there. So I don't think people should be surprised that language models have the ability to be, to be effective in reasoning tasks, especially paired with mechanisms. In the case of Cicero, we were using some search mechanisms. Inside to be able to to achieve reasoning. It's, you know, it's it's a different architecture than than what we have in in Lama. But a lot of the ingredients of how to do reasoning have been explored in AI for forty years.

• 30:32Speaker 01

  And and are published and well known to to anyone who's taken even an undergrad level course in AI. So I'm not saying they there's not any innovation in the work that that OpenAI is doing or in the work that's happening across the community. I'm just saying it's not like a magic ingredient. I I'd be extremely surprised that that.

• 30:51Alex Kantrowitz?

  So what could the next level jump there be?

• 30:54Speaker 01

  I mean, there's a lot of theories of how to achieve reasoning in these models. One of them is to incorporate search as part of the model. And another one is incorporating, for example, a lot more coding abilities. Coding is executable. Coding allows us to essentially dig in through a a a sequence of operations. And another another, you know, direction that many groups are exploring is the use of retrieval based techniques. So you're retrieving information. Some of that retrieval can can make use of information where reasoning is present in the information. So lots of different ways to go about it. We're exploring many of them. Any respectable AI research group probably is too.

• 31:39Speaker 01

  What's really gonna make the difference is is how do we bring this together? Right? How do we make sure to have the right way for these components to integrate? And in some ways, that's still the hardest question in AI. How do we have different components working together in a very coordinated way?

• 31:58Alex Kantrowitz?

  Is there anything that you could see in the sort of research or production that would freak you out? Or are you sort of calm, cool, collected about where we're heading?

• 32:07Speaker 01

  There is stuff. I I mean, I I don't tend to freak out a lot. There's stuff that concerns me every day. You know, we we we review, you know, rigorously the performance of our model for for for different aspects. You know, there's many cases where I see a model and the performance, for example, on safety benchmarks, isn't what I would expect it to be. And then we go back and we keep on on working on it. And so. It's it's not that there's I don't think there's a ton of work to do. I just don't feel that like, you know, freaking out or being fearful about it is the best way to go about it. I think you just have to to look at the the data in a collected way. In many cases,

• 32:48Speaker 01

  we don't even have the the right way to analyze the properties of our model, you know, are this the model safe or unsafe? Does it have, you know, toxic behavior? Does it have bias? There's a lot of work to do to even develop the tools to assess this so we can look at it in a in a rational way. So we invest a lot in that also.

• 33:09Alex Kantrowitz?

  We're here with Joel Pina, the head of Meta's AI Research Division, still called FAIR, Fundamental AI Research. We've talked a lot about the research side on this side of the break. On the other side of the break, we're going to talk about product because Joel's division has recently moved toward the product side of Meta, and we're going to talk about what that means right after this. This episode is brought to you by Avpoint. Everyone's racing to roll out AI right now: co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about: all that AI runs on your data, and most teams have no single way to see it, secure it, and prove it's under control. That's exactly what Avpoint does. For 25 years, they've been

• 33:54Alex Kantrowitz?

  the trusted layer beneath the world's most demanding data, now extended across your entire AI estate: your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence, so innovation scales without scaling risk. Avpoint, the unifying trust layer for AI. Head to avpoint.com to see how enterprises deploy AI with confidence. Learn more at avpt.co/bigtechnologypodcast. That's avpt.co/bigtechnologypodcast.

• 34:32Alex Kantrowitz?

  Violent threats against executives are growing at an alarming rate. When a company experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door. The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a wide-glove enterprise security service, they protect your people with continuous personal data removal, proactive prevention tools and human-led emergency support. So when someone goes looking for your executives, Ironwell ensures they hit a dead end. So here's what to do. Go to ironwall.com/bigtechnology, fill in the quick form and request your free risk assessment. Their team will show you just how exposed your executives are

• 35:17Alex Kantrowitz?

  and how to lock it down before a threat reaches their front door. That's ironwall.com/bigtechnology. Learn how to stop online threats before they become real-world attacks. The link's in the show notes.

• 35:32Alex Kantrowitz?

  And we're back here on Big Technology Podcast with Joel Pinya, the head of Meta's AI Research Division. Your division just moved toward the products or under the product division within Meta. Let me start this segment with this question. It's a broad question. I don't think I've ever seen a disconnect as much as I'm seeing now where the discussion of where this technology can lead and what it does today is so, I would say, even divorced from the products that we've seen. I mean, yes, ChatGPT was groundbreaking and still incredible to use. And so is like some of the competitors.

• 36:12Alex Kantrowitz?

  But beyond that, have we really seen the product momentum when it comes to building on large language models? And, you know, we heard so much about it in enterprise. Yeah, we've seen some co-pilots from Microsoft, stuff like that, the bots in the messaging apps that Meta's creating. But, you know, for all the talk of revolution, it seems somewhat like an evolution. So what do you think about that? And what am I missing here?

• 36:38Speaker 01

  I I do see it as a bigger step change, I think, than than you're articulating it. And I think we have seen the birth of what I would call an AI research product. And so if I take, you know, for example, the GPT family of of models, I I do think there there there is a a real product there, people are using it, some people are using it every day. And so I I don't I don't think we've seen anywhere near everything that is possible, but I think we have to have a very open mind that the the product that are AI first are going to look very different than product we've we've seen before. That being said, I I I will say, you know, as much as we spend a lot of time worrying

• 37:20Speaker 01

  about what is the path on the research side, I I do think we need almost as much exploration on the product side. You know, on the the research side, the space of hypothesis to build these model is huge, but on the product side, like the space of new things you could build with this is huge. And and we don't yet have nearly enough information about what are going to be those products and those experiences that people are going to actually use every day and love using. So I I'm, you know, as as I talk to partners across the company, one of the things I encourage them to do is to really embrace the exploration that comes out of having a completely new tech stack compared to to what they had before and not just take, you know, the products

• 38:04Speaker 01

  that they know and like shove AI into them, but completely reimagine what is possible. So that's been a really, really fun conversation to have.

• 38:15Alex Kantrowitz?

  And one of the things that is going on is Meta's brought a bunch of AI bots into the messaging apps. Can you tell us a little bit about how that's going? I mean, I saw like the was there was like 12 or 20 different bots that are in these apps, and I played with them for a little bit, and then I kind of lost interest, and I haven't like seen any reminders that, hey, they exist. So how's adoption been there? What can you tell us about?

• 38:36Speaker 01

  Are you asking for more reminders that they're there? Because we can do that.

• 38:39Alex Kantrowitz?

  Honestly, yes. Honestly, yes, I think that would be good.

• 38:42Speaker 01

  Okay. And yes, the bots are there, they're they're available. Um, eh, the. The bots are an example of exactly what I mean, right? This is product exploration to some degree at its best in terms of like trying out different things. There's an intuition that there's enough there that it's worth putting it out in the hands of people. There's enough conviction as well as data to support releasing that for people to use. But I think it was very much the kind of product we hadn't done before and we're going to learn so much out of getting that into people's hands. You can think of it as really accelerating that that cycle of development. And there are some bots that are doing quite well, that are seeing quite a bit of use, some bots that are seeing a lot less use.

• 39:27Speaker 01

  I don't have the numbers with me for your listeners to understand. You know, FAIR does the fundamental AI research and we have a sister organization that is more connected to the product and is releasing those bots. We're tracking that really closely. That's feeding back into the product exploration conversation going on. I would say the bots, as well as the Meta AI assistant, are within a category of things that we call AI agents. And so we have a pretty wide exploration within that space of AI agents that you should expect to see new things coming in years to come. And the other example that we explored a bit this year is on the smart glasses, where

• 40:10Speaker 01

  we also have an AI system running on that, which is a very different, very different experience compared to the desktop or mobile cases.

• 40:19Alex Kantrowitz?

  Yeah, so adoption, how is adoption looking with those messaging bots?

• 40:23Speaker 01

  We'd have to, you know, we'd have to get someone else on your podcast to give you more detail on that. Yeah, absolutely. I'm sure we can find you someone who can give you some of that information.

• 40:33Alex Kantrowitz?

  So you mentioned that you have the product teams and you have FAIR, but FAIR used to be in Reality Labs and now it's like directly under the product team within Meta. Why did that move happen?

• 40:45Speaker 01

  So I mean, we had a wonderful set of colleagues and great work happening in Reality Labs Research. The truth is, right now, AI is moving so fast that it's really useful to be close to products that are in the hands of billions of people, to be able to have that quick product innovation, that quick signal back to the research. We were already working in close collaboration with the family of product teams, but this just makes things go a little bit faster. And the GenAI team that is really putting out some of these AI characters and Meta AI Assistant was already in that product team. So bringing us together gives us the ability to be much more coordinated, in particular

• 41:29Speaker 01

  from the research to, you know, building up the products and then releasing them. We're still going to continue to do a lot of work on the Reality Labs side. You know, we've been in that org for a few years. We've built up a lot of exciting projects, going to be maybe a few more years between now and when some of these get on the market. But these projects are not slowing down in any way. I think there's a really good understanding at the company level that, right now, the more we accelerate the AI roadmap, it is going to benefit both the existing products as well as the ARVR and the Reality Labs side of the company. So I think that's really where we are with this one.

• 42:12Alex Kantrowitz?

  One thing that seems like it's really going to be a thing that people talk about this year is video generation. We've just seen a little bit come out this week from Google. I know that you guys are working on it. Tell us a little bit about what that could look like. I mean, it's one thing to sort of type in, draw me a picture, and you get one out from DALL-E, or Meta has one, an image generator as well, but the video generation seems pretty wild.

• 42:38Speaker 01

  Yeah, it's been great to see that. It's not surprising, as soon as you, you know, you have good image generation, every time we've had progress in terms of image generation, the next step is how do we do 3D images and how do we do videos? Like, these are the two dimensions in which people quickly extend any progress in. In image generation. On the video side, I would say we've seen much, much better models coming out. But we haven't totally cracked the problem of generating long-form videos. The temporal coherence is quite tough. And I think, you know, for those of you who know a little bit more about video,

• 43:20Speaker 01

  you know, there's a piece of spatial coherence that you need to be thoughtful of, and that's the piece that image generation has, to some degree, solved in the last year. But the temporal correlation is something that, right now, is harder to do. We get really good quality video generation if you can intervene and kind of set a lot of the frames, and then you kind of use a diffusion model to interpolate in between. But to go from a really high level, for example, a script written in words, and to have like a full-length feature film is still going to take us a little while. One of the biggest problems there is to think about how to do generation in sort

• 44:02Speaker 01

  of a hierarchical way, not just do frame after frame after frame, but actually think of how do you generate globally some properties of your video, and then go through more and more granular resolution over space and over time. This is something that Jan has been thinking about a lot. He's working closely with some of our research teams in New York, in Montreal, in Paris, to make progress on that. And so I'm leaving a lot of that on him to drive. But I know he has a lot of ideas on this topic. And also to achieve that in a way that isn't too intensive in terms of data and compute.

• 44:40Alex Kantrowitz?

  Right, I think that's when you sort of get into like, can it predict and plan? And sort of really understand what reality is. That's some fascinating stuff. Okay, we're coming close to a landing here. Very quickly, we also spoke, we really had some fun conversations when we met the first time. We spoke with the, I think, Chief Technology Officer of Nvidia. And Mark just announced that you have 350,000 Nvidia H100 chips, and we'll end up with 650,000 by the end of the year, Nvidia H100 or equivalent. I'm just curious from your perspective as a customer of Nvidia, what makes those chips so effective for you? Now, it's obviously a technology component, but there's a software side of it as well, right?

• 45:25Alex Kantrowitz?

  So can you talk us through exactly what makes them so appealing? And do you think they're they are going to just be the unparalleled developer of these chips forever? Or are you starting to look at others like ARM, et cetera? Intel, you tell us.

• 45:42Speaker 01

  Yeah, I mean, it's honestly, it's clear to everyone that a lot of the progress in AI has been fueled by the availability of GPUs built by Nvidia. It's not the only solution. Google uses a lot their own TPUs as an example. So there's a few, there's a few others. But, but overall, I think Nvidia's GPUs have been essential to the progress. And we've been fortunate to have many of them to power our own research. There's a couple of things that make them great. One, you know, the GPUs on their own have the ability to parallelize a lot of the computation, which is essential for training these models. And we also have the ability to build them into systems, you know, networked with very fast interconnection

• 46:23Speaker 01

  between them to allow information to be passed around very, very quickly. And when you do that at scale with, with a few thousand GPUs, you can train some of these larger models. So that's really the, the essential ingredients. In terms of the trajectory there, of course, you know, as all responsible organizations, we're looking at all, at all options that could accelerate our work. We keep a close eye on the development of, of hardware. Right now, as Mark has shared, you know, I think the, the betting on the GPUs from Nvidia is a, is a sound bet for, for our research. But we're always interested to see innovation in all aspects of the stack.

• 47:01Alex Kantrowitz?

  Are you going to build your own chips?

• 47:03Speaker 01

  We will definitely be exploring some of that. Yes, yes. I mean, we built a lot of hardware for Reality Labs. We have some specific needs. And, you know, as much as we, you know, look at that for the, the ARVR devices, there's also a great group doing some of that innovation inside our infra team.

• 47:21Alex Kantrowitz?

  All right, last question for you. We started the conversation talking about AI reaching human level intelligence. Think that's going to happen, let's say, five years, over/under? You have a perspective on that one?

• 47:32Speaker 01

  On five years, we're going to see really strong systems across a broad set of tasks. I have some, some strong conviction that, that we're on a path there. After that, you know, I don't want to bin any intelligence into narrow box, whether human or AI, but we will be amazed by what gets done in the next five years.

• 47:52Alex Kantrowitz?

  All right, can't wait to watch it. Joelle, thank you so much for joining.

• 47:55Speaker 01

  Thank you, Alex.

• 47:57Alex Kantrowitz?

  All right, everybody. Thank you for listening. We will be back on Friday with a new show, Breaking Down the News, and we will see you for our Friday show on Big Technology Podcast.

• 48:12Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPact.com, not authorized by any candidate or candidate's committee.

• 48:41Speaker 03

  In this life, you can't win.

• 48:43Speaker 04

  When the world gets you down.

• 48:44Speaker 03

  No!

• 48:45Speaker 04

  Escape to Pluto TV.

• 48:47Speaker 03

  This is objectively fantastic.

• 48:49Speaker 04

  Relax into the movies and shows you already know and love, all for free.

• 48:52Speaker 03

  Shwing!

• 48:53Speaker 04

  Even our news is uplifting.

• 48:54Alex Kantrowitz?

  You stay classy, San Diego.

• 48:56Speaker 04

  Watch School of Rock, Crazy Ex Girlfriend, Anchorman: The Legend of Ron Burgundy, The X-Files, Wayne's World, and so much more.

• 49:02Speaker 03

  We're not worthy!

• 49:04Speaker 04

  The best things on TV really are free. You just have to know where to look. Escape to Pluto TV. Stream now, pay never.