Is ChatGPT A Step Toward Human-Level AI? — With Yann LeCun

Yann LeCun is the chief AI scientist at Meta, a professor of computer science at NYU, and a pioneer of deep learning. He joins Big Technology Podcast to put Generative AI in context, discussing whether ChatGPT and the like are a step toward human-level artificial intelligence, or something completely different. Join us for a fun, substantive discussion about this technology, the makeup of OpenAI, and where the field heads next. Stay tuned for the second half, where we discuss the ethics of using others' work to train AI models. Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/ Questions? Feedback? Write to: [email protected] Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  LinkedIn Presents.

• 0:14Alex Kantrowitz?

  Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. Yan Le Cun is the chief AI scientist at Meta. He's also a professor of computer science at NYU, and he's a pioneer of deep learning, which is the AI methodology behind many of the jaw-dropping advances you're seeing today. And Yan is here today to add what I think is some needed context to the discussion around generative AI. Obviously, we're all seeing this giant technological leap firsthand because we can now literally talk to the technology through advances like ChatGPT. But I wonder, is this stuff getting us any closer to human-level intelligence? And how

• 0:55Alex Kantrowitz?

  smart is it, actually? I think these are the key questions we need to ask about this new wave of generative AI and where it's heading. And we're going to ask those questions today. Yan and I go way back, all the way back to 2016, which is about seven years ago, when I profiled him and his interest in artificial intelligence that can both think and perceive the world like humans in a story for BuzzFeed News. It's always fun to speak with him, and I think you're going to love this episode. All that and more is coming up right after this. This episode is brought to you by Genesis. What does it actually take to put agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and

• 1:37Alex Kantrowitz?

  CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then, I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay? Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razor were able to cut their inference cost by 15 times.

• 2:21Alex Kantrowitz?

  Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 2:32Alex Kantrowitz?

  Jan, welcome to the show.

• 2:33Speaker 01

  Awesome to be here.

• 2:34Alex Kantrowitz?

  Great to have you back. This time we get to do it one-on-one, which is nice. It was great having you and Daniel Kahneman on. And you and I have been speaking for a long time since 2015, 2016, about the possibilities for artificial intelligence. And I think there was like a quick moment where everyone started talking about crypto, but now I think we're focused on the right stuff again.

• 2:56Speaker 01

  That was a distraction.

• 2:58Alex Kantrowitz?

  So let's talk a little bit about the wave of generative artificial intelligence that we're seeing and how that might relate to general intelligence. So, look, I think when we were talking for the first time, we were talking about how can we make a machine that looks like and that thinks like a human? And you had told me, okay, there's going to be a time where they start to predict. And if they can predict, they can plan. And that's how we're going to get close to artificial intelligence. And I said, okay, that sounds nice, Jan, but that's never going to happen. At least that's what I was thinking. Now we're starting to chat with some of these advances like ChatGPT. And I'm starting to think, oh, okay, maybe you wasn't that far off. So where are we now in terms of the pursuit for artificial general intelligence?

• 3:43Alex Kantrowitz?

  And is this a step, a big step forward towards that? Or is it, again, maybe a distraction?

• 3:49Speaker 01

  The short answer is it's not. It's not a particularly big step towards, you know, more like human level. Intelligence. I don't like the term AGI, artificial general intelligence, because I think human intelligence is very specialized. So if you want to designate the type of intelligence that we observe in humans by general, that's a complete misnomer. So I

• 4:12Alex Kantrowitz?

  Right.

• 4:13Speaker 01

  You know, I know maybe that cheap has sailed, but but I want to make that point, that that human intelligence is actually very specialized. So, no, the answer, the short answer is, so first of all, from the scientific point of view, ChatGPT is not a particularly interesting scientific advance. Of course, we don't know all the details because OpenAI has not published any.

• 4:35Alex Kantrowitz?

  Which is interesting for a company called OpenAI.

• 4:38Speaker 01

  Yeah, right now, it's the it's the least open that you can imagine. But they started out as wanting to be open and then they realized they couldn't fund their research unless they went slightly secretive. So.

• 4:52Alex Kantrowitz?

  By the way, can we just pause on that? Because that's interesting.

• 4:54Speaker 01

  Yeah.

• 4:55Alex Kantrowitz?

  What does that say about the the AI research world? That if you wanted to start out as open and you couldn't, you had to go for profit. Now they're like cap profit. Is that is that it's interesting that that it's impossible to fund this type of research without that? And does something structurally need to change because of that?

• 5:13Speaker 01

  Well, there are a lot of nonprofit AI research organizations, right, called universities. But there are also other nonprofits, like, you know, the Allen Institute for AI, for example, is nonprofit in in Seattle.

• 5:30Speaker 01

  OpenAI initially was was nonprofit and then switched to for profit. And originally was publishing everything they were doing and now kind of basically keep everything secret. They've they've become sort of a contract research house for for Microsoft to some extent. And it's because the the funding model is not clear. There's a, you know, reverse phenomenon that occurred at Google when I created FAIR nine years ago. We had a big drumroll about the fact that we're going to be completely open, etc. And we still are. We are holding that line. And as a result, it caused Google Brain at the time to become much more open than they were. Because that's what the scientists wanted.

• 6:10Speaker 01

  You know, if you, if you tell a scientist, you can come work for us, but you can't say a word about what you, what you're doing, you're killing their career. So, you know, so you have to basically buy their life, right? So, and open research is much more efficient. You just get more, more stuff out of it. You get things that is more reliable, uh, you know, and, and you attract better people. Uh, you, you have, you know, a better intellectual impact, which is, which means people have kind of more respect for you. I want to work for you and things like that. Right? So there's a lot of advantages to this, but there has to be an economic model. And the economic model, the only one I know outside of universities and philanthropy is

• 6:50Speaker 01

  a industry research lab inside of a large company that has, that is profitable and is sufficiently well established in this market that it can think for the long term and invest in fundamental research. So that's the case for, you know, certain corners of Google. That's the case for FAIR at, at Meta and not quite for Deepvine. So Deepvine was an interesting thing because they started out as a startup and you can't, you can't, absolutely cannot do research in a startup. You just can't because you just don't have the, the funds, right?

• 7:24Alex Kantrowitz?

  Or the ability to wait long term.

• 7:27Speaker 01

  That's right. And so, you know, you can do, you can do it for two or three years, but then you basically have to focus your entire attention to, uh, you know, building products and, and getting revenue and getting the company to survive. So what allowed them to do what they're doing is that they got bought by Google. But then, still then, after that, their economic model was not obvious because they were kind of sort of an ivory tower separated from Google and to some extent they still are. And, and so, you know, Google was, had the foresight to fund them regardless of whether they were producing something that was useful. But in the current context of, you know, more efficiency and money saving in the tech

• 8:08Speaker 01

  industry, that model might have to change. You know, it's not clear, right? So the economic return after 10 years or so, or nine years that Google has gotten from DeepMind, you know, is not clear, it's worth their investment. So they're banking on like, you know, bigger, longer investment. At FAIR, the business model is very clear. Like FAIR has had a huge impact on the company, mostly indirectly through other groups, right? Because FAIR does not work on products, but there's been like a huge amount of impact. So, long, long ride of story, the OpenAI, you know, could not keep doing what they were doing unless they were commercial, essentially, and made, and it caused them also to make

• 8:52Speaker 01

  kind of wide promises. And so what they have to do to be able to raise enough money from Microsoft and others is to make very flashy demos. And so they're very good at that. So they make very flashy demos. Their technology is not particularly innovative from the scientific point of view, it's very well engineered. So they put together, you know, large scale scaled up system with, trained with very well curated data. I mean, they know what they're doing, but in terms of advance, there's nothing, or not much.

• 9:25Alex Kantrowitz?

  So let me take this from, and by the way, thank you for that little diversion. I think it's good we went into that discussion of how this stuff gets funded and where it goes. But going back to our discussion of the march towards general intelligence, or sorry, I'll use your term, human level intelligence.

• 9:42Speaker 01

  You can use AI, it's okay, I'm not gonna get.

• 9:46Alex Kantrowitz?

  From a layperson's point of view, it does feel like, oh, okay, now I'm talking to AI. Now AI understands what I think and can actually draw it. Now AI can take my voice and start talking on its own.

• 10:00Alex Kantrowitz?

  So why isn't that a step towards this human level intelligence?

• 10:05Speaker 01

  Because the understanding that those the current systems have of, you know, the underlying reality that language expresses is extremely shallow. So those systems have only been trained with text, a huge amount of text. So they can regurgitate text that they've seen and, you know, interpolate for new situations, things like that. They can, you know, even produce code and stuff like that. But they do not understand, they have no knowledge of the underlying reality. They have no, they've never had any contact with, you know, the physical world.

• 10:50Speaker 01

  You know, if I, if I take a piece of paper, let's say, looking for a piece of paper.

• 10:58Speaker 01

  And I, you know, I, I hold it like this, right? And I tell you, I'm gonna lift my, my hand from one side. You can exactly predict what's gonna happen.

• 11:09Alex Kantrowitz?

  Right. And for listeners, that paper is being held horizontally. And so when you take your hand off, half of it drops.

• 11:16Speaker 01

  And I'm sort of, you know, with my two hands and then one hand moves away. So when one part of the paper kind of droops because of gravity. And you know exactly how it droops because, you know, you know, the properties of papers and stuff like that, right? So this type of knowledge that all of us have learned in the first few months of life, none of those systems have any of this.

• 11:39Alex Kantrowitz?

  Hold on, but I could chat with ChatGPT and say, what happens if I'm holding a paper with two hands and I let go with one? And it will tell you, it will droop.

• 11:48Speaker 01

  No, I mean.

• 11:49Alex Kantrowitz?

  It will answer that, but you think, but it just won't understand it.

• 11:53Speaker 01

  No, it might actually not tell you because it will depend, whether it'll tell you or not, depends on whether there were kind of similar situations in text that it's been trained on. But it's not at all. And I can, I can come up with, you know, a huge stack of similar situations. And each one of them will not have been described in any text. So, so then the question you, you wanna ask is, you know, how much of human knowledge is present and described in text?

• 12:22Speaker 01

  And my answer to this is a tiny portion, like most of human knowledge is not actually language related. It's completely non linguistic. I don't know, you do, I don't know, do carpentry, right? Build a piece of furniture.

• 12:36Speaker 01

  Your ability to predict what the piece of furniture is going to look like as a consequence of how you build it is completely non linguistic. And everything that has involves any kind of motor control, planning, things like that, that there's basically no linguistic knowledge. So now think about the entire collection of knowledge in every animal. It's obviously non linguistic because they don't have language, or at least not human type language, languages. You might exclude a few, a few species like dolphins and stuff, right?

• 13:09Speaker 01

  Now, you know, dogs and cats know a lot of stuff and about how the world works. And all of that knowledge, humans have it too, to some extent, not to the same degree, but in, in, in all domains, because we're all specialized. But none of that knowledge is captured by any current AI system, essentially. And that's a lot.

• 13:33Alex Kantrowitz?

  Okay, let's build on that. I'm just going to read the response. I asked ChatGPT, if I'm holding a paper horizontally with two hands and let go with one hand, what will happen? Now, I, I'm not going to say that, that, that you're wrong. Obviously you're right, but I'm just gonna, gonna read it to you for, for the sake of discussion. ChatGPT responds, if you are holding a paper horizontally with two hands and let go with one hand, the paper will tilt or rotate in the direction of the hand that is no longer holding it due to the unbalanced forces acting on the paper. If the paper was initially still, it will also move in the direction of the hand that let go due to the force of gravity acting on it. If the paper was moving in a certain direction before you let go, it will continue in that

• 14:15Alex Kantrowitz?

  direction, but may also be affected by air resistance and other external factors. That, that's a pretty cool response.

• 14:22Speaker 01

  It sounds correct and it's completely wrong. It's exactly wrong. It's actually the exact opposite. That's, that, that's happening, right? It's not moving. I mean, it's the part that you let go that droops, right? Yeah. And this is saying the opposite. Oh, that's true. Yeah, yeah, yeah. So, right, so it sounds correct. It's grammatically correct. The general theme is correct because there probably were descriptions of similar situations that the system was trained on and it kind of memorized it.

• 14:51Speaker 01

  And it tries to adapt the text so that it is relevant to the current situation, but it gets it completely wrong. And, you know, it gets wrong things like, you know, comparing numbers. So you tell it, you know, you do a prompt to say, you know, you know for a fact that seven is smaller than five or bigger numbers like 250 is smaller than, you know, 196. And then you start kind of, you know, telling a story with numbers. And it will assume that what you prompted it with is right,

• 15:30Speaker 01

  even though it's false. But you don't even have to do this, right? I mean, there's very, a lot of situations like this where the system will just not say things that are factually correct. Now, why is that? It's because large language models are trained to predict the next word in a text. And they train on enormous amounts of text and they have enormous amounts of memory.

• 15:54Speaker 01

  But they basically, you know, probabilistically generate the next word. And then re-inject that word into their context of a few thousand previous words that they've said or the prompt. And then generate the next word again and then re-inject that in the input, etc. There are various ways to do this more efficiently, but that's the basic, excuse me, the basic idea.

• 16:18Speaker 01

  So now, the issue with this is that there is no...

• 16:24Speaker 01

  There is no way to specify a task that the system has to accomplish other than by specifying that task inside of the prompt, which is a very circuitous, inefficient and complicated way of specifying a task.

• 16:41Speaker 01

  It's not very controllable. Okay, that's the first thing. The second thing is that system. So that system is not like optimizing an objective. If you want like trying to satisfy an objective. Right. It's just kind of generating one word after the other. And because it's generating one word after the other, it's not doing any planning. So it's not like planning to tell a story or an answer where there is, you know, like

• 17:08Speaker 01

  a kind of a line to the story or, you know, a set of facts and things like this.

• 17:15Speaker 01

  It just generates one word after another. It has no capability of generating commands for a tool like say a calculator or anything like that. Or a physics simulator, for example, you could have simulated that piece of paper and then observe the result and then kind of tell you what the result was. That's what we do in our head. When we are being described this kind of situation, we have our own internal mental simulator. And because we've learned how the world works, we can simulate what goes on and then describe the result. Right. L L M's do not have that. They don't have any internal model of the world that allows them to predict.

• 17:52Speaker 01

  And then in addition to this, you would like when the system produces a statement, you'd like to be able to verify that that statement is factually correct or does not, you know, break any kind of logic of any kind, you know, compared to another statement that was made before. And there is no way in the current architecture of those systems to do this. Right. Until we build systems that have some internal model of the world that allows them to kind of simulate the world, if you want, some way of generating actions on the world to use tools like a calculator or interrogating a database or a search engine. And

• 18:36Speaker 01

  an objective that it has to satisfy for the tasks that we ask it to accomplish. And a method by which it can plan an answer that satisfies the objective, is factually correct or not, depending on the desired behavior. You know, and perhaps interrogates the right sources of information. We're not going to have anywhere, anything resembling human level intelligence.

• 19:05Alex Kantrowitz?

  Okay, and I definitely want to get to the type of research and the models that might get us there. But first, I want to talk a little bit about the hallucination that ChatGPT just had with my interaction with it. Because hallucination is definitely a big issue. And I'll be honest, and this is embarrassing to admit as a journalist. But as I read it, I was ready to believe it. Because it was like, oh, here's AI answering a question with a somewhat plausible answer and stating it so confidently. And that is an issue with these models, right? Is that they do hallucinate. That's probably why we haven't seen Google bring it into search.

• 19:39Speaker 01

  Well, that's why you haven't seen any kind of systems of this type from either Google or Meta, despite the fact that they have the technology.

• 19:46Alex Kantrowitz?

  Okay.

• 19:47Speaker 01

  Okay, so I mean, you have to realize that most of the technology, underlying techniques used in ChatGPT have been invented at Google and Meta.

• 19:56Alex Kantrowitz?

  Right.

• 19:56Speaker 01

  And the whole thing has been built with PyTorch, which is, you know, made by Meta. It's not owned by Meta anymore, but it originated there. So, for example, you know, it uses large transformers, transformer architectures. Those were originally invented at Google. Transformer themselves use something called associative memory. I mean, it's called self-attention, but it's, you know, basically the same principle. Those were basically proposed by Meta many years ago. They they use self-supervised pre-training by removing words. That's a Dinozing autoencoder. Those are techniques that, you know, go back to the 1990s or even 1980s in some cases. They've been popularized by the BERT-style language models, again, were proposed at Google. And then a number of techniques for dialogue systems.

• 20:41Speaker 01

  So there is a very active dialogue system research group at Meta that has proposed lots and lots of methods, which inevitably, whether they say it or not, OpenAI must have been influenced by. And then they use a technique now, now that ChatGPT is available. They're using a technique that is called reinforcement learning through human feedback, RHF, which was proposed by DeepMind, actually. So, you know, they've done a good job at sort of integrating a lot of things that, you know, have been proposed in the literature and sort of engineering a system that kind of produces impressive demo. And they have to produce impressive demos because that's the economic model.

• 21:24Speaker 01

  That's how they're gonna raise money from Microsoft and others. Whereas if you are Meta or Google, you could think about like, you know, putting out a system of this type that you know is going to spew nonsense. And, you know, because you are a large company, you have a lot to lose by, you know, people kind of making fun of you for that. And it's not clear what the benefits are, okay?

• 21:48Alex Kantrowitz?

  Right.

• 21:49Speaker 01

  So we're still working on those things to make them useful.

• 21:51Alex Kantrowitz?

  But didn't Meta put out a system of its own, Galactica? And this was something that was... So, yeah, talk us through what happened there. Because this was a system that was supposed to summarize scientific literature and do lots of other cool things. It comes out and then three days later, it goes back behind closed doors.

• 22:08Speaker 01

  That's right. So what happened? There was a previous system also called Blandabot. And there's another story about that that we'll...

• 22:14Alex Kantrowitz?

  Oh, right. That's the thing that started talking about how Mark Zuckerberg is a sort of money-hungry capitalist.

• 22:22Alex Kantrowitz?

  You didn't get called into Zuck's office and be like, hey, what are you guys talking, telling it about me?

• 22:28Speaker 01

  Yeah, no, no, it just, I mean, they're just trying on the, you know, the general

• 22:33Alex Kantrowitz?

  Conversations.

• 22:34Speaker 01

  Things that you read in the media and in the media, you know, Mark Zuckerberg is very often painted as, you know, some sort of money hungry bad guy, which he is not at all, but that's the way he's painted. So, um, uh, yeah, so let's start with Blunderbuck then. So Blunderbuck was, was put out and several months ago, and it's a dialogue system, a bit like ChatGPT. It's designed to be entertaining more than anything else. And it's capable of having multiple personalities. So we can talk into like several styles and things like that.

• 23:06Speaker 01

  And it has provisions to somewhat verify factual correctness, although not particularly well-developed. But it, we, it does have a lot of kind of guardrails and, and say, uh, and kind of systems to like prevent, prevent it from, you know, saying things that might be offending or, or whatever.

• 23:32Speaker 01

  Or, or even objectionable or even controversial, right? So if you try to get it to talk about anything related to politics or religion or anything like that, it will change the topic.

• 23:46Speaker 01

  And it won't tell you it's because I don't want to talk about this. It would just change topic, right? And so people thought this thing was really stupid and boring, uh, because it doesn't want to talk about anything that's kind of, you know, controversial or fun, which is the kind of stuff you want to talk, you know, everybody wants to talk about.

• 24:02Speaker 01

  And, uh, and it, it's frustrating because it would change topic and, you know, anytime you wanted to, to do that. So, uh, it was not nearly as convincing, uh, but so, so you could, you could, you could say that the reason it was, you know, not that impressive in the end is, was because it was, it was made to be safe. Essentially.

• 24:23Alex Kantrowitz?

  Okay.

• 24:24Speaker 01

  If it's too safe, it's boring. Um, so, um, now let's go to Galactica. So Galactica is a different animal. It's also a large language model. And that large language model has been trained on the entire scientific literature. So this is something like, you know, millions of scientific papers. And the purpose of it, it's used, is to help scientists write papers. So think of it, so it's not going to write a scientific paper for you. It's not going to answer scientific questions, although you could try to use it for that. Uh, but sometimes it might make stuff up. But it's designed to be essentially a, a, you know, predictive keyboard on steroids. Right? So you start typing a paragraph about something.

• 25:07Speaker 01

  And it will, you know, complete the text, the entire paragraph. It will insert relevant citations. Uh, if you say, you know, the, the state of the art in object recognition on the ImageNet database is, it will find the correct reference. It will actually, you know, build a, a table of results with links to the references and stuff like that. Right? Uh, but the same way, uh, driving assistance systems for cars are just that, driving assistance. This is just writing assistance, right? So in the end, your hands have to be on the wheels, on the wheel at all, at all times. Uh, you are responsible for the text that in the end, uh, is finished. It just helps you, it's a tool that helps you write more efficiently, particularly if you are not a,

• 25:51Speaker 01

  you know, native English speaker, uh, which, you know, most scientists aren't.

• 25:56Alex Kantrowitz?

  Right? I mean, I even use ChatGPT that way. I put in the beginning of a paragraph and say, hey, which ways could this go? Understanding that it might not be accurate.

• 26:07Speaker 01

  And that's the way you should treat it really. Um, as, uh, you know, as, as a predictive keyboard on steroid, uh, and, and something that just helps you write, but it's, it's not gonna, you know, write, invent new things, answer questions, do science, or blah, blah, blah. So what happened was that when we put out Galactica. People tried to break it. So people are not scientists, like didn't understand what the use of it was was going to be. And and they would prompt it with things like, you know, what what are the benefits of eating crushed glass? Or something like that. And of course, that's kind of a leading question. So the system will kind of make a story of like, why do you know, to eat crushed glass?

• 26:49Speaker 01

  And then the reaction on Twitter was, oh, my God, you know, people are going to eat crushed glass because they're going to listen to it. Right? Which is, you know, insane. I mean, it's stupid. People are not that that dumb.

• 27:01Alex Kantrowitz?

  Well, yeah, and I think you overestimate people a little bit, but sorry, continue.

• 27:05Speaker 01

  Well, there might be a tiny proportion, but like, you know, are they likely to use things like this? Not clear. You know, particularly Galactica was, which really ultimately was designed to be, you know, integrated into tools that scientists use to write papers. Right? So.

• 27:21Speaker 01

  And then, you know, others more seriously said, oh, this is going to destroy scientific publication because now.

• 27:29Speaker 01

  You know, anybody can generate a a nice sounding scientific paper and then will submit it to a a conference. And this would completely flood and overwhelm the reviewing system that we have in science and destroy science. And I thought that was a completely ridiculous argument because the reason why you might want to submit a paper is because you want to prop up your CV. And so you have to put your own name on it. Otherwise, what's the point? And if you put your own name and it's garbage, it's bad for you. It's bad for your career. Like, you know, if you if you, you know, send a hundred papers that are complete nonsense to a conference with your name on it, it's not going to be good for your career. Like, absolutely not. So so I don't.

• 28:12Speaker 01

  So I think this kind of knee jerk reaction was completely unwarranted. And it really mirrors a lot of knee jerk reactions that have happened in the past when new tools or new communication technologies have appeared. Where, you know, all of all of them were going was going to destroy society. So I think it's the same kind of knee jerk reaction that we're observing with with AI today. You know, this is not to to say that there is no danger, but but it's not like the horrible things that people make it to be.

• 28:42Alex Kantrowitz?

  Right, so then why not keep it up?

• 28:44Speaker 01

  Well, so what happened was the team that worked on it, which is within FAIR, is called Papers with Code. They were so distraught by the reaction that they just couldn't take it. They said, like, you know, we're just going to take it down. Like, this was not a high-level decision. This was not a decision by communications departments or the management. This was them. Okay, it's a small team, a few people. And they felt like really good about what they had produced. They wrote a very long paper, they open sourced their code. They took down the demo, but they left the code. So anybody can download Galactica and run it on their own computer.

• 29:21Alex Kantrowitz?

  Interesting. And this is sort of what we're going to see. I just wrote this story and we've talked about it on the podcast about how the battle over, you know, AI ethics, applications of AI is just going to be extremely intense. And I think we're starting to see some of that. So we've covered hallucination. Good, we got to that. Let's talk about the type of models that you think so. You talk about our move to artificial, or to human-level intelligence, needing and understanding of the environment, things of that can't be expressed with words. Now, when you tell me that, I'm back to kind of where I was in the early days of our conversations, saying there's no way technology is going to be able to do that. But it sounds like you think that there is a chance that it can. How do we get there?

• 30:03Alex Kantrowitz?

  And what type of advances are we seeing today that might lead us to think that we do have a chance?

• 30:11Speaker 01

  So I actually wrote a position paper about this, which is rather long, but the intro is easy readable by non-specialists. And the title of it is "A Path Towards Autonomous Machine Intelligence", where basically I lay out a plan or a path forward to address those questions, to build AI systems that are capable of planning.

• 30:35Speaker 01

  Whose behavior is controlled by objectives that need to be satisfied,

• 30:40Speaker 01

  which can be specified by hand or learned. So things like, you know, factual correctness and blah, blah, blah, and answer this question and, you know, don't spew offensive stuff and things like that. And have internal models of the world or the thing they are interacting with, which could be a person, if there are dialogue systems, then when you're talking to someone, you need to have a mental model of what that person knows and can understand. Okay. If you speak in terms that the person cannot understand, then, you know, communication fails, right? So when you talk to someone, you have something to say

• 31:25Speaker 01

  and you may have to tell them, you know, some background information depending on what you think they know. And then sort of take them to the stage where

• 31:36Speaker 01

  where you think they've absorbed the information that you think would be useful to them.

• 31:42Speaker 01

  So you need

• 31:44Speaker 01

  to talk to someone, you need an internal model of what that person, how that person will react to what you tell them.

• 31:51Speaker 01

  And if it's another type of agent that generates actions in the physical world or even in a digital world, like a robot, that, you know, a domestic robot that, you know, needs to cook or fill up

• 32:05Speaker 01

  the dishwasher. That model, that system needs to have an internal model of the world that allows it to predict what the state of the world is going to be as a consequence of its own actions. Because that's what you need to be able to plan. If you're going to plan a sequence of action to arrive at a goal, you need to be able to imagine, if I take this action, this will happen. And then if I take that action, this will happen, et cetera, et cetera. And so you can optimize your sequence of actions so that the resulting sequence of states that the world is going to follow is going to satisfy your ultimate goal. And then the issue with this is that, you know, how do we get a machine to learn models of the world? Baby humans and animals do this in the first few months of life.

• 32:50Speaker 01

  Mostly by observation and understand a huge amount of background knowledge about the world, basically just by observation. This is the type of learning that we need to reproduce in machines that we currently cannot do except for simple things like text, because you have to understand that text is actually simple compared to the real world. Right. So how do you do it? Okay, so. Sure, it's simple. There's one way to not do it, which is the way large language models are trained. So the way you pre-train a large language model is that you don't train it to just predict the next word in a text. You take a long piece of text, a few thousand words typically, and you remove some of the words. You blank them out, you replace them by a

• 33:35Speaker 01

  blank marker, or you substitute another word, or you do various corruption things. And then you train some gigantic neural net to predict the words that are missing. Okay? So this is called self-supervised learning. And this particular instance is what's called a generative model, which is much more general than the usual kind of colloquial use of the term generative model. It's generative in the sense that it produces signals that are the same as the input. Okay? It tries to fill in the blanks, if you want. Okay? So it generates the missing information, if you want. And this particular instance

• 34:19Speaker 01

  is called a denoising autoencoder. Why autoencoder? Because you give it an input that is corrupted, and you ask it to produce the clean version of that input. That's called a denoising autoencoder. And again, that concept goes back to the 1980s. Nothing new there, except the application of this idea of denoising autoencoder to text using a transformer architecture, which is those, you know, very large neural nets that we use nowadays with 40, 90 layers or whatever, and hundreds of billions, or at least tens of billions of parameters. Works amazingly well, like surprisingly well. So in the process of being trained to filling in the blanks, those models basically learn to represent language as a series of numbers

• 35:03Speaker 01

  that represents the, basically the meaning of the input sentence to the extent that this meaning is useful to fill in the blanks. Okay. So inevitably, by being trained to do this, the, those systems understand a little bit about the real world, but not much. It's very superficial. Now, if I, if I train the system with sentences of the type, you know, the cat chases the blank in the kitchen. You know, the blank there can only be a few things, right? It's either a mouse or an insect of some type, or maybe a laser spot that someone is, you know, playing with the cat or something like that, right? There's only a few options.

• 35:46Speaker 01

  And so those systems are trained to produce essentially a long list of numbers that are scores for each of the possible words in the dictionary for as, you know, how likely they are to appear at that location. And, and to be able to do a good job at this, the system has to understand a little bit about, you know, what's a kitchen and what's a cat and, you know, cats chase mice and things like that. And so it learns that, but it's very superficial. And then what you do, once you've trained the system to do this, you chop off the top layers and then you can use the internal representation as input to a subsequent downstream task that you can train supervised, like for example, translation or, you know, hate speech detection, for example.

• 36:28Speaker 01

  So the, this technique has become completely mainstream in natural language processing. And so much so that companies like Meta and Alphabet and their various services have deployed those things massively for doing things like content moderation. Right? So hate speech detection, for example, has made enormous progress over the last three, four years. And it's essentially entirely due to the fact that we're now using those techniques. Okay. We can do hate speech detection in multiple languages, basically with a single model. With, we don't have to train it with a lot of data in each language, because we don't have a lot of data in every language.

• 37:10Speaker 01

  We have a lot of data in English and others, and you know, French and German and blah, blah, blah. You know, maybe the main language in India, but India has, you know, like an enormously large number of languages that people use, including online, you know, and local dialects and stuff.

• 37:27Speaker 01

  So, you know, how you make the hate speech detection work in 500 languages? And even 500 would not cover everything. So you need those techniques and, and you know, this has made a huge amount of progress. Now, one thing you can do with those things, you can fine tune them to just predict the next word, and that's what large language models are. Okay. Now, coming back to this question of, of planning. Last time we talked with Danny Kahneman, which was amazing discussion. Danny is, is famous for, you know, this book, Thinking Fast and Slow, and for this idea that

• 38:06Speaker 01

  we have basically two ways of acting, system one and system two, or equals system one, system two. So system one corresponds to tasks that you accomplish subconsciously. You don't need to plan anything. It's completely reactive. So think about like, you know, you're, you're an experienced driver and you're driving on the highway. You're barely paying attention. And, you know, you're, you're not calling on your sort of high level cognitive functions to do that. You can do it more or less automatically.

• 38:41Speaker 01

  Anything that involves a real time reaction, like, you know, playing tennis or something like that. That's mostly subconscious. You're not, you just don't have time to plan, right? So it has to be built into your muscle memory, as we call it, right? But even complex tasks, like, you know, you play chess and you are a chess grandmaster, you're playing a, you know, a simultaneous game against 50 bad players like me, you don't have to think. You can just look at the board and just immediately play. You don't have to plan anything because it's so easy for you. You know, it's become kind of a subconscious task. Now, all of those things though, all of those tasks.

• 39:20Speaker 01

  When you learn them, or when you're not very good at them, you have to use your entire cognitive resources. You learn to drive, and you pay attention to everything. You imagine all kinds of catastrophic scenarios. You drive slowly. You're using your photocortex, your model of the world, that tells you, "I'm driving next to a cliff, and I know that if I turn my steering wheel to the right, I'm going to fall off the cliff, and nothing good is going to happen."

• 39:52Speaker 01

  You don't need to try it to know that something bad is going to happen, because you have this model that you've built in your head for the last seventeen years, if you are a seventeen-year-old.

• 40:02Speaker 01

  So that model of the world allows you to predict the consequences of your actions, and allows you to learn extremely quickly any new skill.

• 40:12Speaker 01

  Same with chess. If you're a bad chess player, you will have to think for fifteen minutes or more when you play against a challenging player, and plan all kinds of strategies. So what characterizes intelligence is the ability to predict, first of all, and then the ability to use those predictions as a tool to plan, by predicting the consequences of actions you might take.

• 40:48Speaker 01

  Prediction is the essence of intelligence. Okay, so now, here is the problem. This idea of denoising autoencoder that we use to pre-train natural language processing systems works for text. It doesn't work for anything else. So it doesn't really work for things like video or images. So a natural idea is you take an image, you block some pieces of it, and then you train some system to predict the pieces. We construct the parts of the image.

• 41:15Alex Kantrowitz?

  That's how DALL-E works, huh?

• 41:17Speaker 01

  Not really, no. Okay. It's, it's, you know, uses, I mean, DALL-E 2 uses a diffusion model, which is kind of a slightly different idea. But it has. Get to that later. Okay.

• 41:31Speaker 01

  But if you do it the way I just described, there is like one or two model that sort of work that, that use that. One is called MAE by my colleagues at, at FAIR. That means mass autoencoder. But it doesn't work as well as other techniques. And those other techniques are not generative models. Okay, so there are models that do not attempt to reconstruct missing information. There are techniques that

• 41:59Speaker 01

  attempts to reconstruct missing information, but not reconstruct the image itself, but reconstruct a representation of that image, an internal representation of that image. Those techniques, I, I call them joint embedding architectures. So essentially, in the, you know, the neural autoencoder, you have an encoder that produces, produces a representation of the input, whether it's an image or text or video or whatever. And then you try to reconstruct the uncorrupted version of the input, the input being corrupted, right? That's through a decoder. In a joint embedding architecture, you have two encoders. One encoder sees the perfect version of the input. The other encoder sees a corrupted version or distorted version of some kind. You run those two things through both encoders.

• 42:45Speaker 01

  And then you tell the encoder that sees the corrupted input, can you predict the representation of the full input? But you're not trying to reproduce all the details. And it makes sense for images because, or videos. So let's, let's imagine we're on a scenario with video, right? So I have a video clip.

• 43:02Speaker 01

  And I have a complete video clip. And what I do is I, I mask the, the last half of the video clip, okay? The corrupted version of the video clip is just the, the first half of that video clip. Okay, the rest is invisible. And then the, the complete version, of course, is a full video clip, right? So you run this full video clips with some neural net that produces some representation of that video clip, right? And then you train this guy to predict the representation that that guy has produced. And of course, implicitly, what it has to do is predict the rest of that video clip. But in representation space. Now, why is that better than just reconstructing the piece of the video clip that is missing? Just predicting it.

• 43:47Speaker 01

  The reason is, there is an infinite number of things that can happen after a particular video clip, right?

• 43:54Speaker 01

  And we don't know how to represent the distribution of all the possible things that could possibly happen.

• 44:04Speaker 01

  We cannot do it in pixel space, right? So, for example, you know, you're seeing a green screen behind me right now. You're not seeing the back of my head. Okay, you can make, so if I start rotating my, my head, you might be able to predict what the back of my head will, will look like. And when you look at it, you might be surprised by what you see. Maybe I have a small ponytail or something. I don't, but you know.

• 44:28Speaker 01

  And so, and there is the, you know, an infinite number of variations of what could be possible there. And I could decide to, you know, change the action and move my head in a particular way or, or something. So you can't just predict every detail of what's going to happen. Let's say you want to build a, it's very important if you want to do things like building self-driving cars. Because you, to be able to drive safely, you'd like to be able to predict what cars around you are going to do.

• 44:59Speaker 01

  Or what pedestrians are going to do. You see a kid on the sidewalk and, you know, a soccer ball kind of crossing the road. Good chance that the kid is going to run after it, right? So you slow down.

• 45:13Speaker 01

  So, you know, we, we have this sort of intrinsic models of the world that allow us to predict and then as a consequence plan kind of safe actions. But then, in that same street where the kid is with a soccer ball, it's, you know, lined with trees and there is a pond behind the trees. And it's a windy day, so the tree, you know, all the leaves are moving and there is, you know, ripples on the pond and everything. And you don't want to spend any resources predicting all of those details, which are essentially unpredictable. And so that's why generative models essentially fail when you train them on, on images. Because there's just so many details to predict. The system gets completely lost in predicting all kinds of irrelevant stuff, which we call noise. But, you know, how would the system know?

• 45:59Speaker 01

  So one of the main things I'm proposing in that in that piece is to abandon those generative models, basically, and focus on those joint embedding architectures for getting models to learn, system to learn models of the world, predictive models of the world.

• 46:17Alex Kantrowitz?

  Okay, and maybe that's what gets us there. Okay, let's let's take a quick break here and then come back for about 10 more minutes on the show. We have Jan LeCun with us. He's the VP and Chief AI Scientist at Facebook and known as the father of of deep learning. So plenty to talk about. We'll be back right after the break. This episode is brought to you by Avpoint. Everyone's racing to roll out AI right now. Co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data, and most teams have no single way to see it, secure it, and prove it's under control. That's exactly what Avpoint does. For 25 years, they've been

• 47:00Alex Kantrowitz?

  the trusted layer beneath the world's most demanding data, now extended across your entire AI estate: your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. Avpoint, the unifying trust layer for AI. Head to avpoint.com to see how enterprises deploy AI with confidence. Learn more at avpt.co/bigtechnologypodcast. That's avpt.co/bigtechnologypodcast. Violent threats against executives are growing at an alarming rate. When a company

• 47:42Alex Kantrowitz?

  experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door. The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a white glove enterprise security service, they protect your people with continuous personal data removal, proactive prevention tools, and human-led emergency support.

• 48:09Alex Kantrowitz?

  So when someone goes looking for your executives, Ironwall ensures they hit a dead end. So here's what to do. Go to ironwall.com/bigtechnology, fill in the quick form and request your free risk assessment. Their team will show you just how exposed your executives are and how to lock it down before a threat reaches their front door. That's ironwall.com/bigtechnology. Learn how to stop online threats before they become real world attacks. The link's in the show notes.

• 48:44Alex Kantrowitz?

  And we're back here with Jan LaKoon, the VP and Chief AI Scientist at Facebook, Facebook Meta. I don't know, is it the same? Is it different? Meta, it's called Meta.

• 48:55Speaker 01

  I have two jobs, actually. I'm also a professor at NYU.

• 48:57Alex Kantrowitz?

  And professor at NYU, right? Of course, can't forget that. So, Jan, let's, let's talk a little bit. Let's just go one, one level deeper about this in the time that we have left. So Facebook, I know, released a, a application, or at least has it internally where you can type a sentence. And it will make like, not like an image with DALL-E, but like a small little video that actually resembles that sentence. So is that taking us closer to this ability to predict and understand the world that you're talking about?

• 49:27Speaker 01

  Yes and no. So yeah, there, there is two little demo applications. One is, is called Make a Scene. And this is one, you know, a bit similar to, to, to DALL-E, where you type a description of an image and it just produces an image. It's based on slightly different ideas, but it's, you know, it came out before DALL-E, actually.

• 49:50Alex Kantrowitz?

  Right.

• 49:50Speaker 01

  The, the main author of DALL-E, by the way, is, was an, Aditya Ramesh, brilliant young guy, who, he was an undergraduate student with me and did some research projects in my lab before going on to OpenAI as an intern and then being hired as a, as a scientist. So, so there's this thing called Make a Scene that produces fixed images. And then there is a similar thing called Make a Video that produces short, short video clips, essentially. And they're kind of systems that can do this now from Google as well.

• 50:24Alex Kantrowitz?

  And right.

• 50:25Speaker 01

  Et cetera. They're not really sort of publicly available yet.

• 50:27Alex Kantrowitz?

  Yeah, you can't use it. I saw the blog post. I was like, oh, I want to use this. And then I realized it wasn't public. I imagine the problems that you had with Galactica would pale in comparison to whatever happens with this movie maker.

• 50:39Alex Kantrowitz?

  So then why not release it?

• 50:40Speaker 01

  It's something with with language that people kind of, I think, you know, pay more attention to. No, the the the many issue with those things is it's like, where where does the trending data come from? And things like that. So, uh, you know, before we can roll them out, that has to be trained on data that is

• 50:59Speaker 01

  acceptable and all that stuff.

• 51:02Speaker 01

  Uh, there is a similar system also, uh, in the works. Um, there's been some publication on it, uh, that can generate audio. So it can generate sound effects, uh, also by textual description or simultaneously with the video. Um, there's systems also, um, from FAIR that, uh, produce music. Uh, they can do music continuation. So it's like a large language model, but with music, right? So you feed it audio and it just continues.

• 51:26Alex Kantrowitz?

  Interesting.

• 51:28Speaker 01

  Uh, so you're going to see more and more of those things. And Meta is really interested in those things in sort of creative aids. Because, uh, everyone who is on an online service, whether it's a social network, like, like, you know, Facebook or Instagram, or whether it's going to be, you know, in the metaverse, everyone is going to need to have easy ways to generate, uh, content and be creative without necessarily having all the, you know, being technically astute enough, uh, in in terms of of art, uh, to, uh, to do all that things. So being able to generate, you know, be creative and generate content, I think, is something that's very, very important. And so you're you're going to see things like that popping up on, you know, Meta's services in various, uh, various spaces.

• 52:13Alex Kantrowitz?

  Interesting.

• 52:13Speaker 01

  Generating, uh, images, effects, modifying images, generating video, generating music, sound effects, 3D models. Okay. Which of course is, uh, important for the metaverse. Um, and, uh, and eventually, you know, all the way out to intelligent virtual characters and stuff like that.

• 52:31Alex Kantrowitz?

  Really? Okay. So Meta's going to go in all in on this stuff.

• 52:34Speaker 01

  Oh, totally. Yeah.

• 52:35Alex Kantrowitz?

  Yeah. I want to ask you a question about it because, you know, the thing about, um, you know, art, song, videos, all this stuff. It, there's a musician songwriter, Nick Cave, who did this post about.

• 52:49Alex Kantrowitz?

  Dolly writing, I mean, not Dolly, ChatGPT writing songs and the authenticity of all of it. Now, there's been this debate, is this going to take everybody's job? Or is it too, too soulless and dumb to actually do this? This is sort of the argument that this AI really can't do what humans do. So this person writes, Nick writes, ChatGPT may be able to write a speech or an essay or a sermon or an obituary, but it cannot create a genuine song. It could perhaps in time create a song that is on the surface indistinguishable from an original, but it would always be, it will always be a replication, kind of a burlesque. Songs arise out of suffering, by which I mean they are predicated upon the complex

• 53:30Alex Kantrowitz?

  internal human struggle of creation and, and well, as far as I know, algorithms don't feel. So I'm curious what you think about that, like from the perspective of a songwriter. Can this stuff actually produce?

• 53:44Speaker 01

  Yeah, there's, there's a lot of questions around this, right? So first of all, it is true that current AI systems don't feel, okay? But it's not going to be true for much longer. So if we have systems of the type that I was describing before, that have objectives and that plan actions to optimize objectives, they will have the ability to predict the outcome of a situation, which means if they are able to satisfy their goal, given a situation, they're going to feel the equivalent of elation. If they can predict that they're not going to be able to satisfy their goal, you know, they're going to be disappointed with themselves.

• 54:27Speaker 01

  If they predict that the situation may be dangerous for either themselves or the person they are interacting with, they might actually have, you know, the equivalent of fear. So if you have autonomous AI systems that work by optimizing objectives and they have the ability to predict, they are going to have emotions. It's inseparable from autonomous intelligence.

• 54:50Alex Kantrowitz?

  Fascinating. So maybe we'll have an AI that tries to fulfill some objective and fails miserably and writes a song about it and that will be.

• 54:58Speaker 01

  But, but.

• 55:01Speaker 01

  A big part of art, I mean, it's certainly true for songwriting, but also for novels and for jazz improvisation and all kinds of stuff, right? Is basically a vehicle to communicate human emotions. And Nikave has a point, which is that there is nothing to communicate unless you have emotions that the auditor or reader can relate to, right? So, and that really depends on the person. Like, you know, I'm totally taken by, you know, good jazz improvisation.

• 55:34Speaker 01

  But like words on songs like Nikave's like do nothing to me. I mean, we have different tastes, right? Different ways of reacting to different things. But so, you know, he's right in the way that if you want a genuine experience of art,

• 55:51Speaker 01

  of communicating human experience or human emotions, it has to come from another human. Even if machines eventually have emotions, they're going to be very different from humans. So it's not going to replace this type of genuine art.

• 56:05Alex Kantrowitz?

  Could be a different form of art.

• 56:08Speaker 01

  Yeah, like, you know, project yourself like back

• 56:12Speaker 01

  a couple hundred years ago, where like every, like you wanted to buy a salad bowl or something.

• 56:21Speaker 01

  It would be handmade, right? It would be like a potter that would, you know, kind of.

• 56:27Alex Kantrowitz?

  Make that bowl.

• 56:29Speaker 01

  Right, and just make it by hand and bake it and everything, right?

• 56:34Speaker 01

  As a consequence, it would be relatively expensive for most people.

• 56:38Speaker 01

  And then, and then came industrialization. So now it became possible to build, you know, ceramic bowls for pennies in sort of large quantities. And did that.

• 56:52Speaker 01

  Kill the the whole idea of making balls by hand? No. In fact, handmade objects became more expensive as a result.

• 57:02Alex Kantrowitz?

  Right, and that leads me to to a question I have to ask you, which is that everybody's asking about the training data that that these things are using. In fact, I I I put a tweet out asking, hey, I'm gonna talk about generative AI. So we'll end with this generative AI ethics question. What would you like to know? Everybody wanted to know about how the people who these databases or programs are trained on are actually gonna end up being compensated. And whether we should actually even allow them to train on stuff that people have created without their consent. Here's like two of those comments. So Roger McNamee, an early Facebook investor, now now critic, says, and he didn't know it was gonna be a Facebook conversation, a conversation with someone with Facebook. But he says,

• 57:48Alex Kantrowitz?

  "Why should it be legal for computer scientists, entrepreneurs and investors to profit from unlicensed exploitation of the works of creative humans?" And we also had one more from Marshall Marr. And he said, "The inevitably of the inevitability of intrusive tech is a uniquely American phenomenon." I don't know why American, but anyway, this is what he says. "They they asked you to post baby pictures and are now training lucrative AI engines with your images. This was not disclosed at the time. Pay me now." So when you when you think about that, what's your what's your reaction? And how should we think about it?

• 58:21Speaker 01

  It's gonna be a debate for society to figure out, because I don't think the answer is totally clear.

• 58:27Speaker 01

  You know, for example, photography, the invention of photography, shrunk the market for portrait, painted portrait by a lot. It's not like portrait portrait is disappeared, but it certainly reduced the market for it.

• 58:46Speaker 01

  Recorded music reduced the market for performance musicians. And in every instance of those things, there was, you know, collectives of artists to say like, you know, we have to stop this because this is gonna kill our business. They were universally unsuccessful.

• 59:04Speaker 01

  Okay, so you're not going to stop technology, right? Now, the question is a legal one. So if you assume that current legal

• 59:16Speaker 01

  interpretation of copyright, if you want,

• 59:20Speaker 01

  is used, then you cannot let those machines plagiarize. So if you use a generative model that's been trained on whatever, and it produces, regardless of the process, and it produces a piece of art that is too similar to an existing one, the artist that produced that existing one is entitled to sue the person who is distributing this new piece of art and ask for compensation. Now, but what if that piece of art is not copyrighted? That generated piece of art is not copyrighted, so nobody can profit from it. Who are you going to sue?

• 59:59Speaker 01

  You know, is there grounds for suing? Now, if, again, it's a copy? Yes, there is ground for suing, even if you give it for free.

• 1:00:10Speaker 01

  If that piece of art is in the same style as a known artist, but it's not the same piece of, the same painting or whatever it is,

• 1:00:24Speaker 01

  then that's where things become complicated, because human artists are absolutely authorized to get inspired and straight down copy

• 1:00:38Speaker 01

  someone else's style. That happens absolutely all the time in art. And so would it make sense to apply a different rule for, it's called them artificial artists, right? That generate things?

• 1:00:57Speaker 01

  Like, you know, they can get inspired by whatever they see, they've been trained on, but then they cannot just reproduce. So that would be a perfectly fine thing to do. I imagine that a lot of artists would not be happy with this. But...

• 1:01:14Speaker 01

  You know, that's a, there's a definite possibility. Now, uh, perhaps what you might want to do is what, you know, the early internet also authorized or, or put in place, which is that, you know, you can put a little file called, you know, robot.txt or something that tells search engines and, and callers, do not use my content for, for anything like, you know, you can't index it, you can't, uh, use it for anything. So if you're an artist and you don't want your content to be used, you know, lobby for this kind of stuff and this kind of policy to be, uh, uh, to be respected by, by, you know, calling algorithm.

• 1:01:57Speaker 01

  And then, um, and then it could be like, you know, the next step would be,

• 1:02:04Speaker 01

  like, if you use my artistic production as trending data, you owe me money. And I think that's a bit extreme. So, you know, the answer is probably somewhere in between and not clear what like, and it's not for people like me to decide, like, you know, I have no particular legitimacy to like to help people, this is the right thing to do. I'm a scientist, I'm an AI scientist. I think this is for a society as a large, at large to the, to decide through the usual democratic process.

• 1:02:35Speaker 01

  But, and, but you had to be careful, like in one of the remarks that you, you mentioned, it, it said like, you know, why should like a large tech company in California profit from my, my art, me, poor artist? This is the wrong way to frame this because within a year or two, you know, any teenager in their parents' basement is going to be able to do this. So, um, you, you don't want to.

• 1:03:04Speaker 01

  Use the, you know, current bad press that the tech industry has to motivate people to kind of go after this because that's you're going to lose if you do this.

• 1:03:17Alex Kantrowitz?

  Yeah.

• 1:03:17Speaker 01

  This is not something, you know, the big tech companies are just the first ones to have the technology to do this, but eventually everybody is going to be able to use this kind of stuff and train their own models and crawl the internet, right? I mean, you know, countless startups and, and, and, you know, young programmers.

• 1:03:34Alex Kantrowitz?

  Yeah, and it is a point, I made this point in my most recent newsletter, but we are not going to see the centralization, I'm sure you agree, that we saw with, with, you know, communication online, like with the Facebook and the Twitters and AI, it's going to be much more distributed. And the last thing I'll say is that I'm so torn about this because I have been plagiarized by someone using my words, feeding them into generative AI systems and then publishing them as their own. That's already happened to me. However, I just love the systems. I love using DALL-E to illustrate my, my stories and I love speaking with ChatGPT, where, I mean, I have it on speed dial, it's so fun to speak with. So, okay, that, that, that will do it for us.

• 1:04:16Alex Kantrowitz?

  Unfortunately, out of time, Jan, I feel like we can speak for hours each time, but unfortunately, that's the end of the show. So, thank you so much for coming on, we'll have to do it again soon.

• 1:04:25Speaker 01

  Well, thanks for having me, Jan.

• 1:04:27Alex Kantrowitz?

  Okay, great, it was a pleasure. And that'll do it for us here on Big Technology Podcast. Thank you so much, Jan LeCun, for joining, always great to have you on. Hopefully, we could do it again sometime soon. And definitely, I feel like we could use at least two or three hours talking about this stuff. Thank you, Nick Guattini, for handling the audio, appreciate it as always. Thank you, LinkedIn, for having me as part of your podcast network. Thanks to all of you, the listeners. Some programming notes, we're going to have our weekly episode, recapping the news with Ranjan Roy. I know we're already going to talk about the activist investor that's in Salesforce and plenty more on Friday, so that will be live on LinkedIn Friday, a little bit earlier than a normal time, and then dropping here on the feed right after that.

• 1:05:08Alex Kantrowitz?

  So, keep an eye out, the following week, week from today, Brian Stelter, former host of CNN's Reliable Sources, is coming on, that's going to be a fun discussion, I can't wait for that one. Please, if you can, rate and review the show, it always helps other people find the show that helps us get better guests and the show gets better. So, it's a virtual cycle, rating and reviewing, five stars would be great. That's really applicable on Apple and Spotify, but anywhere you can find a set of stars and you want to hit five of them, that would be awesome. Okay, well, that will do it for us. So thanks again for listening and we will see you next time on Big Technology Podcast.

• 1:05:54Speaker 02

  Now, more than ever, we need someone on our side, but Rob Whitman? Whitman's votes drove up costs, cut Medicaid and risked 350,000 Virginians healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the House Majority PAC dot com, not authorized by any candidate or candidate's committee.

• 1:06:23Speaker 03

  In this life, you can't win.

• 1:06:25Speaker 04

  When the world gets you down.

• 1:06:26Speaker 03

  No!

• 1:06:27Speaker 04

  Escape to Pluto TV.

• 1:06:29Speaker 03

  This is objectively fantastic.

• 1:06:31Speaker 04

  Relax into the movies and shows you already know and love, all for free.

• 1:06:34Speaker 03

  Schwing!

• 1:06:35Speaker 04

  Even our news is uplifting.

• 1:06:36Alex Kantrowitz?

  You stay classy, San Diego.

• 1:06:38Speaker 04

  Watch School of Rock, Crazy Ex Girlfriend, Anchorman: The Legend of Ron Burgundy, The X-Files, Wayne's World and so much more.

• 1:06:44Speaker 03

  We're not worthy!

• 1:06:46Speaker 04

  The best things on TV really are free.

• 1:06:48Speaker 03

  You just have to know where to look.

• 1:06:50Speaker 04

  Escape to Pluto TV. Stream now, pay never.