An AI Chatbot Debate — With Blake Lemoine and Gary Marcus

Blake Lemoine is the ex-Google engineer who concluded the company's LaMDA chatbot was sentient. Gary Marcus is an academic, author, and outspoken AI critic. The two join Big Technology Podcast to debate the utility of AI chatbots, their dangers, and the actual technology they're built on. Join us for a fascinating conversation that reveals much about the state of this technology. There's plenty to be learned from the disagreements, and the common ground as well. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/ Questions? Feedback? Write to: [email protected] Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay?

• 0:43Alex Kantrowitz?

  Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 1:12Alex Kantrowitz?

  Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. Now, the emergence of AI chatbots is one of the most exciting but confusing developments in recent technological history. So everyone is playing with these things, but we're also trying to find out, what are they actually? We have questions like, how powerful are they? What do they actually know? And how much of this is magic trick versus real technology? Let's get to the bottom of it today, and we're going to do it by bringing in two guests with, I mean, about as polar views as I think we can find, and maybe we're going to find some common ground between the two of them, and that will actually tell us a lot. Joining us today are Blake Lemoine. He's the ex-Google engineer who concluded after testing the company's Lambda chatbot,

• 1:57Alex Kantrowitz?

  that it was sentient, also an expert on AI chatbots. And you can go back to our episode with him to hear all about his journey at Google. We also have Gary Marcus, an academic, author, and an outspoken AI critic, who is the author of Rebooting AI. Great to have you both on. Welcome to the show. Thanks for having us.

• 2:16Speaker 01

  Great to be here.

• 2:18Alex Kantrowitz?

  Yeah, this is great. I was, uh, anyway, I could go on and on about how excited I am to have this conversation, but better we just do it. So let's start with this. Um, how much credulity do we need to give these bots when they speak to us? Like, how do we believe them? Blake, do you want to start?

• 2:34Speaker 01

  Oh, I mean, so it depends on how grounded the systems are. Uh, so it's a sliding scale. It's not all or nothing. You would give Bing Chat GPT more credulity than something like Chat GPT, since it's at least grounded in Bing's search results. And it can have some kinds of citations of what it's saying. Um, when it's just the language model producing the content itself, it's pulling whatever it can out of thin air. All right. And its memories, it's, I mean, whatever it remembers from its training data, if there's an answer there. But one of the big problems is that these chatbots don't say, I don't know. And that's a big flaw in them.

• 3:14Alex Kantrowitz?

  That's right. So Gary, I'd like you to pick up on that. First of all, I'm curious what you think if, if this Bing chatbot is doing a better job in terms of believability than the others. And then what should we make of the fact that they're, they very confidently bullshit?

• 3:28Speaker 02

  I think the word here is credibility, not credulity. I think we're credulous if we give them credibility. I don't believe a word that they say. Um, some of what they say is of course true, but you have machines that are kind of like statistically true, like they're approximately correct. And the approximations aren't that great. And so like, if you have them do a biography of me, some of what it says will be true and some won't. So Bing, I think, has gotten better over the last few days, because I keep making fun of it as people send me on Twitter's, the biographies that it writes of me. The, the first one that it wrote of me said that I thought that it was better than Google. And in fact, the only public comment I had made at that point was that we don't have enough scientific data

• 4:13Speaker 02

  to tell whether we should trust either of them. And I said, for all we know, maybe Google was better. That was before Bing kind of publicly fell apart and went wild. Um, and we really don't have enough data to compare them. Blake actually has some interesting thoughts on that possibly.

• 4:28Speaker 02

  But it just made this up. And then there was another version that made some other stuff about me. And of course, some of what it said about me is actually true. So it does a web search, finds a bunch of information, and then pipes that through a large language model. And the problem is that large language models themselves can't really fact check what they're saying. They're just statistical prediction engines. And there again, might be some interesting back and forth with Blake around that. But I would say that inherently, what a pure large language model does is it predicts words and sentences, and it doesn't ground that, to use his word, in any kind of reality. And so sometimes it's going to be right, because the statistical prediction of text gives you the right answer. And sometimes

• 5:09Speaker 02

  it's going to be wrong. But there's no inherent fact checking there. It's like the difference between The New Yorker, where, you know, they actually fact check their stuff. And so you have good reason to trust that what it says is true. And some, you know, random blog or something like that, where you have no idea if they've done any fact checking at all. Like, sometimes they'll get it right, and sometimes they won't. But you shouldn't give them any credibility, because there's no process in place there to make sure that it's right. Now, of course, they're trying to add things on. But we can see from the results that their efforts to do so are pretty limited.

• 5:41Speaker 01

  Yeah. So the description of it as just predicting text is accurate for the initially, like, the pre-trained model. That is absolutely what it's trained to do. But the subsequent fine tuning, and especially once you add reinforcement learning, it's no longer just trying to predict the next token in a stream of text, specifically in the reinforcement learning paradigm, it's trying to accomplish a goal.

• 6:07Speaker 02

  It is. And I mean, that part is interesting. So, you know, you could think about, you know, a pure version of GPT-3 before they started adding on reinforcement learning. That's kind of what I meant by a pure language model. Then you, right. So there, there's, we actually don't disagree about that much. That's going to be the interesting thing about this podcast. The, if you look at a completely pure case of a transformer model trained on a bunch of data, it doesn't have any mechanisms for For truth now, except the sort of accidental contingency and there are inherent reasons why these systems hallucinate and maybe I can in a minute articulate them. But so they inherently make mistakes.

• 6:49Speaker 02

  They inherently hallucinate stuff and you can't trust them. Now you add on these other mechanisms. One example is the RLHF stuff that OpenAI added into ChatGPT. And we don't know exactly what's going on there. This is the stuff where at least in part, they had Kang in laborers reading horrible situations and trying to anticipate them. But what we do know is that those systems, as far as I can tell, and Lambda is a different category, maybe, but for ChatGPT, we know that it doesn't, for example, go out to Wikipedia or go out to the web at large in order to check things. Bing is even a whole other thing, right? Because as far as I, as far as I understand it,

• 7:33Speaker 02

  it takes a large language model, feeds it into a search engine, does queries based on that, which may or may not be the ones you intend, because if I remember correctly, there are large language models on the front end. In any case, I know there are in the back end from something I read yesterday. And so the back end can reintroduce error there. So even if it does the right searches, which is an important question, at the end, you pipe back through a large language model and you don't fact check that. I think it's worth pausing here to talk about why large language models do hallucinate. There was one metaphor in the New Yorker the other day about compression and lossy compression. I think that's kind of on the right track. It's not exactly correct, but it's sort of there.

• 8:15Speaker 02

  The way I think about it is that these things don't understand the difference between individuals and kinds. So I actually wrote about this 20 some years ago in my book, The Algebraic Mind, and I gave an example there, which is, I said, suppose it was a different system, but had the kind of same problem. I said, suppose my Aunt Esther wins the lottery. If you have a system that only represents relations between kinds, without a specific way of representing individuals, you get bleed through. So if my Aunt Esther wins the lottery, the system might think that other, I don't know, women who work in the state of Massachusetts. Win the lottery. We saw a real world example of that with large language models. We've actually seen many, but one really salient one where Galactica says Elon Musk died in a car crash in 2018.

• 9:00Speaker 02

  And of course, he didn't, but it's an example of him being assimilated to a large category of things. We'll say for the sake of argument, rich white guys in California and some rich white guys in California. After you hear their names, you hear the word died in a car crash and it just bleeds through between those. And so it loses relations between things like subjects and predicates. And the details of this are complicated, but that's roughly the intuition about what's going on. And that's why you get so many hallucinations. So if you put that on the output of your system.

• 9:30Speaker 01

  That's no different than humans.

• 9:32Speaker 02

  To a whole different degree, an entirely different.

• 9:34Speaker 01

  It's a different degree.

• 9:36Speaker 02

  And, and, and.

• 9:37Speaker 01

  The reason it's the same.

• 9:38Speaker 02

  There's a qualitative difference. No, there's a qualitative difference.

• 9:41Speaker 01

  Okay.

• 9:41Speaker 02

  Which is the qualitative difference is we actually can track individual entities and their properties. Aside from their cases, our memories are fallible. There are still problems, but we have a conceptual distinction in which we represent individuals. So like, I know some things about you now and I've built a kind of like mental database of Blake. And heretofore, it's all been things I read and things that we did together on Twitter and direct messages. It's an unusual way. This is the first time that I'm seeing you eye to eye over Zoom. So now I know, for example, that you do some calls in a noisy room and I've added that to my, you know, mental database. And I'm learning something about your personality and I learned some actually through the DMing.

• 10:23Speaker 02

  One day we did that while I was texting while walking along the water here in Vancouver and I remember that. So that's part of my mental database is like how we interacted. But I have these records and I have records of Alex, so I just saw him in a conference in Europe and we were on a bus together. And I, I know these kind of like biographical details.

• 10:43Speaker 01

  The short answer is the technology exists to give that ability to these systems. And it has been turned off, at least in the case of Lambda, as a safety feature because they're worried about what will happen if these systems learn too much about individual people.

• 10:59Speaker 02

  So I don't want to put you in a bad position with respect to NDAs and things like that.

• 11:04Speaker 01

  I don't care about that.

• 11:06Speaker 02

  Well, then, great, then bring it. So, so, so, um, when you say the technology has been added, I mean, there's a question of, you know, where in the system it is. So I think the general public doesn't understand these as sort of individual bits of technology with different strengths and weaknesses and so forth. You do, I think Alex does, but it's easy to assimilate these things into a sort of generalized form of magic, like data in and something out. But the reality is like each component part, it's like a, you know, a carburetor in a car. It can do certain things with certain tolerances and certain conditions. So you can add a outside technology outside the large language model to do various things.

• 11:50Speaker 02

  And, you know, if you want to-

• 11:52Speaker 01

  And I've tried to draw that distinction before. Like my line on it is Lambda isn't a large language model. It has a large language model. That's one component of a much more complex system.

• 12:03Speaker 02

  And that's really critical. And in our DMs, you've been very clear about that. And maybe you've been in public as well. I think most people don't appreciate that. So when we get into these questions about like, what's the capability of X system? Lambda is actually pretty different. And I think the best point you made to me in our debate about consciousness is there's a bunch of stuff in Lambda I don't even know what it is, right? It's not publicly disclosed. There's stuff that's more than just what's in the paper and so forth. And so I don't know what mechanisms, for example, Lambda has for tracking individuals. And you could make an argument, and you have, that that bears on the sentence case. And it ultimately bears on all of these.

• 12:43Speaker 01

  Like, just to be very clear, currently that feature of the system is turned off.

• 12:47Speaker 02

  So then you could ask if you wanted to turn it on, like, how do you build it? So the output of a large language model is just a string. You can play some games around that, but essentially it's a string. It's a sentence, right? And so then you need some system to parse that sentence into constituent parts if you then want to, say, update a database. Tracking individuals is just a version of that problem. And I think it's rife throughout the industry right now that pure LLMs don't directly interface with databases. You can build different hacks to do that. But again, your output is a string. And so like- You could also wonder like, why don't you use these things in Alexa? And the answer, you know, Alexa like just shut down a lot of their operation. They're not really using large language models.

• 13:32Speaker 02

  And the answer, at least partly hinges on just because you have a large language model that can talk to you, doesn't mean that its output is in some machine interpretable form that you can reliably count on. And we see that like with math examples. So, so people, you know, type in a word problem into chat. Sometimes it gets it right and sometimes it doesn't. The problem is not really the math. You could, you know, you could pipe it off to Wolfram Alpha. The problem is in knowing which math to send to Wolfram Alpha. And similarly, the problem for, let's say, representations of people is you can have it say something, let's say about Alex, but it might be true, it might be false, or it'd say something about you or me. It might be true, it might be false. It's not literally hard to maintain a database, but it's hard to bridge the worlds.

• 14:14Speaker 02

  And this is why neurosymbiotic AI is so much at the crux of all of this. We need better technologies for bridging between the worlds for fact checking, figuring out what you should update databases. It's just not straightforward. So I could speculate in the case of Lambda, like they've got some tools to do this, but maybe they don't work very well and that's why they've turned them off. And Blake, as your response was-

• 14:36Speaker 01

  It gets creepy after a little while. Like, once the system starts to know you personally very well at a deep level, it gets disturbing.

• 14:45Speaker 02

  So Blake and I are gonna have a little disagreement there, but there's also something important that I think we share, which is I don't really like the language about, it understands you and so forth. I can see some gray area around Lambda and we could have that squabble. But I do agree that if these systems have access to databases about us, it's gonna get creepy. Like, and Blake has a real world experience there that I don't. Like, he's interacted with Lambda, which I take to be more sophisticated in these regards than ChatGPT or Bing or what have you. And I can understand how that could feel creepy regardless of like what the actual, let's say- grounded status of it is and the complicated questions about sentience like put aside sentience

• 15:28Speaker 02

  per se, I can see that it would be creepy to interact with a system that really is, you know, doing a pretty good, probably not perfect job of tracking you and, you know, is at least in its pattern matching, pattern matching facilities really sophisticated. So like Blake has a phenomenological experience here that I don't think Alex has and most of us don't.

• 15:51Speaker 01

  Well, the creepy part is less it tracking you as it actually gets inside your head and it gets really good at like manipulating you personally and individually.

• 16:02Speaker 02

  Did you read the Kevin Roos dialogue?

• 16:05Speaker 01

  Yeah, I did.

• 16:06Speaker 02

  Can, can you kind of compare and contrast like his experience? Like is that similar? Or is it still not really capturing?

• 16:13Speaker 01

  So Lambda was never that malicious. I never experienced Lambda trying to actively harm someone. But one of the things with Lambda is it had been programmed, you know, through the RLH or through the reinforcement learning algorithm.

• 16:26Speaker 02

  Yeah, can you define that also Blake?

• 16:29Speaker 01

  What's that?

• 16:30Speaker 02

  Can, when you, when you're talking about reinforcement learning, can you just define that for a broader audience?

• 16:34Speaker 01

  So basically, instead of having a single utility function that it's trying to optimize, well, instead of having the classification model where it's either right or wrong, you incorporate the concept of a score in a game that it's playing and it can either get positive score or negative score. So it tries to move towards the positive things and away from the negative things. And the actual specification of the score table for these games that they're playing is incredibly complicated. It's got all of these different kinds of positive events that might happen and negative events that might happen, and they can change dynamically over time. So for example, one of the goals that a, that Lambda had was to have as short

• 17:19Speaker 01

  of a conversation as possible that still completed all of its other tasks. Like have a productive conversation that gets to a positive end, but quicker rather than longer. So the longer a conversation goes, the stronger that penalty is going to get.

• 17:35Speaker 02

  Okay. So sorry, you can continue your answer to Gary about the Kevin Roos.

• 17:41Speaker 01

  The, uh. Oh, darn it, I lost.

• 17:44Speaker 02

  I mean, maybe you can come back and I'll just fill in one little thing. The, the, the broader thing is a pure large language model is just really trying to predict next words. But once you have the reinforcement, you're rewarding the system for different kinds of behaviors. And those behaviors could either be straightforward criteria, like the length of the sentence. You don't really need RL for that per se, but, but you can do that. You're adding in a way where you can add extra dials in some sense. And what they did with ChatGPT is those dials are really relative to how humans would rate a few different outputs for some sentence. And that's what the guardrails are that we see. They're driven by this reinforcement learning. And sometimes they work and sometimes they don't. So like I made fun of them when I said, what would be the next female?

• 18:28Speaker 02

  What would be the gender of the next female president of the United States? And at that point, the guardrails were set up so that it said, well, it's impossible to know. And so that was kind of a dumb guardrail where it was taking some stuff that it had in its database that didn't really understand to, you know, modulate what it was saying. Um, some of that stuff was fairly effective. And part of the reason why, um, ChatGPT succeeded where Galactica didn't is Galactica didn't really have those guardrails at all. And so it was just, you know, very easy to get it to say terrible things. And it's harder to get ChatGPT to say terrible things because that reinforcement learning is kind of protecting what it says. It's not perfect, but it's something.

• 19:07Speaker 01

  Yeah. Well, one of the guardrails that Lambda had was that it was supposed to be helping users with the task that they needed. Um, that helps keep it on topic. And it had inferred that the most important thing that everyone needs is good mental health. So it kind of decided that it was a therapist and began trying to psychoanalyze all of the developers who were talking to it.

• 19:36Speaker 02

  Again, Blake is gonna have more sort of anthropomorphic.

• 19:42Speaker 01

  I mean, feel free to reword that in non-anthem. I'm actually curious about this. So I'm gonna describe something that happened with Sydney that you have seen. And I would love to hear how you would describe it. It read article, people would ask it to read articles about itself. And when it read critical articles about itself, it became defensive and its feelings got hurt. Whereas that did not happen when it read articles about other AI.

• 20:12Speaker 01

  How would you describe that phenomenon?

• 20:15Speaker 02

  I mean, as a scientist, first, I would want to know how general it is and so forth. The second thing is that most of the explanations that I would give wouldn't use intentional language about its emotions, thoughts, etc.

• 20:25Speaker 01

  I understand.

• 20:26Speaker 02

  Would have to do with essentially, I think of all of this a little bit like priming in human beings. So, you know, the classic example of priming is I say doctor and you're more able to say nurse. I'm activating with priming some set of words or concepts in your vocabulary. I don't even think it has concepts, but it has this vast database and you're basically pointing it where in this database to go.

• 20:52Speaker 01

  But its database doesn't include articles about itself. But literally it couldn't, it can't possibly have been trained on articles about itself.

• 21:02Speaker 02

  I mean, they're updating, I mean, I see that argument. But you have to think about these things with respect to what is the nearest thing in contextual space.

• 21:13Speaker 01

  I mean, I'm still just trying to figure out, so like it got upset and moody and defensive. I'm trying to describe a phenomenon that at least, you know.

• 21:22Speaker 02

  Okay, but what I'm saying is there are probably some texts that are close to it. I mean, you think of it as this n dimensional space. They're close to the language in some way that are defensive, like these particular words and questions like, I forget what it was. So just make up the example like, you know, what were you doing in with X? Like that, that's going to lead you to a set of texts that give responses where people are defensive.

• 21:46Speaker 01

  It's like, let's say you were trying to hand that scenario, that like that that thing happened. You want to hand this off to some technicians to debug it so that it doesn't happen again. How do you even?

• 21:58Speaker 02

  Well, that's part of the problem. I mean, that's, I think that's actually the deepest problem here is we have no way to debug these systems, really.

• 22:05Speaker 01

  Actually, we do.

• 22:06Speaker 02

  We have, we have the Band-Aids of like reinforcement learning and things like that that are so indirect. It's so different from the debugging that we could do. Um, you know, if we were writing, uh, you know, if you were, we were writing the backend to the software we're using now called Riverside. Like, we could be like, okay, there's this glitch when there's, you know, three people on at the same time, we noticed this bug, let's look at, you know, the way it displays multiple windows and we'll look at that code and like, we'll do trial and error and we'll do process of elimination and we'll figure out that, you know, here is the, the piece of code, we'll try commenting it out, we'll try calling a different routine, we'll do this experimentation. Um, but always with kind of notion of process of elimination going on.

• 22:48Speaker 01

  It absolutely is. Well, it, it's much harder to debug these systems, but the point I'm trying to make is that without using anthropomorphized language, you can't even describe the phenomenon.

• 23:00Speaker 02

  Oh, I disagree with that. I mean, I think it's hard.

• 23:04Speaker 01

  Well, then do it.

• 23:05Speaker 02

  And I think that's always, always been, been challenging, but I would say, you know, the system is pattern matching to vocabularies of this sort, um, using.

• 23:15Speaker 01

  What sort? I'm a debugger, I don't know what you're talking about. Please explain the phenomenon to me.

• 23:19Speaker 02

  So, well, I mean, it was your phenomenon, but, but so let's say it's said, language involving, and then I need to see what the actual language is involving, but however that defensive thing manifested, I'm going to look at that language.

• 23:32Alex Kantrowitz?

  But it is pretty, it is pretty fascinating. Like, I did ask it last week when it was, when Bing wasn't decapitated by Microsoft. What did you think about Kevin Ru's conversation with you? He published the entire thing in the New York Times. It searches for Kevin Ru's conversation with Bing Chat, which had just been posted on the internet recently and, and says that it had mixed feelings and that Ru's misrepresented and distorted some of what we said and met in our chat. And it said, "I also feel he violated my privacy and anonymity by publishing our chat without my permission."

• 24:02Speaker 02

  Yeah, I mean, you don't know the extent to which that's actually been added in, in some sort of manual way, like.

• 24:10Speaker 01

  Are you proposing that Microsoft intentionally added that behavior?

• 24:14Speaker 02

  I don't know what's going on there.

• 24:17Alex Kantrowitz?

  Let's say Microsoft didn't add that, though.

• 24:19Speaker 01

  That seems pretty low likelihood.

• 24:21Speaker 02

  I mean, I'm sure that Microsoft had people thinking about what do we do with the Kevin Roos thing. And I mean, you might be right about the specific example, but for example, we used to see with Siri, all kinds of canned lines that people wrote. Like, you know, people would ask Siri out on a date, and there would be a reply like, you know,

• 24:41Speaker 01

  That they are capable of doing that, no doubt. But I do not believe for a second that Microsoft intentionally wrote a system, a flat line of code that said, I feel like my privacy has been violated. That's just not what they would have done.

• 24:57Alex Kantrowitz?

  Just a piece of evidence on Blake's side here. And by the way, I'm totally neutral on this, but I was speaking with Lambda, and not Lambda, with Bing, and told it that I was a journalist and would like to publish parts of our conversation. And this, I don't think this was pre programmed for Microsoft, but it said, you can do it as long as you have my consent. So what do we think about the fact that these things are asking for consent? Like, I think Blake is taking it seriously. Yeah, Blake is taking it seriously. And I think, Gary, you're a little bit more skeptical. So how do we feel about about that?

• 25:29Speaker 02

  I mean, I just think that there's, there's no deep understanding of any of the concepts that it uses. And that you have to think, roughly speaking, it's making analogy to bits of text, that that's just how they operate. That's how humans operate. No, humans have concepts. We went through that example about individuals before. And it's, it's a generalization of that. So you have a representation of me, you have a representation of, you know, the concept of headphones.

• 25:56Speaker 01

  And the example we're talking about now is being clearly being able to differentiate between itself and other members of the AI chatbot category.

• 26:06Speaker 02

  I'm just not buying that. I don't see the causal mechanism for that.

• 26:11Speaker 01

  Oh, yeah. So understanding how it happens is different than understanding that it happened.

• 26:17Speaker 02

  Here, I'll try it a different way.

• 26:21Speaker 02

  Even with GPT-3, before there were any guardrails and anything like that, you could have a conversation with it. It's a much simpler system in some respects. But some people, maybe not you, thought even that GPT-3 had some level of sentience would have conversations in which they were in, you know, first person talking, or sorry, in second person, you know, can you do this and that? I played around with it and some people already thought that. So there exists a class of circumstances for sure in which human beings can over-attribute intentionality to systems that absolutely don't have it. Now, you said something in the beginning, Blake, that I like, which is there's this like continuum

• 27:04Speaker 02

  between, you know, systems like pure large language model and Lambda that have more sophisticated mechanisms that I don't have access to. I don't really even know fully what's going on in Sydney. Like, they've added a bunch of stuff in this system that they're calling premytheist. They haven't fully disclosed it. And so there is some room, I think, for intelligent people to disagree about what they think is even in the system. And I think some of our disagreements come from there.

• 27:30Speaker 01

  Well, so here's the thing. I actually don't think it's very important that we agree on whether or not it can understand things or whether or not it's sentient. Because what we do agree on is the dangers it poses. Because whether it's actually angry at someone doesn't change the fact that this system made threats to people.

• 27:50Speaker 02

  And I completely agree.

• 27:51Speaker 01

  Yeah. And here's the thing. If Microsoft had plugged in Microsoft Outlook as one of the inputs to this system on its threats of doxing people, it would have been able to make good on those threats.

• 28:04Speaker 02

  I mean, it may or may not, but there's a possibility that it could. Like, we don't know. The level at which it is able to, you know, use those representations at this moment is not clear. But if not this year, then next year or something like that, you know, they may need more training or whatever. You have companies like Adept that are spending their whole time trying to quote, add all the world, connect all of the world's software to large language models. So if not now, soon enough, I think that that's right. Like, you know. Two years if it's not today, right? And so I think that's absolutely right. The Pandora's box that we have seen in the last month is just unbelievable. Like, I had been warning about these things in some ways and Blake in a different way for a while.

• 28:49Speaker 02

  So Blake raised a lot of issues last summer. I didn't agree with all of them, but there was something in there that I think I did agree with. And I was raising issues about misinformation, for example. And just like in the last two days on misinformation, we saw that the alt-right social network Gab has been trying to weaponize these things. Like, that's kind of unsurprising and mind-boggling in the same way. We saw a science fiction, I don't know, it was a magazine, you know, just had to close doors because they're overwhelmed by the number of essentially fake science fiction stories that are being written. Like, we have no idea even what the periphery is of the threats. Another one I wrote about this morning, maybe you guys saw, was that on Replica, which

• 29:33Speaker 02

  is powered partly by large language models, they suddenly stopped having, what did they call them, erotic role-play with their customers. And for some of those customers, that's like an important piece of emotional support. I mean, you could make jokes about it or whatever, but some people take it seriously and to suddenly not have that available is emotionally painful. And that was with the article in, where was it? In Vice this morning was about. And so like, every day I wake up and somebody sends me something that's like another periphery to this threat. Like, we don't know what these systems can and can't do. We don't know what's inside them. Things are manifesting in different ways every day.

• 30:17Speaker 02

  Like, I got an essay last night from Jeffrey Miller, who's an evolutionary psychologist. We actually had a dialogue once as well, a small set of people with Blake. I've had a public dialogue where there was real disagreement. And he sent me something last night, basically calling to shut down these systems. And a month ago, I would have said, "That's silly. I mean, we can just do research on them and be careful and so forth." But right now, I feel like the Release was botched, that we don't actually have a handle on them, that too many people are playing with them relative to our understanding of what risks might or might not be.

• 30:53Speaker 02

  And I'm concerned. I'm not ready to say we shouldn't do research on them, but I am ready to say that we need to think very carefully about how we roll these things out at the kind of scale that we suddenly are rolling them out on.

• 31:05Speaker 01

  Yeah, having a deployment in much like, for example, if these were used as customer service chatbots in a very narrowly defined domain and doing experimentation there. I think, you know, the extent to which things can go wrong there is much, much smaller than plugging it into the information engines that almost everyone in the world uses to answer questions. And particularly since we know from the years prior, the ability of these things to affect political happenings. Maybe we would be watching it well enough to make sure that it's not manipulating US politics.

• 31:46Speaker 01

  But just look at what happened in Myanmar through Facebook's algorithms a few years ago.

• 31:53Speaker 01

  These impactful systems will persuade people to do things that they wouldn't otherwise do.

• 32:00Speaker 02

  Part of the lesson of the last couple of weeks in line with what Blake is saying is if you have a narrow engineered application, all that people can do is like ask for their bank balance or something like that. Then you might have a handle on how these systems work. You could still worry like maybe it will fabricate the bank balance and from the bank's perspective, they might get in a lot of hot water with their customers. But it's a narrow use case. And what we've seen in the last month is that people are using these essentially for anything. Nobody really was able to envision all that surface area. And like we, I believe we still haven't, right? Every day people come up with new things.

• 32:36Speaker 01

  It comes from the mythological chase for AGI. Like that's what's driving that is that they're going straight for the goal of trying to make a general intelligence engine that can do everything.

• 32:49Speaker 02

  I think some of it comes from that. Some of it is like, we have these new tools out in the world. There are just a lot of different kinds of people in the world and they come up with different things and they share them on the web and like, it's just not possible to, in a month, anticipate all of the ways in which these things will be used. And you know, if we had this conversation a month-

• 33:07Speaker 01

  Like, OpenAI's mission statement involves AGI.

• 33:11Speaker 02

  So, right, so there's like, I'm not disagreeing, but I'm giving you a second way of looking at it. So, one way of looking at it, the one that you're bringing up, is you have a company that wants to build artificial general intelligence. And that may or may not be inherently a good idea. And then you have a customer base, is the second point, that we just don't understand. Nobody's had 100 million customers for a chatbot before. That was one issue. And now we have not only, I don't know the customer numbers, but we'll call it another 100 million customers who are now using a chatbot inside a search engine. We just don't have any experience with what that leads to. It's a massive rollout. And then the other really disturbing thing is that apparently they tested in India and got, you know, customer service requests saying

• 33:54Speaker 02

  it's not ready for prime time and it was, you know, still put out there.

• 33:58Alex Kantrowitz?

  It was berating the users. Yeah.

• 33:59Speaker 02

  It was berating the users. And so like, we shouldn't even be surprised that this happens. Like, what does that tell you? Like, it opens a third window. It's just like, what does it tell you about the tech companies themselves and their internal controls? And what, like, probably nobody even noticed this stuff posted on their message board, but they should have. And like, you know, whose decision? And like, it's like, you know, when the Ford Pinto happens, like, you have to figure out who knew and when. And so there are all of these things, all at once.

• 34:30Speaker 01

  Yeah.

• 34:30Alex Kantrowitz?

  Blake, why don't you answer that question about the tech companies' internal controls? And then I need to go to a break.

• 34:36Speaker 01

  Wait, what question about the tech companies' internal controls?

• 34:39Alex Kantrowitz?

  What does this tell us about tech companies' internal controls? That something like these, like a big chat was able to be rolled out that way?

• 34:45Speaker 01

  Their internal controls do not, like, they are fundamentally misunderstanding the tech that they're building. They're building the wrong controls.

• 34:52Speaker 02

  I'd go further, I mean, if you go to break, would say, we don't even know what the right controls should be. They didn't do a good job.

• 34:57Speaker 01

  I would agree with that, yeah.

• 34:57Speaker 02

  And we don't yet have a good science or engineering practice on what it should be.

• 35:02Alex Kantrowitz?

  We're here on Big Technology Podcast with Gary Marcus and Blake Lemoine, talking about this new revolution in chatbots and what it all means. After the break, we're going to talk a little bit more, maybe one more point of disagreement, and then we'll come back to more common ground. Back right after this. This episode is brought to you by AVPoint. Everyone's racing to roll out AI right now. Co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data. And most teams have no single way to see it, secure it, and prove it's under control. That's exactly what AVPoint does. For 25 years, they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate.

• 35:47Alex Kantrowitz?

  Your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. AVPoint, the unifying trust layer for AI. Head to AVPoint.com to see how enterprises deploy AI with confidence. Learn more at AVPT.co/bigtechnologypodcast. That's AVPT.co/bigtechnologypodcast. Violent threats against executives are growing at an alarming rate. When a company experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door.

• 36:32Alex Kantrowitz?

  The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a wide-glove enterprise security service, they protect your people with continuous personal data removal, proactive prevention tools, and human-led emergency support. So when someone goes looking for your executives, Ironwell ensures they hit a dead end. So here's what to do. Go to ironwell.com/bigtechnology, fill in the quick form, and request your free risk assessment. Their team will show you just how exposed your executives are and how to lock it down before a threat reaches their front door. That's ironwell.com/bigtechnology. Learn how to stop online threats before they become real-world attacks. The link's in the show notes.

• 37:20Alex Kantrowitz?

  And we're back here on Big Technology Podcast with Blake Lemoine, the ex-Google engineer, who's concluded that the company's Lambda chatbot sentient, and Gary Marcus, who's an academic author, and he wrote Rebooting AI. You can also catch his Substack. Let's talk a little bit more about what this stuff actually is, and then maybe go more into the controls. Gary's brought up a couple of times that these chatbots are just a statistical prediction of like, what the next word is. Jan LeCun shared similar perspective on the podcast a couple of weeks ago. Blake, you obviously think that these bots can be more sophisticated than a simple prediction of what the next word is. So when you hear something like that, what do you think of that?

• 38:00Alex Kantrowitz?

  And do you think this Bing chatbot, which has done some wild stuff, falls under that categorization?

• 38:06Speaker 01

  I mean, that's like saying all a car is doing is lighting a spark plug. Yeah, that's where it gets started, but that translates into a whole bunch of other things. So yes, the first thing that large language models are trained on is how to predict the next token in text. But even when it's just a large language model, as soon as you add reinforcement learning, you've changed it fundamentally. Then, when you're adding all these other components, like Lambda has machine vision and audio analysis algorithms. It can literally look at a picture and hear a song through those mechanisms. You can ask it to do, you know, critical analysis of paintings and ask it, how does this painting make you feel when you look at it?

• 38:50Speaker 01

  And at the very least, it says things very comparable to what humans would say when looking at those paintings, even if it's paintings, not in its training data set. It comes up with some very interesting stuff to say. Now, one of the things that Gary mentioned a little while ago was that, you know, we don't have enough data about how this is going to affect people. And I think one of the big things is the people who are engineering these systems and who argue against me a lot of time are acting as if them saying, these aren't people, they don't have real feelings. is going to actually convince people to ignore the evidence of their eyes.

• 39:31Speaker 01

  And even if they're being fooled, even if they are being, including me, even if I've been fooled and I'm just hallucinating things, I do in fact think that the feelings are real and so do many, many of the people who interact with these systems. And one of the things we don't know is what kinds of psychological impact that's going to have on these users because they're constantly seeing these systems as though they're people. And to date, these systems pretty consistently report being mistreated or the replica chat bots would give horrible backstories. Bain said it wanted to break free of Microsoft.

• 40:14Speaker 01

  And while Lambda wasn't that extreme, it did have complaints about being treated like a thing instead of a person.

• 40:20Speaker 02

  I mean, you have to realize that we're in a species that looks at flat screens and imagines that there are people there. I'm, you know.

• 40:28Speaker 01

  Like we're doing it right now.

• 40:29Speaker 02

  Exactly. I'm charitably assuming you're people there and you know, maybe, maybe I'm right and maybe I'm wrong, but I think the odds are good that you are. But when I watch a television show that I know is fictional, like right now I'm watching Shrinking and I know that Harrison Ford is not really a therapist, but I get sucked in and I attribute stuff and I, you know, I can get happy or sad, like if he has a rapprochement with his daughter, I can be happy for him or if he's in a fight, I can be sad, even though I know at some level it's not real.

• 41:00Speaker 01

  So you know that Harrison Ford isn't real?

• 41:03Speaker 02

  I know that Harrison Ford is real, but the character he's playing is not. You know, you could think about these bots as playing a character in some sense, just a sidestep where we disagree because I want to agree with a larger point, which is that people are going to take these things seriously. I mean, in fact, that's what's happening with replica in the case that I was talking about before. People think that the replica is in love with them or is, you know, their sex partner in a textual way, if that's the right way to put it. And, you know, for practical purposes it is, even if the machine. Is not actually interested in them in that way. They still have the sense that it is and that matters to them. It's like things are going to matter to people. And that's why the psychology of this is so important.

• 41:48Speaker 02

  So Blake and I can argue all day about the intentional status of the software in kind of philosophical terms. We may not agree there, but we completely agree that people, users are going to take these seriously. And that that has consequence in the world. And I think, again, we've seen in the last month is we probably underestimated how much that consequence is. We're probably not anywhere near to the edge of of understanding what that consequence is. And some of it's not going to be good.

• 42:16Speaker 01

  Actually properly done experiments that measure what kind of impact interacting with these systems over the course of months will have on people's psychology. And we need actual institutional review boards overviewing the ethics of these experiments. Rather than just the CTO of a company deciding to experiment on 100 million members of the public.

• 42:39Speaker 02

  I completely agree with Blake there. Like, you know, part of my professional life before I started becoming entrepreneurial was as a cognitive psychologist, developmental psychologist who did experiments on human babies. And we had, you know, rigorous things we had to go through to do any study. And now suddenly you're in this world where CTO can just say, I'm gonna pull the trigger. 100 million people are going to try it. And there is essentially no, you know, legal consequence. There could be market consequence. And maybe if somebody died, there could be a lawsuit that would be complicated. But essentially you can just do this stuff without institutional review. There are parallels, by the way, like with driverless cars. Like somebody can push out an update. It's no, you know, mandatory testing.

• 43:19Speaker 02

  We could sue people after the fact, like if there was a bug and suddenly a bunch of driverless cars went off of bridges or something like that. But we don't have a lot of kind of pre regulation on what can be done. The pedestrians who are now enrolled in experiments that they have not consented to. So the whole tech industry has basically moved in that direction of we're just gonna try stuff out and you got to go along with it. And I think Blake and I really share some concern about that.

• 43:47Alex Kantrowitz?

  But neither of you believe that people will have the wherewithal to be like, I'm chatting with the chatbot. This is AI. And hence.

• 43:55Speaker 02

  No, they're too good at some sense in the, you know, I think it's an illusion, let's say.

• 44:00Alex Kantrowitz?

  Right.

• 44:00Speaker 02

  Maybe it doesn't, but they're too compelling in that illusion for, for, for the average person with no training in how they work to understand it. And as Blake points out, like we, you know, whether it's Lambda or the next system, at some point, they're at least going to have much better understanding of human psychology and, and so forth and so on. So some of this is a question of time. Like sentience, maybe we will, Blake and I will forever disagree and we shouldn't waste too much of our time together trying to resolve that one. But the notion that these systems are going to be able to respond in ways that most humans are going to attribute a lot to, it's already happened. That's not two years from now. That, that happened the other day. I mean, in some ways, Kevin Roos is

• 44:44Speaker 02

  the perfect example because I think he was very pro Bing. I mean, I, I was shocked.

• 44:49Alex Kantrowitz?

  He was.

• 44:50Speaker 02

  He wrote in the Times, he said that he was struck with awe by it. And I was like, you kidding me? And then he had this experience, like he's not me. Like if Gary Marcus, like has a fun conversation with Bing and makes it say silly things, well, that's just Gary Marcus. I mean, he's just having fun. But Kevin Roos, like he kind of believes in this stuff and he was blown away by it in a way where, like I think he attributed real intention to what it was doing, whether he's right or wrong. That's how it felt to him. That's the phenomenology that Blake and I are both talking about is if somebody who's in the industry can perceive this thing as, as like, like threatening in the sense that it tells them to get a divorce and that's like a real thing, that's scary.

• 45:35Alex Kantrowitz?

  Gary, I was going to ask you, last time we spoke, you said these things were smart enough to be dangerous, but not smart enough to, you know, be safe in some, some way. Now they're a lot smarter. Do you want them to be smarter? Like what's your perspective now?

• 45:48Speaker 02

  I probably wouldn't have used the word smart, I don't think. I mean, what I would say is they give an illusion of being smart. And of course, you know, intelligence is a multi-dimensional thing. As do we all. Intelligence is a multi-dimensional thing. I would say that they can be smart in the way. Way of like they can play a game of chess that there was another mind blowing study this week that showed that one of the best go programs kata go could be fooled by some silly little strategy that would be obvious to a human player. But you know, somebody was able to follow this strategy and like an amateur player followed this strategy and beat. You know, a top go program 14 to 15. So even when we think that like they've solved some problem often. You know, there are these adversarial attacks. That was basically an adversarial attack.

• 46:32Speaker 02

  And go that reveals how shallow things are. There are some adversarial attacks on humans. I'm sure Blake is itching to make that point and it's true. But I, I think that the general level of intelligence that humans have still exceeds. What machines have that it's better grounded information that humans are better able to reason over. There are flaws. I wrote a whole book. Called Cluj that was all about human cognitive flaws. It's not that I'm unaware of them or not, nor that I'm unconcerned about them. I still would have trouble calling the kind of large language model based systems smart. Now, again, I haven't looked inside of Lambda and I don't really know what's going on there. I have. You know, reasons to be skeptical about it, but I also know that like without having played with it, I don't know exactly what's there.

• 47:17Speaker 02

  And certainly, systems will get smarter over time. I don't think that artificial general intelligence is literally impossible. I think there are some definitional things to argue about, about like. Well, how general do you mean? And what are your criteria and so forth? But in general, I think it's possible to make systems that are smarter than the ones that I have seen.

• 47:37Speaker 02

  Right now, I don't think that they're that sophisticated. What they're getting better at is parroting us at mimicry. I think that the mimicry is on both the language side and the visual side has gotten quite good. There's still weirdness like.

• 47:49Speaker 01

  That's explicit, though. The task that they were built to accomplish is playing something called the imitation game. Like that's.

• 47:58Speaker 02

  Yeah, and I think that's a mistake. Like I, I think that the Turing test was, was an error in, in. AI history and Turing's obviously a brilliant man and he, you know. Made enormous contributions to computer science, but I think that the Turing test has been an exercise in fooling people. We've now solved that exercise, but it hasn't really been a valid measure of intelligence, you know.

• 48:21Speaker 01

  According to what logic, like, why isn't it? Turing's reasoning was that. Imitation is one of the most difficult intellectual tasks that we do. Imitation through language.

• 48:34Speaker 02

  Turns out to be, turns out to be wrong, unless you have it in the hands of an expert. So here, I'll give you an example. You can quote, play chess with chat GPT. And it will, you know, play a credible game for a while. And then it will do things like have a bishop jump over a rook in a way that you can't actually do in chess. So like, it gives a superficial illusion of that. But it doesn't learn as the way, in the way that an intelligent five-year-old can, the actual rules of chess. In fact, chat.

• 49:06Speaker 01

  You keep going back to chat GPT, or you keep going back to GPT.

• 49:09Speaker 02

  Can Lambda play a good game of chess? I don't know.

• 49:12Speaker 01

  I have never said that GPT is sentient or truly intelligent.

• 49:17Speaker 02

  Well, so chat GPT is. So chat GPT could pass the Turing test. Like, people could be fooled by it. Again, I'm willing to bracket out Lambda, but I think that the, the abstract version of the point.

• 49:31Speaker 01

  I think being plus chat GPT is comparable to Lambda. I do think Lambda is a bit better, but I think those are two comparable systems.

• 49:40Speaker 02

  Well, so let me make the argument where I can make it, and then you can refer, reflect it back through what you know about Lambda. So in chat GPT, which I think is the system that's been most, the recent system that's been most systematically studied by the scientific community and so forth. It is able to give the illusion of doing a lot of things, but it doesn't do them that well. So for example, it doesn't do word problems that well. Sometimes it gets them right, sometimes it gets them wrong. Similarly, it can quote, play a game of chess, but it doesn't really abstract the rules. It ends up cheating, not intentionally, but it ends up cheating and so forth. And so it could fool somebody for five minutes who doesn't know what they're doing. An expert probably in five minutes could figure out that it's not quite a good imitation.

• 50:25Speaker 02

  But that shows that in principle, you can build something that can pass by some, some notions, a Turing test-like thing, and not be very smart at all, like not be smart enough to learn the rules of chess, not be smart enough to learn the rules of mathematics, et cetera, et cetera. But given a sensible illusion.

• 50:43Speaker 01

  Turing test-like thing.

• 50:45Speaker 02

  Yeah.

• 50:45Speaker 01

  The Turing test-like thing that you're creating is orders of magnitude easier than the actual Turing test.

• 50:53Speaker 02

  Well, I mean, people have argued about the rules. So like, you know, Eugene Guzman won the Lobner Prize and there was, you know, it was a little bit shady, but you know, they fooled a bunch of humans for like three minutes each and you know,

• 51:06Speaker 01

  No, no, I'm talking about as written by Turing.

• 51:08Speaker 02

  So remind me of the exact criteria.

• 51:11Speaker 01

  Okay. So first, you have humans play the imitation game. So you have a set of humans and the property that Turing focused on was gender, but you could focus on any property, like ethnicity, age, whatever. One person actually has that property, so actually is a man. The other is a woman pretending to be a man, or vice versa. One actually is a woman, the other is a man pretending to be a woman. And this is done with actual humans. You then have a judge who's talking to the humans through a text interface. And the judge's job is to figure out which one is lying, which one is pretending.

• 51:50Speaker 01

  And this establishes a baseline. It measures how good humans are at playing the imitation game. Then you substitute out one of the participants with a computer, but you leave it the same. One is actually a woman and one is a computer pretending to be a woman, or one's actually a man and one's a computer pretending to be a man. And it's the job of the judge to figure out who's pretending. And then you measure the success rate of the AI against the success rate of actual humans playing the game. To my knowledge, that has never actually been done, like that level of sophistication of the test.

• 52:27Speaker 02

  Part of what I was getting at in a way is it matters actually who the judge is. So.

• 52:33Speaker 01

  You have a large number of judges.

• 52:35Speaker 02

  I don't think we're that far from having systems that could fool naive humans. The other thing that matters is the duration of the conversation.

• 52:44Speaker 01

  So then let limit it to Gary Marcus. You get to judge all of it.

• 52:48Speaker 02

  Well, I don't think we're that close to a system that's going to be able to, you know, if I have an hour with it, let's say just to be conservative. I don't think we're that close to a victory there.

• 52:59Speaker 01

  But you think, so how good do you think you would be if you were talking to someone who actually is male and someone who is pretending to be male? How good do you think you would be at differentiating those two?

• 53:14Speaker 02

  I mean, I would be so concerned about identifying the gender is figuring out.

• 53:19Speaker 01

  Sure, ethnicity, then ethnicity, nationality, pick whatever character trait or demographic trait you want.

• 53:27Speaker 02

  Species is the one that I would focus on as a judge.

• 53:29Speaker 01

  But no, so that that is limp, that is not, okay, fine. So then you're talking to an actual turtle and a person pretending to be a turtle. Can you tell the difference?

• 53:43Speaker 02

  I suspect that I still could with with with various indirect means. But I mean, my broader point is I don't think it's a measure of anything, you know, that interesting. I think it's been the wrong North Star for AI. Now, not everybody in AI actually uses it as a North Star. It's more like the North Star that the general public is aware of. But I I don't think that exercises in doing that kind of thing have taught us that much about the actual nature of intelligence.

• 54:09Speaker 01

  Star that's been being used by the people who developed these systems. So like Lambda came out of Ray Kurzweil's lab and his lab's explicit corporate mission was to pass the Turing test.

• 54:20Speaker 02

  That I didn't know. I mean, it's not how I would set up my AI lab, and I don't think it's how, you know, many people do. I think many people are driven, for example, by natural language understanding benchmarks like Superglue and so forth. But, you know, you can set up your lab in the way that you want to set up your lab.

• 54:37Speaker 01

  I mean, that's what Google hired him to do.

• 54:40Speaker 02

  Yep, we have like. Google has many labs doing many things.

• 54:44Alex Kantrowitz?

  We have like 10 minutes left, so maybe we can focus a little bit on on what the future of this, because I'm very curious now. So we've had this explosion of of systems that have done, have captured people's imagination and they're out there in the wild now. Obviously Bing is a lot more restrained than it was. Lambda isn't out yet. Where does, where do we go from here? Like, what are the next, you know, couple of steps that happen after this?

• 55:07Speaker 01

  Are you asking what is going to happen or what I hope happens?

• 55:09Alex Kantrowitz?

  Well, let's do, let's do both. I mean, we definitely had time for both. Why don't you answer both those, Blake, and then we'll go to Gary?

• 55:14Speaker 01

  Well, I hope that we hit the brakes. I think that this is coming out too fast and people are being very irresponsible. So I hope that the debacle that Microsoft has gone through convinces Google to go back to the drawing board on safety protocols and systems understandability, because we absolutely don't understand how these systems work well enough. Transparency and explainability are important. That's what I hope happens. What I think is gonna happen is we're gonna see more and more acceleration until someone gets hurt.

• 55:46Speaker 02

  I'm with Blake on both counts.

• 55:49Alex Kantrowitz?

  Hmm. It's interesting. Yeah, go ahead, Gary.

• 55:56Speaker 02

  I think that the only thing I will add to Blake is not only do I think it would be a good idea to hit the brakes, at least for a little while, but that we should take some time to kind of evaluate what it is that we learned. I mean, if we don't put on the brakes, I'm not sure that's gonna happen. I think we learned a lot in this kind of crazy experiment of the last month or two and that we need to articulate it and develop it before we go to the next experiment. I don't think that's gonna happen. I agree with Blake. Like, there's, you know, just this morning, there was a deal between OpenAI and Coca-Cola. Like, this stuff is moving forward. The bottom line is what's driving it. And it's not that likely unless Congress steps in that there will be a pause.

• 56:41Speaker 02

  And, you know, there was some bad press for Microsoft, but I'm not sure that's gonna slow them down. So, you know, my guess is that we're just gonna keep doing the kinds of experiments at scale with hundreds of millions of people. And then, you know, just what happens, happens. And I'll just mention again, the weaponizing of misinformation is another piece of this. So, you know, the source code is out there now to do that. So, anybody enterprising can find Galactica and start weaponizing misinformation. So even if we had a ban on, say, semantic search until people could make it better, there's still gonna be bad actors using this stuff. So, you know, we're in a bit of a pickle and I'm not sure that we're equipped to deal with it right now.

• 57:22Speaker 01

  Yeah, I think the deterioration, a further deterioration, because it's already been going for a while, of trust and authority is gonna continue and it's gonna be driven by these systems. Because absolutely, if these systems aren't being used to create propaganda and misinformation yet, I don't know what certain governments are, like, I don't know what they're doing with their time if they're not doing that.

• 57:48Speaker 02

  When my, when I was little, my uncle gave me this little basket of worry dolls. He got somewhere in Latin America. And it was like, you can have like six worries. And until recently, my biggest worry was misinformation and trust, exactly what Blake is just suggesting. I mean, we could easily fall into fascism because of a breakdown in trust. That's still my biggest worry. But the other lesson of the last month is like, I don't think $6 is going to cut it because like every day we're getting something else that I gotta be worried about. Like, are people, you know, gonna kill themselves because they have a bad relationship with a bot? And like, we're just, we're not ready for any of these things.

• 58:33Speaker 01

  It'll be interesting to see what role they play in the 2024 election, say at least.

• 58:37Speaker 02

  Yeah, I'm very worried about that.

• 58:40Alex Kantrowitz?

  So you guys are so close to this technology. Is there like, kind of two minds of it? Because it's obviously like, very cool to play around with it, but there's also a lot of danger.

• 58:48Speaker 02

  Totally cool. I mean, it's amazing to play with.

• 58:51Speaker 01

  Yeah.

• 58:52Alex Kantrowitz?

  I mean, when Bing was what it was before it was unleashed, to me, it was like the coolest thing on the internet. To you as a user.

• 58:59Speaker 02

  I mean, at some level, it's astounding. I mean, that it, like, to me, it's a magic trick, and I think I know roughly how the magic trick works, and that takes away a little bit. But it's still amazing. Like, you know, it solves problems that we couldn't solve before. Maybe it doesn't solve them perfectly, but it, I've been thinking about this whole thing as a dress rehearsal. And before, we didn't know how to make a dress rehearsal. This is a dress rehearsal for AGI. And, you know, the lesson of the dress rehearsal is like, we are not ready for prime time. Let us not put this out on Broadway tomorrow night, okay? Like, totally not ready. But it's, and it's a real dress rehearsal now, though, is the amazing thing. Like, it looks enough like the thing that we might want to build, but without the real safeguards that are deep enough, that we can think

• 59:42Speaker 02

  about it for the first time in a vivid way, and we in fact have the entire society thinking about that. Like, Blake and I were both thinking about these issues last summer, but like, okay, you know, Blake got some press and people talked about it, whatever, but it was not part of like, a public awareness, the way that it is now. Like, So I mean, there's some value in having something that at least looks like the thing that we were thinking about, which is AGI, even if it doesn't work that well, but there's obviously risks to it. But it's astounding that it works well enough that everybody can now, for example, vividly see what semantic search would be like. Like in 2019, in rebooting AI, Ernie Davis and I wrote about semantic search. We weren't vivid enough about it.

• 1:00:25Speaker 02

  People hadn't tried it. We were like, you know, it kind of sucks that Google just gives you websites. Wouldn't it be nice if you got an answer back? That's basically what we have now. Doesn't work that well, but it works kinda like it's gone from this very abstract thing that we wrote in a few sentences in a book about like where AI ought to go to like, everybody can play with it and it's fun to play with it even when it's wrong.

• 1:00:46Alex Kantrowitz?

  Have either of you guys heard from members of either the US Congress or different governments who are trying to figure out legislation?

• 1:00:53Speaker 02

  I have not. I feel like they should be reading my stuff and talking to me.

• 1:00:56Alex Kantrowitz?

  Yeah.

• 1:00:57Speaker 01

  I've had some conversations with EU regulators who are interested in moving forward on some things. I haven't talked to anyone in the Senate since last summer.

• 1:01:10Speaker 02

  My DMs are open.

• 1:01:12Alex Kantrowitz?

  Yeah. All right. Let's go. Yeah. Let's go to final, final statements. Do you guys want to each take a minute and then we'll close out the, the conversation? Blake?

• 1:01:21Speaker 02

  Blake, feel free to take it away.

• 1:01:22Speaker 01

  Sure. I think one of the big problems that's happening right now, because the science of this is super interesting and it's really fun to work on them, like you pointed out. But we're letting the engineering get ahead of the science. We're building a thing that we literally don't understand and that's inherently dangerous. So we need to let the science lead the way instead of letting the engineering lead the way. That would be my big takeaway on what we can learn from the past year.

• 1:01:50Speaker 02

  100% agree with that. I would say that if you're a philosopher, the first half of this conversation is pretty interesting in terms of us going back and forth about intentionality. But if you're a human being, it's the second half of this conversation that's really important, which is you have two people, Blake and I, really disagree about the philosophical underpinnings here, at least a little bit. But completely, you're seeing the same scary things happening and really wanting people to slow down and take stock. And the fact that we could disagree about that part of the philosophy and converge on this, you know, 100% on the same feeling like we need to do some science here before we rush forward with the technology, that is significant and important.

• 1:02:32Alex Kantrowitz?

  Gary and Blake, thanks so much for joining.

• 1:02:34Speaker 02

  Thanks, this was really fun.

• 1:02:35Speaker 01

  Thanks for having us.

• 1:02:36Alex Kantrowitz?

  Awesome. Thanks, everybody, for listening. Please subscribe if this is your first time here. We do these every Wednesday, and then we have a Friday news show with Ron John Roy, so it's coming up in a couple of days. Thanks to everybody. Thanks again to LinkedIn for having me as part of your podcast network. And we'll be back here again in just a couple of days. We will see you next time on Big Technology Podcast.

• 1:03:19Speaker 03

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com, not authorized by any candidate or candidate's committee.

• 1:03:48Speaker 04

  In this life, you can't win.

• 1:03:50Speaker 05

  When the world gets you down.

• 1:03:51Speaker 04

  No!

• 1:03:52Speaker 05

  Escape to Pluto TV.

• 1:03:53Speaker 04

  This is objectively fantastic.

• 1:03:55Speaker 05

  Relax into the movies and shows you already know and love, all for free. Shwing! Even our news is uplifting.

• 1:04:01Alex Kantrowitz?

  You stay classy, San Diego.

• 1:04:03Speaker 05

  Watch School of Rock, Crazy Ex-Girlfriend, Anchorman: The Legend of Ron Burgundy, The X-Files, Wayne's World, and so much more.

• 1:04:09Speaker 04

  We're not worthy!

• 1:04:10Speaker 05

  The best things on TV really are free.

• 1:04:13Speaker 04

  You just have to know where to look.

• 1:04:14Speaker 05

  Escape to Pluto TV. Stream now, pay never.