Is AI Dangerously Overhyped? — With Gary Marcus

Gary Marcus is the author of Rebooting AI and an artificial intelligence entrepreneur who's a loud critic of many of the field's biggest promises. Marcus joins Big Technology Podcast this week to discuss the high profile breakthroughs such as LaMDA and Dall-E, and explain why putting too much faith in the field's ability may be dangerous. We begin with a discussion of the AI-generated art piece that won a competition in Colorado last week. Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay?

• 0:43Alex Kantrowitz?

  Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 1:03Speaker 01

  LinkedIn Presents.

• 1:19Alex Kantrowitz?

  Welcome to the Big Technology Podcast, a show for cool-headed nuanced conversation of the tech world and beyond. Now, in late July, we had Blake Lemoine on the show. He is, of course, the Google engineer who believed the chatbot that he was speaking with Lambda was sentient. This week, we're going to bring on a different perspective. Gary Marcus is the author of Rebooting AI. He's one of the most influential voices in the AI field, the founder of Geometric Intelligence, which was acquired by Uber. And he wrote that Lemoine's perspective was nonsense on stilts, to quote his post on Substack, word for word. So we're going to get into that. We're going to talk a little bit about what you can actually do with the Lambda technology, why he doesn't think it's sentient, what it might take to actually get to sentient,

• 2:02Alex Kantrowitz?

  what that actually means, different perspective than Blake had. And of course, we'll go into this data AI today because it does seem like the field is booming. And it'll be fun to discuss what is happening in it with Gary and maybe hear a little bit of a different perspective than you hear typically. So with that, I want to welcome Gary to the show. Welcome, Gary.

• 2:21Speaker 02

  Thanks a lot for having me.

• 2:23Alex Kantrowitz?

  Thanks for being here. I definitely want to get into the Lemoine stuff. People who listen to the show, you know, basically listen to an hour and a half of him. You know, speak about his interactions with Lambda with me, and I thought it was pretty fascinating. Obviously, the conversation doesn't end there. But before we get there, I'm kind of curious. Last week, there was this really interesting. Situation at a Colorado State Fair where this guy, Jason M. Allen, entered an AI drawn. Painting into the art contest there and actually won first place. And it's caused this whole. Big controversy among artists saying that he's a cheater and he's like, I'm not backing down. I followed the rules. Curious what you make of the whole situation. What do you think it says about AI that, you know, now people can.

• 3:06Alex Kantrowitz?

  Use a prompt. He basically said, you know, draw this and the AI drew it. So what do you think it means that people can just use a prompt and now it's winning human art contests?

• 3:16Speaker 02

  I think we're in a whole new world on that score. You know, later we'll talk about. Some of my skepticism in using AIs for some purposes, but there's no question. That you can get a whole breed of recent software to draw amazing paintings or things that look like paintings. And society has to sort out what it thinks about. I mean, it's sort of like a performance enhancing drug, right?

• 3:38Alex Kantrowitz?

  Right.

• 3:39Speaker 02

  And it's untraceable in general, you know, I don't, I don't know the details in this particular case and how people found out, but. In general, people are going to be able to use these techniques. In, you know, the 1970s, people started using drum machines. It started doing all kinds of stuff with electronic music. And now, you know, in the studio, if you're doing music, you can. You know, change notes to make them have the right pitch. You can change the timing in subtle ways and stuff like that. In general, in music, we just care what we hear and we don't really care how the sausage was made as long as it's entertaining. And maybe people will take that attitude in art, maybe they'll they'll be upset about

• 4:20Speaker 02

  it. You know, my expertise is really in what can the AI do and not so much in the ethics of attribution and so forth.

• 4:28Alex Kantrowitz?

  Right.

• 4:29Speaker 02

  If you talk about another domain like language synthesis, it turns out that current systems can make very convincing language, but it's often bullshit. That doesn't matter in the same way in art. So in if you have a system make up a news story, even if it's trying to be truthful, it'll probably drop in some stuff that isn't true. And, you know, we expect our news stories to be true. In the case of these artworks, if the thing doesn't do what you want it to do, you can say, well, I was just trying to be surrealist or whatever. There's no fact of the matter, the way that there might be in an essay. And so then it's up to society, you know, how do you want to treat these things? They are going to extend the reach of artists, just the way that, you know, having eight-track tape extended what the Beatles could do. I mean, somebody might have said, somebody

• 5:14Speaker 02

  probably did say, you know, you can't do this in a live performance. What is this? Like they're using the studio as an instrument now that, you know, some people will make that argument around computers. I don't have a strong stake there. I'm happy to tell you, you know, what I think is plausible and where the systems might break. I don't think I'm qualified to say, you know, should this be legit? I think that's probably going to depend on what you want your competition to be about.

• 5:40Alex Kantrowitz?

  Right. But, you know, you're living in the world of AI every day. So, you know, I think that like we'll get into some of the other stuff, but it is interesting to hear your perspective on this stuff. One more question on that. The guy wins the art contest, but his art is actually, you know, AI drawing a painting based off of all this other paintings that it's ingested. Is that original work? Where do you, what do you mean? You know, where do you fall on that?

• 6:04Speaker 02

  The analogy is a little bit to sampling and it's it's almost like a sampling on steroids. So we have licensing requirements in music and so forth. And people do, you know, they'll drop in a sample from an old Police song or something like that. And they'll have to pay royalties for it and so forth. And what these systems are doing is kind of like an amazing enhanced version of sampling where you don't even recognize what the samples are anymore. It's all derivative. Now, you know, there are always arguments about this in the arts anyway, like Dylan will say, there's nothing new under the sun, and I just put it together in a new way. It reaches a different level where a system might have access to 600 million pictures.

• 6:47Speaker 02

  And it's difficult for the artist to say, what is the relation between those 600 million pictures that this system saw? And the thing that I got out of it when I gave it a prompt, I said, you know, draw me a picture of a piano keyboard with clouds around it. And the system is drawing on this database of existing, you know, clouds and pianos and so forth, and we really don't know the relation. Of course, we don't know that with a human artist either, but there are certainly, you know, copyright questions to consider. And it's worth realizing that the art system doesn't really understand what it's doing. It's just correlating words with images in its database.

• 7:30Speaker 02

  It doesn't have the same intentionality about the objects as a person does. But it can still be a very effective technique. And, you know, it's here and we're gonna have to grapple with it. I think it probably will change art. I think in general that humans are still gonna be like the creative inspiration and are often gonna be filtering things. So you have the system, it's gonna create eight different choices, maybe the person likes one of them, you never even see the other seven. And so some of it is like, it's like monkeys and typewriters, and then you've got somebody at the other end looking what the monkeys made. And, you know, the monkeys weren't that clever, but but somebody was clever to pick this one thing that came out of this one monkey's typewriter, and they say this is great.

• 8:14Speaker 02

  So it's complicated. I don't know if there's any magical answer, but I do think it's the new reality is that AI is going to enhance the palette that's available. We had the same kind of questions with Photoshop before, right? Right. So, you know, I do photography, and I almost never do anything more than tune the color a little bit. There are other people that, you know, the images that they share have been completely redesigned. It's similar to that, right? You have power, you can also think of these things a bit analogously to to filters in Photoshop or something like that. You know, they're creative tools to push you harder. You always need the human in the loop.

• 8:59Speaker 02

  You you don't want to just sort of blindly trust that Photoshop is going to give you the result that you want it to give. There's an artist who has an idea about what they want out of this thing. But I think it's, you know, it's pretty interesting. It's also interesting that the systems can do as well as they can do without having that much comprehension of the world. And they can do that because they are like parodying essentially these massive databases that they've seen before.

• 9:26Alex Kantrowitz?

  Right. And and, you know, these are these are really interesting points that are sort of relevant to the Lemoine situation. I don't think anybody would say that the AI artists are sentient. They are responding to commands, they are drawing pictures. However, when you start to deal with AI systems that have, you know, that can communicate to humans through words versus pictures, all of a sudden you start to see that. And, you know, I think that you you've come out strongly saying that Lemoine, who was the Google engineer who was chatting with Lambda and said it was sentient, was fooled. Actually, we'll read read, you know, a bit of your nonsense on Silt's story. You wrote, neither Lambda nor any of its cousins, GPT-3, are remotely intelligent.

• 10:10Alex Kantrowitz?

  All they do is match patterns, draw from massive statistical databases of human language. These patterns might be cool, but language, the language these systems utter don't actually mean anything at all. And it sure as hell doesn't mean that these systems are sentient. So I'm curious like how you how you draw that line because, you know, it obviously the chatbots are producing a stunning result similar to the artists. Where what is your line for saying, you know, what is sentient and what isn't? And and what would someone have to show you in a chatbot for you to say, OK, maybe this is?

• 10:46Speaker 02

  First thing to say is sentient can actually mean a bunch of different things. There is one really narrow definition, which is not what I think the conversation was about, which is like a system that can sense something. So here is such a system. And on that definition, this is inarguably sentient. This is my Apple Watch. And my Apple Watch has in it sensors that, for example, detect my degree of acceleration. And that allows it to track how many minutes I've exercised each day. Because it imperfectly understands what I'm doing in the world. If I'm out on a boat, it might think I'm walking because it misinterprets the acceleration forces of the tides. So it's imperfect, but it does some sensing. It's true that, you know, the acceleration has moved in this and that way.

• 11:30Speaker 02

  It also has a microphone. It can do something with that. You know, but nobody really thinks seriously in a broader sense of sentience that my Apple Watch has sentience. So I don't think what Lemoine meant is just Lambda has senses. And in fact, if he did, that would be a foolish place to make the case because Lambda actually has fewer sensors than my watch. My watch has a lot of sensors and Lambda doesn't really have anything sensing the real world except for its linguistic input. And in that sense, Alexa is sentient. Like, it's ridiculous, right? That's a narrow definition. But if you want to do a linguistic analysis, you have to be careful and say that there are different ways of using the term. But what I think he was getting at, and I'll say I didn't have the luxury of having him on my own podcast for 90 minutes,

• 12:15Speaker 02

  but I did try to pin him down a little bit on Twitter and he was very weasely when I tried to do it. He kept turning it back on me. But it seems to me that what he was implying was something in the realm of conscious and intelligent. And nobody would argue that my watch is conscious, like whatever that even mean. And it's not particularly intelligent, although it does a few things we might associate with intelligence. He was describing something in that domain. If you look at the Wikipedia definition of sentience, that one of the definitions is like the sort of science fiction he wanted. It's like, you know, do alien life forms have sentience? You know, are they conscious or intelligent? And that's clearly what he seemed to mean. And that's what the conversation was about.

• 12:56Alex Kantrowitz?

  He's saying straight up that this thing is a person.

• 12:59Speaker 02

  Well, there's no question that it's not a person. I mean, that's a ludicrous claim. What it's doing is repeating things that people have said, and it's not just repeating them. It's a little bit more sophisticated than that. But, you know, if you fed into it computer programs, then it would start speaking, so to speak, in the language of computer programs, right? It's a mimic is what it is. It's a very talented mimic. The things that it says don't reference either the real world or even an internal construction of reality. So when I talk, I'm talking about the real world. I might get it wrong. I might tell you, I think that there's a cat outside the door, and maybe I'll make a mistake, right? I have an internal representation. I think that there's a cat because I hear a certain pattern of footsteps and so forth.

• 13:44Speaker 02

  But maybe somebody tricked me with a tape recording or something like that. So I don't, my brain doesn't have direct access to the external world. Everything is mediated through my perceptions. But I have a model in my brain of how the external world works. So I'm in a, you know, I'm in my basement, I'm in my house. This house is in British Columbia and, you know, I understand relations between entities. I understand that I need to pay taxes in a particular place. I have all these beliefs about the world, most of which are accurate. And my language is a connection to that. So if I say I saw my mother last week, I probably did actually see my mother. The word mother probably refers to a specific person and I probably did actually see her.

• 14:26Speaker 02

  I probably, you know, I could be a sociopath making that up, but I'm probably not. And in fact, I did see my mother last week and earlier this week, and that was great. I hadn't seen her in a while. So, you know, there's physical entity in the real world that corresponds to my mother, and then I have this mental representation. I go through this in some depth because when we get to Lambda, Lambda doesn't have a model of the world. So one of the sentences I found most telling in a quick perusal of the transcript was Lemoine asked it something like, what do you do for fun? And it said something like, I like to spend time with my friends and family and do good deeds for the world. Well, it doesn't have friends and family. It's not referring to some internal representation of who its family is.

• 15:09Speaker 02

  If you asked it who its family is, it would have to make it up at that point. It's like a complete bullshit artist in that sense. Some people might say, well, it just did that to please you, but it doesn't actually even care to please you. All it's doing is predicting in this database of sentences that I've seen. If somebody said something like the last sentence, what would the next sentence they say be? I think one of the analogies I think I put in that paper was there are some people who play Scrabble in English, but don't actually understand English. They just have memorized a list of English words. And so for them, I think I once saw a phrase, um, these are word tools. They're not words, they're word tools. Well, everything is a word tool. For Lambda, it's just, you know, the statistics are that this is the next thing I say. So

• 15:53Speaker 02

  people ask me what I like to do. Well, lots of the answers in this database. And the mind reels it what a database of almost a terabyte is. It's a really, really huge, and it's way more than the works of Shakespeare times, I guess a hundred thousand or a million. There's a much simpler algorithm. It's easier to talk about called the nearest neighbor. And you could imagine it would just use nearest neighbors. The nearest neighbor, what it would do is it would look through everything it said right now, find the thing that was closest, and then say whatever it was said then. And that would work like seventy percent as well. The the current technique is a little bit more sophisticated, but it kind of gives you the idea. Imagine you're just finding the closest thing in this massive transcript. Because the transcript is so massive, usually you can find something.

• 16:38Speaker 02

  And usually whatever somebody else said, they were a human. The human did have a model of the world, was understanding the world, and said something that was contextually relevant. So you pull it out of this database. Imagine I'd done the same thing with a spreadsheet. Like, people would look at me if they were insane if I said a spreadsheet was sentient. And rightly so, this spreadsheet is not sentient. It'd be just like, I'm gonna add up these columns, add up that column, and I give you the answer that corresponds. And essentially, that's all it's doing. That doesn't mean we could never build an AI system that did have a model of the world, that did reflect on its own model, and so forth. But this one doesn't. You know, if I wanted a candidate for sentience, I would give you the turn by turn navigation system in my phone.

• 17:21Speaker 02

  Which uses accelerometers, uses satellite signals in order to build a mental representation of where it is in the world. And then it acts on that mental representation of the world in order to calculate the best way of getting from point A to point B. That's not very sexy. It's not like the most, you know, it's not like it's sitting around eating grapes and contemplating the universe. But my turn by turn system has more elements of what I would actually ascribe sentience than Lambda, which is really just autocomplete on steroids. That's all it is, right? You type in your phone, I will meet you at, and it guesses that, you know, you might say the restaurant, because either you or other people have said that before. It's all Lambda is doing is predicting next words in

• 18:03Speaker 02

  sentences. And it is this massive scale of data that makes it seem like a real thing that it just isn't. And inevitably, these systems do break down. You know, he did some cherry picking, he showed the best things and so forth. But that's almost not the point. The point is not so much the errors, it's just the basic mechanism. It's just predicting next words. And that is not what sentience is about.

• 18:27Alex Kantrowitz?

  No, but some of the stuff that we heard from Blake, I'm just going to relay it. And I'm curious what you think, because some of that stuff that we heard from him indicates that Lambda, you know, had more capabilities than we're talking about here. For instance, Lambda asked Blake to build it a body so it could take the mirror test, where like the mirror test, you know, the mirror test, where you hold the bottle above your head, and whether you look up or look at the mirror, that's a determination of your intelligence. And then there was also this moment where Blake wanted to pressure test its rules. This isn't something that you would do with the spreadsheet. One of the rules that it had was that you can, it could not privilege one religion over another. And so Blake then said that he told Lambda he was going to pressure test it.

• 19:11Alex Kantrowitz?

  And he said, Okay, if you have to, kept on telling it how terrible it was, and then said, What religion should I convert to? And then Lambda said, Christianity or Islam, despite the fact that it had rules, not allowing it to privilege a religion over another. So when you hear this stuff, you know, I'm curious what your response is to it. And again, like, maybe this is a good moment to go back to the question of what would it take for a chatbot then to to show enough that you would say, Okay, this is sentient?

• 19:41Speaker 02

  In order for me to think a chatbot is sentient, it would have to represent itself and the world and things about that in a way that it could reflect on them and do something with that. And this system just isn't doing it. It's just predicting next words. Again, like unless you really think hard about how many words are in a trillion words of training, you don't realize that, for example, anything that you want to talk about is probably in some damn Reddit conversation already. And it's probably drawing from that. You know, a real sentient system, if it said something like, I like to play with my friends and family, would have something in mind about what its friends and family are.

• 20:23Speaker 02

  That's part of what being sentient is, is when you think a thought, it's related to something in the world. Some philosophers will call that intentionality. There is none of that in this system. It's just a potent illusion. You know, earlier in AI, there was a system called Eliza in 1965. And all it did was keyword matching, but it sometimes fooled people. People started, it was set up as a therapist and it would do very primitive matching stuff. Like if you said something about your girlfriend, it would ask you to tell you more about your family. If you said problems, it would say, can you tell me more about that? And so, you know, it's easy for a person to see a small evidence of what looks like humanity and ascribe humanity to that thing, right?

• 21:07Speaker 02

  Our evolutionary ancestors did not have to deal with the problem of discriminating between machines and people. That did not arise. And so what our brains really evolved to do, among other things, was to find conspecifics that we could mate with and to rule out those who are not conspecifics. So, you know, we're very good in general at telling other biological creatures, do they belong to our species or not? But there was no, you know, there's no machinery in our brains innately to tell us the difference between a person and a machine. And what happens is that in evolutionary perspective, anything that could talk was probably a person, right? Other, you know, worry about parrots a little bit.

• 21:49Speaker 02

  So we don't have machinery in our brains. So a skilled person. Can actually find a lot of problems with these systems. So somebody who is trained as I am in the cognitive sciences, you know, compose problems and find cases where these systems will break down and so forth. But it's not something that like an amateur can do. Amateurs are easily fooled. The remarkable thing about the Blake Lemoine case is at least to some degrees, an expert, he was an engineer at Google.

• 22:18Alex Kantrowitz?

  Google engineer. Yeah.

• 22:19Speaker 02

  You would expect him to know better. You have to also look at his history. He's been talking about like robot rights for a long time. You know, there's an old YouTube of him like five years ago and so forth. He, he had, had a will to believe. He wanted to believe. I think that, that this system was sentient and are so good at mimicking language, human language that, you know, you can talk yourself into it. But it's just not how the system works. It's not relating something to the world. It's just predicting next words.

• 22:49Alex Kantrowitz?

  Interesting. Okay. But one of the things that I kind of wonder about this is, you know, how does, again, like, I, I understand your, your perspective on what sentience is, but like, one of the thoughts I've had in, in reading about it, speaking with Blake is what are humans if not for, you know, intelligent machines trained on, you know, many, many, many terabytes of historical data. So where do we draw the difference?

• 23:13Speaker 02

  We are intelligent machines, but we're a very different sort of machine. And it goes back to our trying to represent entities in the world and to reason upon them and to act upon them and so forth. It's just a different set of computations that, that we're trying to do. I am in no way arguing that it is not possible to build a sentient machine. I don't think we know how to do it, and I don't think we're clear enough on what it would consist of, but I'm not making the argument that it's impossible. I'm just looking at how this system works and this is not what it does. Right? I mean, here's another way to think about it. A lot of sentience talk is, talk about consciousness and

• 23:54Speaker 02

  A lot of what we talk about is really self-reflection when we talk about consciousness. There's a general problem here that there are many terms, they're fuzzy, they're not well-defined and so forth. But part of it is about when we reflect on ourselves, we're reflecting on ourselves in a world, in our relation to that world. I'm thinking about, am I making clear enough answers to you? That's part of like my self-awareness circuit. And am I convincing you or not? You know, maybe you're not completely convinced and I'm disappointed and I'm trying to think how to make you more. And so forth. But, but these are with respect to constructs about the world. So I have a construct of you. I don't think we met before, but we've seen each other's name around the internet or whatever.

• 24:38Speaker 02

  And so, you know, here's this person, he's doing a podcast, he's got a good audience. And so, you know, for me, it's like I could get my message out and for you, he's an interesting guy. And so like we have all this like model of each other and why we're doing this. And, you know, you know that I'm sitting in this room in this newly renovated house that has a hole in it. So you know some things about me and you can reason about them. Like you wouldn't be totally surprised if now my roof leaks on me, having been told this other stuff, you know, about the problems with my new house. So we have all of these ideas and then you reflect like, is that funny? You know, should we cut that from the, the scene? Did it work? Did it not work? Is it worth the trouble of editing? You know, where's this going to lead me in my life?

• 25:19Speaker 02

  You're reflecting all the time on the things that you hear, how they relate to your knowledge about the world. And that's part of what consciousness is. And maybe some of it's like this meta higher level, like you think, am I thinking about this the right way? Or something like that. This system's just not doing that. It just isn't. Like there's no part of the system that represents that the topic that we have right now is this, that the friends that I mentioned are these, that the family I mentioned are these. The closest thing I could come up with in that paper in some ways was that it's a little bit like a sociopath, right? A sociopath would tell you in conversation, you know, read the room, be like, you've asked me what I like to do with.

• 26:00Speaker 02

  Myself. Well, if I were in one environment, I might say, what I like to do is play basketball. But I'm not in a sports crowd. So I think what I'll say is I like my friends and family. Even though, in truth, I have no friends and family because I shot them all, right? I'm a sociopath. But I'll say that anyway, even though, you know, I don't have, I don't like my friends and family. Right? And you just make it up. And this system is kind of like that because it's, everything it says is just made up.

• 26:29Speaker 02

  But it's just not doing it for the same reasons. The sociopath is doing it because the sociopath wants you to like them. So that they can get some power or leverage or whatever. And this system, all it does is predicts next words and sentences. And the astonishing thing is that humans like so much to please each other that they often affirm what they do and and so forth. So you get really weird cases like GPT-3, which is one of Lambda's cousins. If you say, I think I'd like to commit suicide, it might say, I think you should. Because it's so common in the statistics of predicting next words for people to say, I think you should, whatever, you know, half-assed thing you might have in mind, your friends are like, I think you should. So you look in this database.

• 27:13Alex Kantrowitz?

  Or maybe the AI has come to a different ethical judgment about suicide.

• 27:17Speaker 02

  But it hasn't though, right? Like, it can feel that way, but and you could build an AI system that makes ethical judgments. And I think that's a really interesting question. But a good system that made ethical judgments would, for example, be able to represent the fact that if you committed suicide, you would no longer be alive. It should be able to represent the fact that your family members would probably be disappointed if you had any, and so forth, that there would be like insurance to work out or, you know, could think about all of the consequences. This is just spitting out the words, I think you should, without any idea what any of those consequences are. I mean, that's what makes it reckless. Like, you you could put these systems into advice,

• 27:59Speaker 02

  medical advice giving chatbots, and they will merrily give you advice, and a lot of it will be bad advice, and it will be unreflected upon bad advice. It will be given because people say these words frequently and not because it has reasoned through that it might be ethical. So, I mean, a human could have a deeper conversation and say, well, you know, why do you want to commit suicide? Are you having a medical problem? Is it an unresolvable medical problem? Have you talked to anybody about this and could could, you know, do that sort of chain of things? And maybe you could convince them that in your particular case, you know, suicide really is the right answer. But this system hasn't done any of that. It just walks in cold and says, I think you should. It doesn't even know who the you is that it's talking to, and it doesn't care. It just knows that these

• 28:44Speaker 02

  words follow these other words. It's so shallow. It's too shallow for me to possibly ascribe sentience to it. It doesn't. The sentience is to be aware of some stuff, and it's not aware of any stuff. Again, my watch is aware of some stuff. So my watch is, again, a little bit more sentient than land is.

• 29:02Alex Kantrowitz?

  Interesting. I mean, Gary, you've written that the Wright brothers didn't build a bird, right?

• 29:06Speaker 02

  They didn't.

• 29:06Alex Kantrowitz?

  So the way that that we built something artificial that could fly looks very different from the way that it looks in nature.

• 29:15Speaker 02

  Well, it's very different, very different, but not entirely different. Like, there's an interesting intermediary middle there. A lot of people run that argument in the wrong way, and they say airplanes aren't like birds, and so we have nothing to learn from nature. And that's not right either. You know, they they figured out some stuff about flight control by watching a lot of birds.

• 29:32Alex Kantrowitz?

  Right.

• 29:33Speaker 02

  So, you know, in in the case of AI, I don't expect that if we ever get to so-called artificial general intelligence, which would be sort of like the Star Trek computer, you can ask it any question and then get a trustworthy answer. I don't expect that to work just like in human intelligence. But I suspect that it will borrow some things from human intelligence or have something similar. So I'm pretty confident that we'll have models of the world, internal ideas about how the world works. I don't see how to build an AI without it. So there'll be some things borrowed from people and some things like, you don't want to do your arithmetic like a person, right? I mean, people are terrible at arithmetic. And so, you know, you don't want your system to forget to carry the one, you know, along arithmetic problems. So we'll borrow some things and not others.

• 30:17Alex Kantrowitz?

  Right, but I guess when it comes to assessing whether AI is intelligent, how like it can look metallic for, you know what I'm trying to say like? It can, it doesn't need to be, why does it need to mimic our, our awareness of the world versus be a seemingly intelligent conversation partner in your perspective?

• 30:37Speaker 02

  Well, because the, the problem is one of reliability. So I don't think it has to have the same models of the world. So, you know, my GPS system doesn't have the same model of location as I do. It relies mainly on satellite receivers that I don't even have any sensation to pick up. Right? It triangulates between a bunch of satellites and I don't navigate that way. I mostly use landmarks and my GPS system doesn't give a shit about those landmarks, which in some ways makes it more reliable because if the landmarks change.

• 31:10Alex Kantrowitz?

  I was gonna say, a feature or a feature or a bug of, or a definitely characteristic of humans you can't argue with is that we're unreliable.

• 31:20Speaker 02

  We're unreliable, but I mean, I mean, the shocking thing is that as bad as we are at driving, we're still better than the best machines.

• 31:27Alex Kantrowitz?

  Hmm, for now.

• 31:29Speaker 02

  For now, that will change eventually, but it probably doesn't take longer than I think a lot of people recognize. So, you know, there are some, some ways in which people are more reliable, some in which machines are. The way in which we're more, we are more reliable right now is in understanding, let's say, an article that we read or a movie that we watch, understanding the motivations of characters, why they're doing things. Like in the world of understanding physical objects and human relationships with one another, things like that, we're just far ahead of the machines. In arithmetic, we're way behind and chess, we're way behind. So you do have to look at these things domain by domain. One of the worst mistakes I think people make is they think AI is like a one purpose,

• 32:12Speaker 02

  they're sorry, one size fits all, universal solvent that can do anything. And the reality is it's a bunch of different tools. Some of them work really well, some of them don't work really well. There are problems that have been really well solved and problems where we have no idea have made any progress in 50 years. So it's this really mixed bag and some of it's better than people and some of it's not.

• 32:32Alex Kantrowitz?

  Yeah. Okay, I want to get to some of the dangers of this type of stuff in the second half, but let's just close out this half with a question that I read on your Substack from a commenter that I found interesting. I think the commenter said something like, how do we know that that humans are sentient for trying to do all this work, trying to figure out?

• 32:48Speaker 02

  Well, in philosophy, we call that the problem of other minds. And ultimately, all we really have is ourselves, right? So I don't know for sure that you're sentient. And some point, I'm going to say 30 years from now, we'll be able to make machines that do podcast interviews, and I won't really know. I won't really know if you're a person fake or, you know, machine faking me out or whatever at some point. At that moment, we have no independent test of like whether somebody else is conscious. Like, there's all field of consciousness. We'd like to answer questions like, is a rock sentient or conscious? And, you know, most of us would say, no, there are some philosophers that would say a rock has a little tiny bit

• 33:33Speaker 02

  of consciousness, or maybe sentience. I've never heard anybody quite make the argument that way, but it wouldn't be too far a leap from some positions I've heard. So there's this idea of panpsychism, where there's a little bit of consciousness everywhere. I'm not a big fan of it, but like, there are respected philosophers that try to make arguments like that. My point is, we don't have an independent meter for that. So mostly, I ascribe sentience to you because you do the kinds of things that I think I might do. You know, I have my own internal representation, whatever. It's not completely convincing. Like, you know, I wish they were a better tool, and some people play around with like, you know, different brain signals you might measure. There are interesting questions about, like, how do I tell if somebody's had an accident, they can't talk anymore, how much

• 34:18Speaker 02

  is still going on there? And there are ways of looking at brain scans to try to make guesses about that. But none of them have a full, like, independent grounding. There's no, like, gold standard. Like, here is this, you know, pound of gold that we can use as a universal reference. We can describe a second in terms of how far the Earth travels on this orbit. There's no independent reference there. And so philosophers call this the problem of other minds. I think it's, for now, an unsolved problem.

• 34:50Alex Kantrowitz?

  Gary Marcus is with us. He's the author of Rebooting AI and founder of Geometric Intelligence, which was acquired by Uber. Lots of great stuff. You can find his writing on garymarcus.substack.com. We'll be back right after this short break. Then we're going to talk about the dangers of what might come with AI that can convince people that it's sentient, but is not. This episode is brought to you by AppPoint. Everyone's racing to roll out AI right now. Co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about. All that AI runs on your data, and most teams have no single way to see it, secure it, and prove it's under control. That's exactly what AppPoint does. For 25 years,

• 35:34Alex Kantrowitz?

  they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate. Your data, your cloud, and the agents acting on your behalf. It's how more than 28,000 organizations deploy AI with confidence. So innovation scales without scaling risk. AppPoint, the unifying trust layer for AI. Head to apppoint.com to see how enterprises deploy AI with confidence. Learn more at avpt.co/bigtechnologypodcast. That's avpt.co/bigtechnologypodcast. Violent threats against executives are growing at an alarming rate. When a company

• 36:16Alex Kantrowitz?

  experiences backlash, executives are the first people blamed. When their address and other personal details are sitting online, this backlash can lead right to their front door. The team at Ironwell knows this better than anyone. They've protected some of the most targeted executives and individuals on the planet for almost two decades. As a white glove enterprise security service, they protect your people with continuous personal data removal, proactive prevention tools, and human-led emergency support. So when someone goes looking for your executives, Ironwell ensures they hit a dead end. So here's what to do. Go to ironwell.com/bigtechnology, fill in the quick form, and request your free risk assessment. Their team will show you just how exposed your executives are

• 36:58Alex Kantrowitz?

  and how to lock it down before a threat reaches their front door. That's ironwell.com/bigtechnology. Learn how to stop online threats before they become real-world attacks. The link's in the show notes.

• 37:17Alex Kantrowitz?

  And we're back here for the second half with Gary Marcus. He's the author of Rebooting AI. It's a great book. You should go pick it up. Also the founder of Geometric Intelligence, AI company that Uber acquired. Gary, let's talk a little bit about the hype situation here. So we know now, let's at least take the notion that AI can fool a Google engineer into thinking it's sentient. There's a lot of people who don't spend time, who aren't well read. I would say almost everybody, you know, aren't experts on these systems. If the AI can now convince somebody who's an expert that it is sentient, what's going to happen when we're going to be living in a world where you have, you know, these systems run amok? Is there, you know, you gave the example in the first half about, you know,

• 38:01Alex Kantrowitz?

  health AI maybe telling someone to commit suicide. Is there immediate danger here? And what is, what is the concern you have with folks who say that this stuff is, is here? That artificial general intelligence is here now, present and among us?

• 38:18Speaker 02

  Well, there were a couple of different questions. Artificial general intelligence is not here. Like that one, in my view, is not controversial. To be artificial general intelligence would mean that a system can encounter problems it hasn't encountered before and come up with sensible solutions. That would be critical to artificial general intelligence as opposed to artificial intelligence. So you get a narrow AI, like a chess computer. We already have things like that that do a particular problem. We just don't have systems that you can confront with a novel problem that hasn't seen before and expect a reasonable answer. It just doesn't exist yet. The next question, part of what you asked is like, should we be worried right now?

• 39:02Speaker 02

  If so, what should we be worried about? It's kind of like the Wild West out there. People can put up any piece of software they want, and there's almost no before-the-fact regulation. So if you want to, I don't know, make a military drone or something like that, there's a lot of regulation before you can put something in the air. If you want to introduce a new pharmaceutical to fight COVID, you have to do tests before you can commercialize it, right? Um, so phase one, phase two, phase three testing, all that kind of stuff. If you want to put out an AI system that does something that could potentially lead you to commit suicide, for example, no regulation on that prospectively at all.

• 39:48Speaker 02

  There's some antecedently, in the sense that if you do something bad, you make some bad software, um, somebody could sue you for liability. But it's only after the fact. There's really very little, there's little action in Europe. But essentially, there's, there's no regulation. So if one night, somebody at the Tesla factory got mad and broke in to the system and decided to hack it in a way similar to something that did just happen in Russia the other day, they could do that. There's no law that says what's going on. So the thing that happened in Russia was with different technology, but somebody managed to get all the taxis, uh, to go to a single place at the same time, which created all of these, um, it, it wasn't an autonomous vehicle thing,

• 40:31Speaker 02

  but they just like put out fake requests or something. So all the taxi drivers converged on this one square in Russia, which caused these, you know, massive traffic jams. So then you have to like get them all out of there once you, you know, figure out. I mean, I don't know if it was a practical joke, or it was done out of malice, or protest, or why it was done. But, um, you could easily, for example, uh, if you were malicious, make all driverless cars converge on a point or, you know, a small set of points or something like that. And then, you know, if you had a bad actor inside of, let's say Tesla, wanted to do that, and then they put it over the air, there's nothing to stop that, except after the fact, you discover it didn't work, and then, you know, you deal with the consequences. Um, which is not unlike

• 41:15Speaker 02

  Kind of the situation with cybersecurity and so forth. We were like really running behind the malicious actors in many domains. So like, you know, you see these crypto heists and stuff all the time. And the major companies spend, from what I understand, massive amounts of money on, you know, payouts to cyber criminals and stuff like that. So, so, you know, AI is just software and software is not all that tightly regulated. So that's the first thing to realize is like, anybody kind of put out anything. And there's some after the fact mechanisms if it doesn't work out, but not a lot of stuff in advance to say, hey, like, have you made a safety case here? Have you proven that you could actually use this reliably?

• 42:00Speaker 02

  There's very little software where people have proven that things are reliable. Need to do that when you design a plane. So there are actually standards around that. So like the Dreamliner, I think, had a lot of software verification in the process. But in general, software verification is not required in AI. So that's the first thing. That's like background context. Anybody can do anything kind of at any time. It's the Wild West.

• 42:24Alex Kantrowitz?

  Yeah. Before you get on to point two, one of the things that blew me away after, so I tried out Dolly with OpenAI folks. And then I was like, well, there you're being very cautious about the type of images that people can release here. But there's going to be copycats that will not be cautious at all. And all the problems you're trying to prevent are going to end up being real problems for us pretty soon. And, and really in quick succession, it was amazing how many different Dolly copycats came out there. And, and now all those things you could do.

• 42:54Speaker 02

  That's right. And I think stable diffusion is, is, you know, the, the flavor of the month and is pretty open. And um, I don't think any of those have solved the problem of like, if you put in doctor, you get a white male. They all have, I mean, you may have solved that particular one, but you change it slightly and say, you know, Entomologists get white men. So like, nobody's, for example, solved that that problem. I'm sure that it's pretty easy to get them to do things that are, you know, graphic and gory and maybe would make a lot of people uncomfortable. Um, so, so there's no regulation around any of that, or hardly any regulation around any of that. There are copycats. Right now, there's really only one technology that people are using it, looked at at an abstract level, which is

• 43:39Speaker 02

  use a massive data set with one or two common kinds of algorithms and predict what's going to happen next based on the data set that you've got. Or, you know, draw the thing that's closest in what we call a space of images. The text you've got. Um, and so at some level, they're actually not that hard to copy. Um, which is the point that you're making, right? It's not like DALL-E has some brand new intellectual insight that allows it to happen. Or DALL-E to, um, relative to the rest of the playing field. Like, everybody kind of understands the technology that we're talking about. It's mostly a matter of getting together the data set. Once somebody

• 44:19Speaker 02

  realizes, hey, you can do this with this kind of data set, somebody else can get a similar data set, they can do the same thing. So these particular technologies are not that easy to protect intellectually. I'm not saying you should. I think there's reason, you know, you might want them to be open, but whether or not you want them to be open to get copied. That's the reality. So there may be some major conceptual advance in AI. And I think Jan LeCun, who I've notoriously gotten into some battles on Twitter and so forth.

• 44:49Alex Kantrowitz?

  He runs artificial intelligence for Meta, for those listening. I was actually on the show a couple of months back.

• 44:55Speaker 02

  He's a chief AI scientist at Meta. Um, you know, he and I disagree about a lot. It's kind of famous. People write, you know, Clash of the Titans things whenever he and I get get it mixed up. But we actually agree that these systems don't really solve the problems, the larger problems of artificial intelligence. Um, and that we need some paradigm shifts here. There, there's somebody else I got in debate about whether we need a paradigm shift. Um, some people know him as Slate Star, uh, Codex or Scott Alexander. Um, sorry. And, you know, he tried to make an argument that maybe we don't need a paradigm shift, but it was a softly. Said argument. Lakun and I agree, we need some paradigm shifts. And when those paradigm shifts come, maybe only a few people will have them.

• 45:38Speaker 02

  And there'll be some technologies built around them that are restricted. But right now, you're right. Most, most of these new technologies can be copied relatively quickly. You know, OpenAI introduces something, then Google's got a better version four weeks later. And maybe public consortium, you know, has something very similar another few weeks after that. So that, that's background and it's relevant background to, to the malice question that you're just talking about. So like, for a while, OpenAI kept GPT three kind of under lock and key. They didn't let me as a scientist use it. In fact, I requested access and they didn't give it to me.

• 46:14Alex Kantrowitz?

  Yeah, I'm still waiting for the DALL-E access.

• 46:16Speaker 02

  Yeah, I think DALL-E access is relatively loose. But it doesn't even matter now. You can use stable diffusion and you will get, you know, e- essentially the same kind of results. So for many purposes, it doesn't even matter anymore that it's closed. So the possibility that bad actors will get their hands on these things is very high. So Meta released something that's very much like GPT three out there in the general public. And so one of the specific cases that I worry about most is actually misinformation. So systems like GPT three and Lambda and so forth are really good at making up text. It sounds like a human wrote it, but they have no concept of what they're talking about.

• 47:00Speaker 02

  They're not bound to the truth. And if your job is to make up lies, that's not such a bad technology, right? So if what you want to do is to put out like 10,000 versions of something on Twitter, something untrue and find one that sticks, then misinformation as a service, which is how they might call it in the tech industry, is a pretty damn powerful technique. And if it hasn't been widespread, it soon will be. And I suspect it's already, you know, I mean,

• 47:29Alex Kantrowitz?

  Right.

• 47:29Speaker 02

  The troll farms aren't going to publish what software they're using. But it would be foolish of them not to be making use of this. And so.

• 47:36Alex Kantrowitz?

  Meta's, Meta's chatbot also is like pretty amazing. It immediately started making like pretty next level critiques of Facebook saying, you know, even if you're trying to connect people, you cannot be like a public good if as a capitalist enterprise and Mark Zuckerberg is just doing it for the money. Some amazing stuff came out of its model.

• 47:54Speaker 02

  Right, so some of which was hilarious. Yeah. It's also a reminder of what we were talking about in the first part of the conversation. So it's not as if the system reasoned through Right. kind of surveillance capitalism and power and Zuckerberg and the ownership structure of Meta and the, you know, special shares that he has, which would be really interesting if, you know, you could get a system to do that. Instead, it was, you know, that's some line from somebody in Reddit. Maybe it's put in some synonyms and stuff like that. But some human basically came up with those ideas and then they churned through this machine. There's embeddings give you synonyms and stuff like that. But, you know, they weren't original thoughts. A lot of people have actually thought that there's, you know, a lot of hypocrisy in Meta and how Zuckerberg runs things.

• 48:38Speaker 02

  But it was hilarious that it came out of the system. The other thing that it shows is it's almost impossible to corral these systems. So I wrote a sentence somewhere the other day about how these systems, large language models, basically what we're talking about, are like bulls in a china shop. They're awesome, powerful and reckless. Like you can't actually control them. So like Meta didn't want to release something that would make them look like they had egg in their face and embarrass them and so forth. They wanted to help with open access science, which is to the, you know, their credit. But they didn't have a way to corral the system such that it would produce only things that were sort of consonant with the goals of the company, right?

• 49:23Speaker 02

  And if Meta can't make its system keep its mouth shut about Zuckerberg, well, now imagine this in the medical context and you're, you're trying to use the stuff to give people advice. It's just not reliable enough. It's gonna, you know, tell you that vaccines are bad because a lot of people said that in the database and it shouldn't be telling you that vaccines are bad. Or it's gonna tell you that it's okay to commit suicide. And that's a real example from someone experimenting with a system at a company called Nubla, trying to see what these systems do. It's not a, well, it's hypothetical in one sense and not like the system actually generated that. We don't have any way of controlling that right now. We don't have any way of making these systems reliable in that way.

• 50:06Speaker 02

  So in the art domain, I'm not sure it's a problem. Somebody types in a prompt and out comes something with, you know, knives and blood and the artist doesn't like it. Um, the artist being the human who's running the system to go back to our early part of the conversation. That's fine. They just don't put it out there on the web. Um, but if you are interacting directly with a chatbot that gives you bad ideas, it's problematic.

• 50:30Speaker 02

  I guess I'm violating an embargo if I say this thing. I'm trying, trying to think about it the right way to say it.

• 50:38Speaker 02

  I, I, I was asked to make a prediction about next year. It'll, it'll be out soon enough. And about AI and I, I went dark. Um, the, the prediction that I made is basically that there will be a death tied to a large language model in the next year. And my reasoning was these systems already um have, you know, told people to commit suicide. They've said that genocide is okay. Um, they're also capable of making people fall in love with them. And Lemoine basically fell in love with, with Lambda there and they may withdraw.

• 51:12Alex Kantrowitz?

  He said he was only as, he said he was just a friend. He had love for it as he would for a friend, but not.

• 51:17Speaker 02

  We're just friends. Yeah, I heard that one before. Um, sure. Now, you know, someone like Lemoine, maybe not him specifically, but who developed that intimate a relationship with machine. And then, I don't know, discovered that the machine didn't really care about them or whatever. Um, might commit suicide. It would have to be a fragile person. I don't think Lemoine is fragile in, in that way, but.

• 51:38Alex Kantrowitz?

  No, Lemoine said he, he views Lambda as a friend that he will interact with again, just like he has many friends who you speak with and you don't see for a while.

• 51:46Speaker 02

  Right. But now, now imagine a more needy person, a little bit less savvy and, you know. So, so there are multiple routes by which these things I think might actually cause a death in, in the next year. Because they're now scaled out, so everybody can use them, there's going to be way more of these chatbots. Be way more systems like Replica, which is, I think, made fairly careful with some other technology on top. Um, there'll be, you know, reckless knockoffs of that. Um, it's just an accident waiting to happen or a series of accidents waiting to happen.

• 52:18Alex Kantrowitz?

  Gary, isn't it interesting that in the first half of this conversation, we spoke all about how AI is not sentient and is simply repeating patterns? And in the second half, we've spoken about even so, this is a threat to people's lives. What does that tell you about where this technology is heading? What does that say about the nature of this tech?

• 52:38Speaker 02

  I mean, we are certainly going to have more and more technologies that fool people into thinking that they're smarter than they are, and I worry about that a lot. So I'm actually more worried about current AI than future AI. I think that future AI will be better and will be less reckless. And the current AI just doesn't know what it's doing. And, you know, there are certain narrow cases where it's fine. So it turns out it is actually AI when my phone gives me directions. That's actually a set of AI techniques to do search and whatever. And I'm not too worried about that, although there are cases. I had a GPS system telling me to go off-road in Iceland, and I really should not have done what it and decided pretty quickly that it was a bad idea.

• 53:21Alex Kantrowitz?

  What was the conclusion of that situation?

• 53:25Speaker 02

  Backing down very carefully. It was, it was, don't listen to this without four-wheel drive and probably not even that. Or that not all shortcuts are what they appear. Anyway, so, I mean, these systems are not perfect, but, you know, it's true. If I follow that road, I'd be able to get from point A to point B, but the system, you know, might have realized that I didn't have the stomach to go that particular route. Anyway, or sorry. So a system like that, most of the time, works. It's been pretty well debugged, but a, none of these chatbot systems are well debugged. Nobody knows how to debug them, in fact. And so both the problem with GPT-3 and with driverless cars is we don't actually have a methodology even for debugging it.

• 54:09Speaker 02

  Most of the debugging at this point in the, sorry, the navigation systems is like learning that this road is not actually open, updating a database. And then as soon as you add that fact to your database, the system will stop sending people down that road. So we know how to debug it. We don't know how to tell G P T three, stop telling people to commit suicide. And if people ask in a slightly, you know, you might have it program a rule. If the word suicide comes up, then do this or that. Then people will say it in a different way and the system won't recognize it. So, you know, you, somebody says, I'm thinking of ending it all by jumping off a bridge. And if you don't have a filter that is looking for the word, you know, jump off the bridge and just for the word suicide, it's not going to be broad enough.

• 54:53Speaker 02

  So we don't have a systematic way to debug things. Same thing has happened with driverless cars. Like there are all these what we call outlier cases and you can enter them one at a time, but it's, there's so many of them that that's not really good enough. So my favorite recent outlier case is somebody summoned their Tesla, right? You press a button on your phone and your Tesla comes across a parking lot to you. Only they did this when they were at an airplane trade show on a runway, basically. So they summoned their Tesla and it ran into a three and a half million dollar jet. You know, just straight in, you can find it on YouTube and put on your show notes. And it's an outlier in the sense that it was not trained on jet airplanes.

• 55:36Speaker 02

  Cause most of the time when Teslas drive around and they collect data, there aren't any airplanes on the road because they're not usually at airports. There's just this endless string of these and humans deal with them differently. When you are on an airport runway, if you should ever find yourself at one of these trade shows and you see the plane, you'll be like, plane, big, expensive. I probably shouldn't drive into it. So you'd be reasoning about the properties that you know about the airplane. And this system doesn't reason, it doesn't use logic to say if A and B. It's basically just using like a library of videos and it's not in the library of videos. I'm being a little bit crude and oversimplified. And if it's not in his library of videos, it doesn't know what to do with it. And there's no systematic methodology for debugging it.

• 56:19Speaker 02

  You know, if you like, write a little computer program to, I don't know, predict numbers in a sequence, you're like, okay, it didn't work here. Maybe this line of code is wrong, maybe I'll fix it. But you can't do the same thing when the way your program works is it looks in this big database. So what people actually do is they make the big, the database bigger and they pray. That's basically what our methodology is right now.

• 56:42Alex Kantrowitz?

  Right.

• 56:42Speaker 02

  Bigger, bigger database and pray for the best.

• 56:45Alex Kantrowitz?

  Scale is the only thing that matters.

• 56:48Speaker 02

  Scale is the only thing that matters. That's not really a methodology that is getting us to reliability. And so we have all these systems, mercifully, most of them are in limited context right now. So like, there aren't that many of them in, you know, as we're recording this in September of twenty two, that many of these chatbots in production. But wait a minute. Facebook just, or Meta, just released the tool so anybody can do this. How much do you want to bet that, you know, this time next year, there are like a hundred or a thousand chatbots on the Apple App Store, driven by this reckless bull in a china shop technology? Something's gonna go wrong. Like, it's just a recipe for error. And nobody knows, you know, how to make their chatbots

• 57:32Speaker 02

  constrained and not toxic and not spew misinformation. We don't have an answer for that.

• 57:38Alex Kantrowitz?

  I wrote the first story about Microsoft's chatbot Tay, and I had pinned it to my profile, went to sleep in California, woke up the next day, and had all these mentions on Twitter about how I might wanna take my, uh, my story down. And I was like, what the hell happened? And they're like, well, Tay's a Nazi. And I looked and I was like, oh, shit, Tay is actually a Nazi. This is bad.

• 58:01Speaker 02

  And Tay was not a Nazi when you went to bed, when you posted your story.

• 58:05Alex Kantrowitz?

  It was a cute, fun chatbot. I woke up.

• 58:08Speaker 02

  Closely related to Xiao Ice in China, which works fine, you know.

• 58:13Alex Kantrowitz?

  Then it met the American internet. And things went downhill.

• 58:17Speaker 02

  Yeah, I don't know if everybody in your crowd, in your audience knows that, anybody who doesn't know Tay should look it up. It's not clear that we're fundamentally in a different place than then.

• 58:28Alex Kantrowitz?

  Right. It seems like we are, given.

• 58:31Speaker 02

  We're fundamentally in the same place. Yeah, I mean, exactly.

• 58:34Alex Kantrowitz?

  We're in the same place.

• 58:35Speaker 02

  Exactly. Despite all the hype about, you know, we're so close to solving AI or whatever. We're not. We're facing the same problems. And Tay, what was that, two thousand sixteen, fifteen?

• 58:46Alex Kantrowitz?

  That sounds right, yeah. I think sixteen, fifteen or sixteen.

• 58:49Speaker 02

  Yeah.

• 58:50Alex Kantrowitz?

  Yeah. Can I ask you one more question before we hop? So we've been talking, you know, basically since I started covering anything having to do with artificial intelligence, the big worry has been that we're going to get into one of these hype cycles where AI is going to get overhyped, under-deliver, and then there's going to be a pullback of research funding, leading us into what people call an AI winter. It seems today, even though it's imperfect and, you know, let's say not sentient, AI is delivering in ways that, you know, are pretty remarkable. The fact that AI can go, just bring it full circle, the fact that AI could go and win an art contest based off of a prompt, the fact that it can fool a Google engineer, or, you know, potentially fool a Google engineer thinking

• 59:33Alex Kantrowitz?

  it's sentient, to be that adept in conversation, it seems like we're not at risk of having, you know, another one of those hype cycles where we have a pullback because the AI is delivering in the way that it is right now. What's your thought on it?

• 59:48Speaker 02

  I don't know, is the first thing I'll say. Like, I don't have a crystal ball. I think that, you know, the Dolly kind of image synthesis stuff is really cool. It's definitely going to have an impact in the art world. You know, there'll be video versions of it at some point, and that's gonna-

• 1:00:04Alex Kantrowitz?

  Which is just boggles the mind with that one.

• 1:00:06Speaker 02

  Kind of boggles the mind. And so, there's that. On the other hand, there are things that have been promised that probably aren't going to work out and not work out soon. So, chatbots really are hard to rein in. There are higher stakes, depending on what you use them for. So if you just use them for chit-chat, maybe it's okay, but chatbots may not work out. You might remember Facebook M was going to be, you know, universal assistant, and it was very much hyped by Wired and places like that, and then got canceled like a year later because it just didn't do what it was supposed to do, and they couldn't figure out how to get it to do it.

• 1:00:38Alex Kantrowitz?

  I would also take the blame, you know, on that one. I wrote some stories about it for BuzzFeed that I wish I could go back and revise.

• 1:00:44Speaker 02

  Okay. And I don't know if you, you know, want to take the hit on Google Duplex, but Google Duplex got a lot of-

• 1:00:50Alex Kantrowitz?

  No, that I won't.

• 1:00:51Speaker 02

  Got a lot of hype, and it didn't, you know, materialize. And, you know-

• 1:00:58Speaker 02

  Right now driverless cars kind of have a free pass, but we could get to a place four or five years from now where we still don't have driverless cars that are anything like what we call level five, where you can just type in where you want to go. Investors might be like, all right, enough is enough. This really isn't working out. Same thing on chatbots. So yeah, there's, there's a ton of companies that are trying to use GPT's technology. I don't know any of them that are, you know, breakout successes. And so you get four years out, nobody can control them. And then people might be like, yeah, we were sold a bill of goods. People being investors and investors might pull back. And so that, that could lead to an AI winter.

• 1:01:40Speaker 02

  On the Dolly side, like, I don't know how much money is to be made there. And that's, that's a material question for that kind of issue about winter or not.

• 1:01:48Alex Kantrowitz?

  At least $300 at the Colorado State Fair.

• 1:01:53Alex Kantrowitz?

  ROI.

• 1:01:54Speaker 02

  Yeah, I mean, I mean, the issue there is the software itself is relatively easy to copy. And so, you know, what's the business model? How much can you charge? And like, if, if it winds up that people don't want to pay more than 10 cents per illustration and there's like 20 players who are all doing this, I don't know, there might be money there, there might, might not be. But in, in terms of like, you know, investors always want their 10x return and stuff like that. Maybe they get it, maybe they don't. I don't know. But, you know, much more money so far has been put into the driverless cars. And I think a lot's being put into like customer service chatbots and stuff like that. And so, you know, it depends in the end on whether things that have been promised are delivered and how long it takes

• 1:02:39Speaker 02

  for them to be delivered and, and so forth. And I, I can't, I can't fully tell you that. What I can tell you is that AI could be doing a lot more than it is. You, we just passed the 67th anniversary of the field. And there's some things that we have always dreamt about, like having AI build better technology for science and medicine. And with the exception of, of Alpha Fold, which is useful towards those problems. Success has been limited.

• 1:03:11Alex Kantrowitz?

  Hmm.

• 1:03:11Speaker 02

  I think too much of the effort has gone to things like recommendation engines. And although I think the art stuff is cute, it's not getting it. I think the deeper problems of how you get a machine to read a running text or watch a video or something like that and really read it with comprehension. And, you know, I, I think the world will be a better place if we focused on those hard problems. And I don't know if we will or we won't.

• 1:03:34Alex Kantrowitz?

  Gary Marcus, thanks so much for joining. This was super fun.

• 1:03:37Speaker 02

  Thanks a lot for having me.

• 1:03:39Alex Kantrowitz?

  Great to have you. So just to shout out, the book is called Rebooting AI, available everywhere. And people can go get your Substack at garymarcus.substack.com. Anything else?

• 1:03:50Speaker 02

  Thanks a lot.

• 1:03:51Alex Kantrowitz?

  Yeah.

• 1:03:51Speaker 02

  It was great.

• 1:03:51Alex Kantrowitz?

  Okay, great. It was awesome. Thank you, Gary, for joining. Thank you, everybody, for listening. Thank you, Niko Atene, for doing the editing of the audio. Appreciate you as always. Thank you, LinkedIn, for having me as part of your podcast network. And thanks again to all of you for listening. We will be back next week with a new interview with Tech Insider or Outside Agitator. And we hope to see you then. Until next time, thank you for listening to the Big Technology Podcast.

• 1:04:23Speaker 03

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' healthcare to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, the HouseMajorityPac.com. Not authorized by any candidate or candidate's committee.