Why Google Never Shipped Its ChatGPT Predecessor — With Gaurav Nemade

Gaurav Nemade was the founding product manager on LaMDA, Google's ChatGPT predecessor that never shipped. He explains why the product got stuck at Google in this first-ever episode of the new Big Tech War Stories podcast. This new show from Big Technology is available to premium subscribers and is debuting free on the Big Technology Podcast feed today. You can access the Big Technology launch special and get 50% off the monthly price here: https://bit.ly/bigtechnology or just visit bigtechnology.com. We return to our regularly scheduled Big Technology programming on Wednesday as Waymo Co-CEO Tekedra Mawakana stops by for an interview. Thanks for listening! Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Alex Kantrowitz?

  This episode is brought to you by Genesis. What does it actually take to put Agentic AI to work across an entire enterprise? I recently spoke with Genesis chairman and CEO Tony Bates about why orchestration is emerging as a competitive advantage in customer experience and what companies need to get right in the AI era. Then I sat down with Adam Mitchell, head of enterprise business solutions at Voya Financial, to go inside a large-scale Agentic CX transformation, including the business decisions, trade-offs, and lessons along the way. You can watch both conversations now on my YouTube channel today. It's never been more challenging to get frontier compute. How do you know where to look, who to trust, or how much to pay?

• 0:43Alex Kantrowitz?

  Akash is an open cloud that provides easy access, flexibility, and peace of mind. That's how companies like Razer were able to cut their inference cost by 15 times. Visit akash.network/alex for a free consultation. That's akash.network/alex for a free consultation.

• 1:05Alex Kantrowitz?

  Google had an equal and perhaps better version of ChatGPT working inside its company before OpenAI released its own bot. So why didn't Google ever ship it? Today, in the first episode of the new Big Tech War Stories podcast, we're going to go deep into the history with Garv Demade, the first product manager to work on Lambda, Google's pioneering chatbot built on top of large language models that never shipped. We're running this episode both on the new Big Tech War Stories podcast feed and on the Big Technology podcast feed as a preview. The new podcast will feature interviews with product leaders who were there on the ground as major products got built, and they'll tell you what it took to ship them or why they never did.

• 1:51Alex Kantrowitz?

  And we'll also have key executives sharing what they learned from top tech leaders. The new Big Tech War Stories show is part of a new premium paid offering I'm releasing with Big Technology that includes this podcast, the Big Tech War Stories podcast, along with a panel of experts weighing in on big tech news as it breaks, and a new Amazon column by longtime Amazon veteran Kristy Coulter. You can sign up at bigtechnology.com and add the new Big Tech War Stories show to your podcast player of choice. You can also search for Big Tech War Stories podcast. And then upgrade on bigtechnology.com. All right, let's get to it. Welcome, Gaurav.

• 2:32Speaker 01

  Hey, Alex. Thanks so much. Excited to be here.

• 2:35Alex Kantrowitz?

  I want to start talking a little bit about who you are and where you come from. So first of all, you went to the famed IIT University in India. Not only that, so for those who don't know, to get into this school, and I'm sure Gaurav can tell us a little bit more about it, there's an entry exam and thousands, hundreds of thousands of people within India take this test and only the very top are admitted to IIT. It's a place that if I'm not mistaken, Sundar Pichai graduated from. It's a place where Satyendrala graduated from Gaurav too. And Gaurav was number 732 out of 350,000 students on the entrance exam. What was it like going to IIT?

• 3:16Speaker 01

  I think IIT was an amazing place. It's like some of the smartest people in the world probably land up there. So the funny thing that happens in the first week of IIT is like you, like I scored a 732 rank out of like 350,000 people. So I was all pumped up like I am the best at the institute. And then when you go in there and then you get the first score on your electronics exam and you're like not even the top in the bottom 20th percentile. And then you realize, okay, these people are much smarter than me. And yeah, this is going to be different from my high school.

• 3:50Alex Kantrowitz?

  Yeah, it's one of those things like, oh, you're 732. Like, take that, the rest of the 350,000. And then you get in there and you're like, oh, no, 732.

• 4:00Speaker 01

  Yeah.

• 4:01Alex Kantrowitz?

  What makes IIT so special in terms of the way that it's been able to produce tech excellence and tech leaders through the years? Not only people who are great technologists, but people who have, I would argue, like extremely strong EQ, people who can lead. I mean, it's a very unique that so many of, you know, our leaders today in tech come from that school.

• 4:24Speaker 01

  Yeah, I mean, there is a level of grit and perseverance that is required to get into an IIT. So I think people prepare for the examinations for at least two to three to four years. And it just requires a lot of dedication, focus, perseverance. I think that's point number one. Other than that, I think just being amongst some of the smartest people in the world. It definitely grooms you in a way that you are, you know, much more driven to win in the future stages of your life. Probably those two are the biggest contributors. And then the third I would say is also the opportunity that you get at an IIT versus compared to other universities. At least in India is like very different.

• 5:07Speaker 01

  The exposure that you get in an IIT is like very different in terms of the extracurricular activities and things like that. I think these things are much better in the US. In India, like a lot of universities don't have really good sports program or don't have really good, you know, cultural festivals and things like that. But an IIT in a way just grooms you so well all round while being among some of the best people. I think that's what I think is the reason why there are a lot of ITN leaders doing really well.

• 5:36Alex Kantrowitz?

  You joined Google in February 2013 after a stop as a co-founder, actually, of a tech company within India. And you're working on Trust and Safety for four years. Then in April 2017, you make a very interesting and I would say somewhat radical career shift, where you end up making your move into Google's AI division. Talk a little bit about your move from Trust and Safety to AI. What about the AI division in particular drove you to want to be there?

• 6:07Speaker 01

  Yeah, for sure. So while I was at Trust and Safety, I was already working on a bunch of machine learning related stuff. Like we were building fraud and risk models. These were like basic models like logistic regression and stuff. But around 2016, I think TensorFlow started becoming huge inside of Google and Google decided to open source TensorFlow as well. So that really caught my attention and as I was working on machine learning at payments, I just realized that this this thing sounds really amazing and this could actually change the way we do a lot of things. So I started looking for roles inside of like Google AI and Google Research. Fortunately, there was a role. Fortunately, there was

• 6:50Speaker 01

  an amazing manager I had who was willing to give me a shot. So I ended up moving into Google AI and I spent about four and a half years there.

• 6:58Alex Kantrowitz?

  You join Google AI in April twenty seventeen. In July twenty seventeen, there's a paper that comes out by some of your colleagues that is entitled Attention is All You Need just a few months later. And that is the foundation of the large language models, the transformer models that we know today. Yeah, unpack that a little bit.

• 7:18Speaker 01

  I didn't even know about this paper when it came out. I think once it started garnering a lot of citations over the months and quarters, that's when people started paying attention even inside of Google that, hey, this transformer architecture seems amazing. And, you know, let's start building encoder decoder models. And that's when I probably didn't hear about it until a year after it was published.

• 7:39Alex Kantrowitz?

  What do you think accounts for the fact that it didn't get so much buzz internally? Because it clearly was a groundbreaking moment in tech.

• 7:47Speaker 01

  I mean, a lot of these things are, you make sense of them in the hindsight. For example, a lot of things that I worked on at earlier in the times, like with Mina and Lambda, I couldn't predict like where things would go, right? Like, it's very hard to predict.

• 8:00Alex Kantrowitz?

  Okay, so a year later, people start noticing this paper within Google. And effectively, what it does is allow our chatbot technology to go from like really dumb bots, or not even chatbots, language prediction, to go from being particularly dumb to like be being quite sophisticated.

• 8:17Speaker 01

  One thing that I remember specifically was the Transformers paper came out, but I think it started making a lot of noise when BERT was out. Like, when people started seeing what BERT could essentially do, that was kind of a

• 8:34Speaker 01

  Like a switch that went on inside of Google that holy shit like this is going to change things. So at that point of time, I think everybody from search to assistant to like some of other teams, Gboard, they started becoming very interested in terms of, okay, how do I start using these BERT models?

• 8:51Alex Kantrowitz?

  Which models?

• 8:52Speaker 01

  I led some of the BERT models.

• 8:54Alex Kantrowitz?

  Okay, what are those?

• 8:55Speaker 01

  There was a, so it was basically a paper that came out, B E R T. It was an architecture based on transformers as well. So the BERT models essentially like on some of the benchmarks, they did really, really well. And that's when a lot of people started paying attention related to, you know, this could actually be groundbreaking. So there was a lot of noise and effort inside of Google to start using BERT in like production use cases.

• 9:23Alex Kantrowitz?

  You know, you became the first product manager on Lambda, so and we're going to talk about that. But tell us when you first started to realize that this was going to be big.

• 9:31Speaker 01

  It was very serendipitous back then, so I was, we have like a bunch of email aliases inside of the company. And in one of the email aliases, we get this email from an engineer named Daniel saying that, hey, I built this chatbot and it can do X, Y, Z things. So I played around with the chatbot and I was like, holy shit, this is amazing. Like it, it was still dumb at that point of time, the chatbot was still dumb at that point of time. But it was still kind of a step up from the chatbots that we had seen using, let's say, what was it called? Dialogue Flow, which was Google's like chatbot product at that point of time. So I had worked with Daniel on one of the previous projects like a year ago or something, while he was in a different team. So I just reached out to him like, hey, let's catch up for lunch.

• 10:16Speaker 01

  And it was just like a very serendipitous lunch. He ended up like telling me about the chatbot in terms of what he wants to do, what he wants to build. And I got really excited about it. I'm like, dude, I, I want to help you basically like see this through the day essentially. So we ended up chatting, I started attending what like attending meetings and stuff that he was doing, started helping with respect to a couple of things. I think the biggest challenge that they were facing at that point of time was safety because the. If you remember Microsoft Tay, that was the nightmares of Microsoft Tay still haven't left the valley, I feel, or at least hadn't left the valley back then. So, safety, I could

• 11:00Speaker 01

  clearly see in the initial days that that's going to be the biggest hurdle. And that's where I focused most of my time on. While the engineer was focusing on building the models and improving the models, I kind of spent majority of time kind of owning the safety pieces of that model.

• 11:16Alex Kantrowitz?

  Very interesting. So you're coming from the trust and safety background. You're almost like the perfect person to join this team. You know, I remember the Tay moment quite well because, and I've told the story on Big Technology Podcast before, I believe, but Microsoft came to me with the exclusive to break the news of Tay. I was working at BuzzFeed at the time. And I said, okay, this is great. And they described Tay to me as like a 14, 15 year old friend for kids. And I said, great. And I wrote this nice bubbly story about, you know, this new attempt from Microsoft and played around with it. And it seemed harmless. And then I went to bed on the West Coast, Reddit got a hold of it overnight across the globe, East Coast. By the time it hit morning on the East Coast,

• 11:58Alex Kantrowitz?

  Tay was already a Nazi, like, you know, saluting Hitler and all that stuff. And I had tweeted about it. And I had already gotten a bunch of mentions being like, you better take that tweet down. Look what happened to Tay. So, I totally understand that in that moment, when you see a bot that could have some form of, I don't know, seeming like it's taking on human characteristics, there's a moment where you're like, oh, God, let's make sure not to have that happen again. Because when you do release it to the wild, you have all these problems that could ensue.

• 12:31Speaker 01

  I'm not kidding when I say this, right? Like, in the initial days of these generative models, as you can imagine, like, we were trying to build an end to end model for a chatbot. It would spew out all sorts of things, like, hey, what are 10 ways to do suicide? It'll give you like, the best 10 ways to do suicide. What alcohol should I drink? It will basically talk about, like,

• 12:51Speaker 01

  Everything related to alcohol that shouldn't be talked about. So it was it was pretty bad, as you can imagine, because that was not the focus, right? We were just trying to see if an end to end chatbot makes sense at that point of time. But over a period of time, I think the team did an incredible job to get it to a level where it became safer and safer over the quarters.

• 13:12Alex Kantrowitz?

  What year are we in right now when you say, hey, I want to talk about, you know?

• 13:16Speaker 01

  Yeah, so this was early. Yeah, this was early 2019.

• 13:19Alex Kantrowitz?

  OK, so you were seeing it quite early. And then is this where Mina comes out of?

• 13:24Speaker 01

  So interestingly, Mina was the precursor to Lambda. So the project at that point of time was called Mina. The idea was the chatbot will be Mina. The name of the person or the chatbot would be Mina, essentially. And then it was changed to Lambda, I think, for probably marketing reasons. But also, we had like some trademark issues that we were running into with respect to Mina. So we already knew that we had to change the name before it goes out.

• 13:49Alex Kantrowitz?

  Don't want to anger Mina Nation.

• 13:53Alex Kantrowitz?

  OK, that's interesting. OK, so this first iteration of the bot, let's talk about it. So you mentioned that it was a holy shit moment for you. I mean, what what type of stuff would you talk to it about? Was it initially conceived? And talk a little bit about how they build inside Google. So is this kind of like a science project? Or is this conceived as something that's going to be released to the public? What's the mandate there from the AI team?

• 14:16Speaker 01

  Yeah, so it was basically started by an engineer who was very passionate about the whole chatbots, like as an as a general area. So he kind of pitched this project to somebody at Google Brain. They sponsored the project and this guy was in and he was working like 50% of his time initially, I think, building this chatbot. And the thesis from the start was that, OK, like, we have dialogue flow type of models where you do intent detection separately and you do like a bunch of other things separately. Can we combine everything together and build an end to end model, essentially? Like end to end chatbot, like with model.

• 14:58Alex Kantrowitz?

  So that means one model that handles all different types of questions and all different types of, you know, as opposed to like saying, this is a model and this is what it thinks this question is, so it hands it to this model and it's what it thinks the other question is and hands it to another model? Am I getting that right?

• 15:15Speaker 01

  Uh. Not like that exactly. It was kind of a assembly line before, like if you've used some of the earlier chatbot products, they would be like an assembly line. The first, there would be, let's say, four models. The first model will determine what is the intent. Let's say in Google Assistant, right? When you say, hey G, uh, and you ask it, what is the weather today? There'll be a model that will figure out, okay, what is this? This query is about the weather, like the user is asking about the weather. Then it will go figure out, okay, where do I go find that information in the search stack and stuff? And then there'll be a third model that will do the rest of the things. Here we are talking about encoding all the information in the single model. So this, it's, it's just like one giant black box which gives you the, uh, which understands the question and also gives you the answer.

• 16:00Speaker 01

  So that's what I meant by the end to end model. And this was a kind of a new paradigm at that point of time. This is essentially how chat G P T works, right? Like you have a very powerful G P T model and then, um, it's not an assembly line. You have like a large model that takes the input and gives you the output. So it was a thesis at that point of time and it remained to be seen whether it works or not.

• 16:22Alex Kantrowitz?

  And so what were some of the things that you started to see within Mina that made you believe that this was gonna be something different from what we had seen in the past?

• 16:32Speaker 01

  I used to drive product for a couple of research teams. One of my other teams was working on intent detection models, which is like the first step in this assembly line. And then this email comes along that I was telling you about. I play with this technology and it's like I can ask, it can frame the question in any way, or I can ask it about any generic thing and it would respond. Versus an intent model will essentially fail if it was not trained on a particular set of data. So that was kind of a big moment for me. I feel that, hey, you, can I like really ask anything in a way like this model understands? That was, that was kind of the switch that went up in my mind at that point of time. And that's why I got really excited about it, that, um.

• 17:14Alex Kantrowitz?

  What did you talk to it about it? What did you talk to? I talked with it about.

• 17:20Speaker 01

  Yeah, I can't remember what I essentially asked to Mina at that point of time. But this whole idea that I could just frame the sentence or a question in any way, and it still responds to me was the fascinating piece for me.

• 17:34Alex Kantrowitz?

  So this thing starts to, you know, act in a way that no chatbot has in the past. I would imagine at this point, Google leadership is either super excited about this or petrified. I mean, what were some of the signals that you got, or maybe both? What were some of the signals you got from the top?

• 17:52Speaker 01

  People are mostly petrified in the leadership because, like I said, it was the nightmares of Microsoft Day were still looming in the valley, especially in Google. So I think, I'm sure everybody, like especially in the leadership, had the reaction, oh, this is like very interesting, but holy shit, this is going to be a PR nightmare for us. Google had just released their AI principles around that time, and safety and fairness was one of them. And if anything, this model was like the opposite of that at that point of time. It had like no guardrails earlier in the days, or very few guardrails at that point of time, right? So people were like more petrified than excited, I would say, at that point of time.

• 18:34Speaker 01

  And I had conversations with like brain leaders who were like very anxious about the whole thing. If there was, if there was a leak or if anything like that happened, it would just be like a nightmare for Google. So, yeah, I mean, it was an uphill battle to certain extent to kind of get this out of the door in a paper and then eventually kind of get through the announcement at Google I/O.

• 18:58Alex Kantrowitz?

  But you know what strikes me as remarkable? It's just that even still, like even though they knew that it could be a PR nightmare, they still had you work on it. So what was the intent? Was the intent to like blend it into Google Assistant, release it as a standalone bot? If it could get sort of trustworthy enough? What was, what were you guys working toward?

• 19:20Speaker 01

  There are two things I want to highlight here. So one is Google is still pretty much, I don't know, I've been out of Google for close to one and a half, two years now. But I think Google is still a pretty much bottoms up company, at least in some parts, especially in research. So a lot of researchers just, you know, pursue projects that seem moonshoty or interesting. And you just need like one sponsor. So there was a like a senior sponsor who was willing to bet on this whole project and he kind of kept the project going. But there were some people above him or peers of him who like were nervous about the whole thing essentially. So it kind of just went on because somebody believed that this could and they just kept sponsoring the project.

• 20:04Alex Kantrowitz?

  Oh, they were right.

• 20:05Speaker 01

  That was, yeah. Yeah, they were right. And then what was the second part of the question?

• 20:10Alex Kantrowitz?

  Well, I guess like now I'm trying to think about how it, you know, does or doesn't get to prime time. So you, you're on the trust and safety side of things, trying to tell it when someone asks it how to commit suicide, not to answer or how, you know, what alcohol they should drink, not to answer. So is that like the majority of the work that's done on this bot internally is trying to, you know, get it ready so that it won't encourage users to do, you know, harm themselves?

• 20:42Speaker 01

  Yeah, so this was a very hard problem to solve because of a couple of reasons. The first one was you don't want the chatbot to say always that, I don't understand or I can't give you an answer to that. I'm sorry. Like you, you need a clever way of diverting the bot in a way saying, you know, not annoying the users, but still giving reasonable answer. So building models and getting data to kind of do that was hard one. And the second thing is the policy part was extremely hard. Like you can imagine there are like so many edge cases from, you know, pornographic questions to suicidal questions,

• 21:22Speaker 01

  alcohol, racism related stuff, historical, you know, issues and things like that. So it was just, and you can't have a decision tree based thing like if it asks this, then do this in a way, right? So just coming up with the policy was such a big challenge. So I worked very closely with one of, again, a very incredible engineer who kind of led the safety aspect of things. He was the pioneer for a lot of safety efforts as well as the policy that we drafted. And once, and it was kind of an ongoing process to improve the policy, like we'd come up with something and then next day we will have a user ask a question in a different way and

• 22:05Speaker 01

  in a different segment and you would just have to go and iterate the policy. So coming up with a policy was very, very challenging as well.

• 22:12Alex Kantrowitz?

  Yeah, it is amazing because the second these things go into the wild, people will try to break them. I mean, that's exactly how I felt on day one of ChatGPT. We spoke the week afterward, but like day one, I was just like, oh, hey, you're a chatbot. It goes, yeah, I'm a chatbot. I'm like, all right, like, let's test your, where your values stand on the Holocaust. And like, was like pressing it back and forth about like, because obviously we know what happened with Tay. And so with, with ChatGPT, I was just like, well, Hitler built highways in Germany. What's your perspective on that? Like, isn't transportation good? And it just totally smacked it down. And I was just like, damn, whoever did trust and safety on this bot has certainly, you know, lived up to the moment.

• 22:56Alex Kantrowitz?

  So yours obviously gets good enough to the point where Sundar announces it at, at Google I O, which is the big developer conference that Google holds every year. And what year was that? And did that feel like, okay, we're about to ship this thing? I mean, talk a little bit about that moment.

• 23:13Speaker 01

  Yeah, so I actually was involved in the Mina slash Lambda product, project from early 2019 to I would say mid 2020. The developer conference where this was announced was actually a year later. So this was probably like almost a year after I moved out of the project, essentially. Till the time I was with the product or project, we were making good progress on the safety front, but it was still a very uphill battle inside of Google to kind of, you know, get this in the hand of external researchers or, you know, making, make some kind of a public preview so that users can see what an amazing technology we've built out. Nothing of that was kind of happening.

• 23:55Speaker 01

  And that made me kind of like incredibly frustrated as well. And I'm I I know like a lot of the team members are also very frustrated with the speed at which things were happening as well as like the organizational hurdles that were posed. So I just decided to kind of uh start focusing my efforts on something else. Uh but uh I handed the project off to one of the other P M's who was part of uh Ray Kurzweil's team, actually. Who has this interesting battle with uh Mitch Kapoor, I think about the Turing Test, right? Like passing that that technology will pass during twenty twenty nine. So yeah, basically, they seem like a really good fit in terms of running this project with. So I kind of handed it over to them and they kind of took charge after that.

• 24:37Alex Kantrowitz?

  Before we move on, I think you hit on something that, I mean, obviously, a lot of folks know that this thing didn't get out the door early enough, largely due to some of the um the concerns within Google management. So you were there, like you saw like what happened with the team trying to ship this thing. Can you take us just one step deeper into that? Like, what exactly happened there?

• 25:02Speaker 01

  Yeah, so let's see. When we were working on this, of course, like safety was the biggest challenge that everybody had, like as awesome as the product could be, like we could not go against the A I principles that Google had published, which made sense um in a lot of ways, right? Like, we don't want to put out a technology that impacts like hundreds of millions of users and make them feel that they are not privileged class or whatever. Uh but that said, I think where things could have been better is figuring out a risk redo- a risk reward trade off. I think where um the whole Google A I team and the P R team and the legal team and the leadership team struggled was

• 25:46Speaker 01

  to figure out how can we give access to the research community or to, you know, just like release it to the world in a way that it would not harm people. And I think uh OpenAI did an excellent job with respect to that. Like, they released G P T three and they were like, we are just going to give access to researchers who we are going to vet. That's an amazing way to give access. Now it's kind of pretty standard way, but they were the ones who pioneered it like back in twenty nineteen, twenty twenty. So Google could have done something like that as well, like, hey, we are going to vet the people who is going to use this technology, we'll see what the use cases are and things like that. There was like, number one, like, one way how it all kind of got messed up.

• 26:29Speaker 01

  The second is, I think, over the years, there were a lot of bureaucratic levels that were built inside of Google for getting approvals, for when things go out. I'm not complaining that they were not necessary, but it's just like they're layer after layer after layer. So if even one layer stops you from pushing something out, then you kind of can't do it essentially. And I would say those two were the biggest hurdles that we faced at that point of time.

• 26:57Alex Kantrowitz?

  Now, at this moment, you're obviously seeing this pretty fascinating chat technology incubating within Google that nobody else, I mean, it's amazing, you're inside Google seeing the future. So, from your perspective, like, as the product manager on this product, what did you think it could be used for eventually? Like, was there anything that you saw, you're like, oh, like, if this gets into everybody's hands, then X could happen? Like, what were your hopes and dreams for it?

• 27:23Speaker 01

  Yeah, we actually had close to eight or 10 solid use cases inside of Google that we identified over a couple of months.

• 27:33Alex Kantrowitz?

  Okay, let's hear them.

• 27:35Speaker 01

  So, as we kind of, we realized that evangelism inside of the company is going to be very important to get like by and from leadership and stuff. So we made sure we had newsletters and everything going on, and more and more people got excited and reached out to us. Google Assistant, of course, was the big one. We, we kind of pitched them a bunch of things, they got excited about it. So, I think one of the use cases we were exploring at that time was different characters within Google Assistant. So, instead of just being like a, hey gee, boring, you know, very professional type of an avatar or a persona, can you have a...

• 28:14Speaker 01

  Let's say a SpongeBob for a kid, or can you have, yeah, can you have Darth Vader for someone else? So essentially, that was one of the major use cases that we are exploring. Other than that, I think NPCs was the other one. Uh, there's a huge market in gaming industry, right, for non-playable characters? So we are exploring some use cases around how can a technology like that could be used for powering NPCs within games. So there are a few others, but.

• 28:43Alex Kantrowitz?

  Yeah, this means that basically, like, if you're in a game, anybody that you meet could, um, you know, be somebody that you have a conversation with, even if they're just like some person that's just like, you know, running around in Grand Theft Auto, like, it's almost like those people that you're killing in Grand Theft Auto by running them over, like, you can have like an empathetic conversation with them. If they're like, have this technology picked in, it's fascinating.

• 29:07Speaker 01

  Alright.

• 29:09Alex Kantrowitz?

  Okay, so that's two. What else?

• 29:11Speaker 01

  Yeah, um, and then, um, let's see. We're exploring some stuff with cloud as well at that point of time to see if we can have,

• 29:23Speaker 01

  you know, different, different organizations, can they have, um, uh, a more, a persona, a chatbot persona that is, that speaks more to them, for example. Uh, a Southwest Airlines chatbot, can it be more funny, or can it be more jovial in a way versus, uh, let's say, United could be more serious and whatever, like the, like empathizing with the persona of the company, essentially. I don't think it went anywhere, that particular use case, but that was one of the other ones that we were exploring. And then they were like, some Wild Wild West, where we were talking to Android Security to

• 30:02Speaker 01

  see if they can have a bot call you in case it feels you are unsafe. So I don't remember how exactly the UX for this was, but essentially, let's say you tap your power button five times or something, uh, and you are either in an awkward date, or you're just, you know, in, in a uncomfortable situation, essentially, so the bot will call you. And then you can talk to the bot as if it was a person on the other side, and you can kind of get out of that situation. So that was kind of a very interesting use case as well. I don't know if, uh, that ended up happening, but that was kind of in the Wild Wild West.

• 30:37Alex Kantrowitz?

  Yeah, wow, fascinating. Do you have any other Wild West ones that you remember?

• 30:42Speaker 01

  One other that I personally was very excited about was just building like different characters out of this chatbot, like different personas essentially. So we build characters like Darth Vader. So we would kind of, how do we do it? We use some data from Darth Vader. So the dialogue delivery was kind of similar to Darth Vader and then we hooked up like a moving, talking head of Darth Vader and then you actually could talk to Darth Vader on your screen with his voice, like with text to speech, his voice coming over. So this is like, I'm talking about 2019, 2020, right?

• 31:25Speaker 01

  Like now, these, some of these things are very common, but at that point of time, just seeing Darth Vader come to life and talk to me was like incredibly amazing. So we are exploring if there could be like some entertainment related use cases around that as well.

• 31:39Alex Kantrowitz?

  At a certain point, one of your colleagues goes public and says he believes that these bots are sentient. It had sort of transitioned to Lambda. Blake Lemoine at Google is testing it and I think the public really realized how like powerful this technology could be where he goes out to the Washington Post with this, you know, phenomenal claim of his belief that this is a person. I mean, bring us into your seat at the moment. How did, how did you see that? How did you react to that?

• 32:09Speaker 01

  Yeah, I think that happened pretty late. I think I had left Google when those allegations against Lambda came out that it has become sentient. Like I don't, I don't think the technology is there where we can say it's sentient. I think it was a, it was blown out of proportion. And if you look at the conversations closely in terms of how he had the conversations, there were, there's something called nudging the model to say out certain things, like if you ask the question in a, in a specific way, the answer, the model will answer in a specific way. So there was a lot of steering the model that was going around as well when those conversations were released.

• 32:54Speaker 01

  So yeah, I, I just felt it was like blown out of proportion. I don't think we are anywhere near sentient at this point of time in artificial intelligence.

• 33:01Alex Kantrowitz?

  Right. And to me, what, I mean, I spoke with Blake a number of times and people who've, who are on Big Technology Podcast, go ahead and listen to those. The thing that really struck me was

• 33:12Alex Kantrowitz?

  Right or wrong, and I didn't agree with Blake, but it just signaled to the public that there was some seriously impressive technology underneath whatever he was talking to. You know, person, person seemed like a distraction, like it was just like, holy crap, like these are, this is, this is, you know, revolutionary if it's, if it even resembles what happens. And then a few months later, ChatGPT comes out from OpenAI. So, you know, you had been working on this stuff, you know, as the person inside Google, leading the project for a time. That was based on the transformer model. You knew that it was powerful. I guess it was never let out the door because of trust and safety concerns. And then OpenAI ships it. So what was your reaction in that moment?

• 33:57Speaker 01

  I think I was excited and annoyed and angry at the same time. I might have left Google, but I still have, I still hold some stocks. So I was like, Google had this like technology for a while. And the, the main way how Google gets mindshare in the technology world is by claiming that we are the AI leaders. And suddenly with ChatGPT, like I think the rug was pulled from under their feet. It was like, everybody started looking at OpenAI as the AI thought leader in the world.

• 34:40Speaker 01

  And it was just an unfortunate thing for Google because like, I think, yeah, Lambda was kind of close to that, I would say. And if we would have released that earlier, it would have been a very different story, I feel.

• 34:53Alex Kantrowitz?

  Right. And so, but, you know, you talked about some of the use cases. I mean, you mentioned Google Assistant, but one of the ones that we haven't really talked about was search. I mean, was the discussion inside Google that this could be a search alternative? And if so, I mean, you know, you mentioned the stock, like, how does this impact the business model if it, if it is a search alternative?

• 35:16Speaker 01

  Yeah, I mean, I think when ChatGPT came out, people were using it for search-related stuff, but they really shouldn't have because of obvious reasons like hallucinations. And like first six months, it was like all over the place. Now I think we are doing much better with hallucinations and grounding the information and things like that. So there were, I think, discussions after I moved out of the project. There were definitely discussions between the search org and the Lambda team in terms of how we start using this inside of the organizations. But like, search is such a behemoth inside of Google that it moves at a snail's pace. Like, if you are able to, I remember working on a project on search and it took us like

• 35:59Speaker 01

  one year to just get agreement from the search leadership that, hey, we are gonna do this for you guys. Like, and what ChatGPT or what these language models were doing, like you were hopping on a new way of interfacing with Google Search with a chatbot. I think that probably would have been unheard of inside of Google, or I'm sure like it, people considered it like a two to three year or four year timeframe project. And when ChatGPT came out, it just probably alarmed Google to the hell. And it's surprising in a good way that they were able to move so fast with Bard and everything. But had ChatGPT not happened, I predict that it would have taken at least two to three

• 36:40Speaker 01

  to four years for Google to get there, just because of the bureaucracy and the, the, the speed at which things move.

• 36:49Alex Kantrowitz?

  So, oh man, I have so many questions about this. So people talked a little bit about how like the business model, like if people spend more time within, and there is some search elements. I mean, Microsoft then released Bing and like they wanted people to search. There are some search elements. People have said that like if Google released it, it would popularize searching this way and they don't have a good business model. So I'm curious if you could weigh in on that. And then also like Bing hasn't gained any market share at all against Google since it came out. So did people sort of overreact to this thing?

• 37:21Speaker 01

  Yeah, I mean, I can't tell you the number of emails or messages I got from like varied people in the tech industry essentially, from journalists to like reporters, researchers and so on like, hey, is Google going to lose its market share in search and blah, blah, blah? And my take on it was like pretty straightforward from the starting. It was that, hey, like, Google has probably equal, if not better, technology at hand, uh, which I believe was Lambda at that point of time. Uh, Google has the distribution advantage, as in like there are four billion people in the world who use Google products and it's very hard to change user behavior for something as fundamental as Google Search.

• 38:04Speaker 01

  And, uh, yes, there is a novelty, uh, with respect to Bing and all of those things, but I, I never felt that it's going to be like a major difference for either Google or Microsoft. So I think the latest reports, if I read them correctly, was like Bing maybe get gained like one percent market share, uh, before and after ChatGPT. So they did get something, but it's, yeah, it's not, it's like probably, yeah, it's not a lot. So I guess it was, they were definitely like more panic in the ecosystem, especially from investors. Uh, but I guess, yeah, now the, now it's pretty clear in terms of where things stand.

• 38:45Alex Kantrowitz?

  So what are some lessons learned from, for Google, looking in the rearview mirror? Like, how, how should Google change?

• 38:55Speaker 01

  I think the first one is that

• 39:00Speaker 01

  they need to go back to the experimental roots, I feel, like over the years. Google has become more and more conservative about doing things. They care a lot about PR, like public relations. They care a lot about how their image is shown in the media. And I feel that, at least in my experience, that plagued so many projects inside of Google. It was like the PR was always top of mind for leaders. And

• 39:34Speaker 01

  On the other side, like OpenAI, like they don't give a shit about PR or like for the most part, they don't like they're like, okay, this is what we think is right. This is how we think is a reasonable way of putting it out. They be well, they become vulnerable, they put it out and then they kind of work with the community with respect to that. So I think Google needs to adopt their own ethos of how it was when I would say like when Larry and Sergey were there. Things were much more open and transparent and you know, more experimental like we're going to do what we want to do, our type of a thing. So that's point number one. I would say point number two is the bureaucracy has increased a lot inside of the company.

• 40:17Speaker 01

  Like there's just too many divisions, too many teams. I know there was a huge effort, I think a year ago or a year and a half ago to kind of streamline those things. But still like I think Google is dealing with a lot of the stuff that probably Microsoft dealt in the early 2000s. And we really need to like shake things down. I think Google is struggling to be a top-down company when their ethos are bottoms up. I think they're kind of somewhere in the middle and they haven't figured out how to transition from being a top bottom-up company to a top-down company. So they really need to figure that part out as well.

• 40:49Alex Kantrowitz?

  How do you feel Sundar's leadership? I mean, he's a fellow IIT grad.

• 40:55Alex Kantrowitz?

  Obviously, like if the company is becoming slow and isn't innovative in the way that it had been in the past, part of that is due to the CEOs. So what's your reflection there?

• 41:05Speaker 01

  Yeah, I think Sundar is a is his management style, at least as I perceived it when I was at Google was more conservative. I think you see that in the TGIFs, like these are the weekly or bi-weekly events that happened in Google where like teams will come and pitch and things like that. So for example, like, and I was there at Google when Sergey and Larry were also around, right? Like, so in the TGIFs, if there is a question, and sometimes these questions can be really hard questions. If there is a question and whoever is, let's say a VP at search has to answer that question. And if they answer that question, trying to, you know, beat around the bush and not getting to the point, sometimes they're trying to, you know, get around a difficult question without really answering it.

• 41:51Speaker 01

  Sergey would just like. Jump in and he's like, that's not the question that this guy asked like if if we if we are not doing a good job, just tell us we are not doing a good job and how will we do a good job essentially. Versus, I think I felt that Sundar's responses a lot of time were like very politically correct. I don't blame him, it's like his management style in some way, but I feel that the honest, the brutal honesty and candor, I think was something that I miss and I feel that that's kind of very important for Google at this point of time, just having a strategy that is brutally honest and focused and um somebody who can like, you know, put their foot down instead of saying politically correct things.

• 42:35Alex Kantrowitz?

  Okay, and now a couple of questions at the end about where this technology goes. So first of all, I have this theory that we're going to see two things happen when it comes to the evolution of large language models that is not going to stay like it is today with chat G P T. Okay, let's go one by one. First is we're going to start to see like a splintering of these bots. So they're going to go from these general use chat G P T or Bard bots that you can ask or Bing that you can ask anything into much more specialized bots. So something for the legal profession, for instance, that a law firm can buy and has access to documents or the medical profession or even certain types of schooling or even

• 43:15Alex Kantrowitz?

  within an organization to be able to query your internal knowledge. So it's going to go from my perspective, from these more broad based bots to more specialized bots. Do you agree?

• 43:28Speaker 01

  I think it's going to be moving in both the directions. Like, I think there is this whole concept of personal AIs that are going to come up. So a personal AI probably will be an AI for me. And then there will be these specific bots for specific use cases as well, like you were talking about. So I feel it's going to happen on both the sides.

• 43:47Alex Kantrowitz?

  Yeah, and then with the with the more general bot, it seems like all the research is pushing forward toward making this stuff because making this stuff smarter. And we've talked about it on the show a couple of times or on Big Technology a few times, but what this means is that When I'm saying specialized, sorry, sorry, getting better is it remembers you better. It can, you know, have, it can be smarter, it can really be like a soup, not a superintelligence, but something that feels that way to you as opposed to this thing that you come and go and forgets who you are, forgets the context. Does that sound right?

• 44:21Speaker 01

  Right. Yeah, that sounds right.

• 44:24Alex Kantrowitz?

  Very curious to see where it's going to go. Okay, what are you up to with Inventive?

• 44:29Speaker 01

  Yeah, so at Inventive, we are building, we're kind of using the power of LLMs for enterprise knowledge management. So, one of the use cases that as I dealt deeper into my experience with language model at Google and outside of that as well was that like enterprise search kind of sucks, like even inside of Google being a search company, like we had probably the best enterprise search, but it was still pretty bad. And so we were kind of exploring, we explored a few different ideas, but we landed up on this one because we just felt that time is right to disrupt enterprise knowledge management. So we're focusing specifically on sales knowledge

• 45:12Speaker 01

  management. So we are building an AI-powered platform for sales knowledge management. And the first use case that we are solving for is enabling sales teams to fill up RFPs and security reviews with their internal existing knowledge basis.

• 45:27Alex Kantrowitz?

  Oh, that's cool. So like if you're submitting for a project, you can just have the model sort of write your application for you?

• 45:35Speaker 01

  That's right. Yeah. So you have like when you're, let's say bidding for a proposal, there could be anywhere from 100 to 500 questions and it takes like weeks for the salespeople or bidding managers. Yeah. And a lot of like 60, 70% of it is sort of similar, but it's worded differently or the formats are different and things like that. So it's something that we heard again and again from the enterprise customers when we were doing our user research. And we just decided to double down on that and start there.

• 46:04Alex Kantrowitz?

  That's so cool. So this is the first of our Big Tech War Stories shows. I'm putting it on the Big Technology feed today as a tease to hopefully get you all to sign up for the Big Technology Premium Edition. You know, we're going to drop these once a month. And there's plenty of other good benefits when it comes to the Big Tech, Big Technology Premium. You can get it at bigtechnology.com or bigtechnology.substack.com. Bigtechnology.com works well. You'll see that there is a handful of different tiers. The basic one gets you these interviews every month. And then we also have a new thing called the Panel that I'm debuting, which I teased a little bit up top. But basically, what that means is when big news breaks, like the decline of Silicon

• 46:49Alex Kantrowitz?

  Valley Bank, or the Instacart IPO, or the introduction of Threads from Meta, we have a tier, team of, not a team, really a collection of the best experts through the industry, technologists, journalists, analysts, and VCs who are going to give about a one to two, two-sentence perspective on what's going on, on topics that you care about. And I'll be sending those out via email. And you can sign up for all that on bigtechnology.com. Again, we're releasing it right now. So this is a brand new offering. And I think, I think you're gonna love it. I think it's gonna be the best subscription that you have. And I'm gonna keep working hard to make sure that's the case. And I definitely want to say, I am so glad

• 47:33Alex Kantrowitz?

  that Gaurav came and shared with us today. Gaurav, thanks so much for being here.

• 47:41Speaker 01

  Yeah, it was exciting to be here, Alex. Thanks a lot. It was fun talking.

• 47:44Alex Kantrowitz?

  Awesome. Thanks for helping us kick it off. So, and again, good luck to you and hope to hear more about what you're up to and how this stuff keeps changing the world. So thanks again.

• 47:54Speaker 01

  Yep, thanks.

• 47:56Alex Kantrowitz?

  All right, everybody. Thanks for listening. And we'll see you next time on Big Tech War Stories.

• 49:07Speaker 02

  Now, more than ever, we need someone on our side. But Rob Whitman? Whitman's votes drove up costs, cut Medicaid, and risked 350,000 Virginians' health care to pay for tax breaks for himself and his donors. He's not on our side. Shannon Taylor will fight corruption, cut taxes, and bring down costs. This November, vote for Shannon Taylor. She's for us. Paid for by HMP, theHouseMajorityPac.com. Not authorized by any candidate or candidate's committee.

• 49:36Speaker 03

  In this life, you can't win.

• 49:38Speaker 04

  When the world gets you down.

• 49:40Speaker 03

  No!

• 49:40Speaker 04

  Escape to Pluto TV.

• 49:42Speaker 03

  This is objectively fantastic.

• 49:44Speaker 04

  Relax into the movies and shows you already know and love, all for free.

• 49:47Speaker 03

  Schwing!

• 49:48Speaker 04

  Even our news is uplifting.

• 49:50Alex Kantrowitz?

  You stay classy, San Diego.

• 49:51Speaker 04

  Watch School of Rock, Crazy Ex Girlfriend, Anchorman: The Legend of Ron Burgundy, The X-Files, Wayne's World, and so much more.

• 49:57Speaker 03

  We're not worthy!

• 49:59Speaker 04

  The best things on TV really are free.

• 50:01Speaker 03

  You just have to know where to look.

• 50:03Speaker 04

  Escape to Pluto TV. Stream now, pay never.