20VC: Does Value Accrue to Incumbents or Startups in the AI Race, Why Model Size Matters More Than Data Size, Why Artificial General Intelligence is Far Away, Why Carpenters Will Be Paid More Than Software Engineers & Future of Jobs with Richard Socher

Richard Socher is the founder and CEO of You.com. Richard previously served as the Chief Scientist and EVP at Salesforce. Before that, Richard was the CEO/CTO of AI startup MetaMind, acquired by Salesforce in 2016. He is widely recognized as having brought neural networks into the field of natural language processing, inventing the most widely used word vectors, contextual vectors and prompt engineering. He has over 150,000 citations and served as an adjunct professor in the computer science department at Stanford. In Today's Episode with Richard Socher We Discuss: 1. The Decade-Long Journey to Becoming an AI OG: How did Richard first make his way into the world of AI over a decade ago? What are 1-2 of his biggest lessons from working with Marc Benioff? How did 5 years at Salesforce impact how he both thinks and operates? 2. Models: Does Size Matter: How important is model size? Is data size more important? What are the biggest misconceptions people have around models today? How does Richard respond to the suggestion that "many startups are wrappers around LLMs"? Are hallucinations a feature or a bug? 3. Where Does Value Accrue: Where does Richard believe most of the value will accrue; startup or incumbent? Which incumbents are best positioned to win? Which are the laggards and behind? What do many not see about the startup vs incumbent race in the AI war? 4. Open vs Closed: Which Wins: Does Richard favour Yann LeCun's open approach? Or is the world of AI more closed? What are the biggest challenges of an open ecosystem? What are the nuances that make both challenging? 5. Richard Socher: AMA: Why will carpenters be paid more than software engineers in 10 years? Why is AGI still way off? Are people too unrealistic? How much money does Google make off search every day? Why does that leave them vulnerable?

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  The future is already here, it's just not equally distributed. And I think it will meaningfully change so many jobs. The more data there is about a job, the more that job is likely to be automated. But at the same time, there are still a ton of jobs where no one collects data. What that means is that the tasks that are physical are getting more and more expensive. And they're going to become the new bottlenecks. And then they're going to slow down overall progress so that the GDP can't like 100x because of AI. It can only increase less than we think it might when we're deep in the bubble.

• 0:33Harry Stebbings?

  This is 20 VC with me, Harry Stebbings, and I'm so excited for the show today as our guest is a true AI OG. As being widely recognized as having brought neural networks into the field of natural language processing, inventing the most widely used word vectors, contextual vectors and prompt engineering. I'm so excited to welcome Richard Socher, founder and CEO of U.com. Previously, Richard served as the chief scientist and EVP at Salesforce. Before that, Richard was the CEO and CTO of AI startup MetaMind, which was acquired by Salesforce in 2016. And check this out, Richard has over 150,000 citations. Bloody hell, that's more than I've done podcasts. But before we dive into the show today, you know all those mind numbing tedious tasks that seemingly take up half your day.

• 1:17Harry Stebbings?

  Well, Coda is here with their new AI powered work assistant that helps you and your team not just finish tasks, but make progress so your product team can bring a feature to market faster by using Coda AI to tag customer feedback, draft PRDs, suggest target audiences, summarizing product discussions and more. Your sales team can engage more customers by bringing in data from sources like Salesforce and then use Coda AI to suggest action items, meeting agendas or lead scores. And your marketing team can drive an impactful launch with Coda AI summarizing user insights, creating briefs generated from notes and writing mock press releases to visualize taglines. With Coda AI, you can reimagine your to do list and how you collaborate. So you're not only finishing tasks, but really making progress.

• 2:02Harry Stebbings?

  If you want a work assistant that lets you get back to work, you can get started with Coda AI today for free. Head over to coda.i slash two zero VC that's coda.io and get started for free. And speaking of amazing tools we cannot live without, travel and expense are never associated with cost savings, but now you can reduce costs up to thirty percent and actually reward your employees. How do you do this? Well, Navan rewards your employees with personal travel credit every time they save their company money when booking business travel under company policy. Does that sound too good to be true? Navan is so confident you'll move to their game changing all in one travel corporate card and a spend super app that they'll give you a two hundred and fifty

• 2:44Harry Stebbings?

  dollar in personal travel credit just for taking a quick demo. Check them out now at navan.com forward slash two zero VC. And last but by no means least, we need to talk cash. As of the twenty ninth of June, you can get five point five percent yield on your cash with twenty six week treasury bills. But buying treasury bills, it's not that easy and you have to navigate a website that looks like it was made before I was born. Enter public.com. Their treasury accounts make it simple to earn a high yield on your cash and it takes twenty seconds. Here's how it works. Sign up at public.com, easily purchase twenty six week treasury bills that automatically roll over at maturity for a compounding yield. Plus, there are no minimum

• 3:26Harry Stebbings?

  hold periods. You can access your cash at any time with the flexibility of a bank account. Of course, to receive the full guaranteed yield, of course, to receive the full guaranteed yield, you do have to hold to maturity. But here's the thing, these are T bills, which means your investment has the complete backing of the US government, making it one of the safest places to park your cash. Go to public.com forward slash two zero VC to lock in a historic five point four percent yield on your cash.

• 3:55Speaker 02

  You have now arrived at your destination.

• 3:58Harry Stebbings?

  Richard, I am so excited for this. I have been looking forward to this one for a while. So thank you so much for joining me first. Thanks for having me. It's a great podcast. You are very kind. I appreciate all ego inflation, but I'd love to start today with a little bit of context because you've been in the world of ML and NLP for a long time. How did you make your first forays into the world of ML and NLP? And what did that?

• 4:20Speaker 00

  Like, boy, that goes back to two thousand and three is when I started linguistic computer science at Leipzig University. Kind of the forays and the early days of natural language processing. But then I actually felt like there wasn't enough math in it and I switched to computer vision. Did my masters in medical computer vision and a lot more sort of statistical and pattern recognition, statistical learning. And then during my PhD, I saw some folks work in deep learning and neural networks for very small images. You know, thirty two by thirty two pixel images of digits and things like that. And I thought, couldn't we use these ideas that they're using in vision for natural language processing?

• 5:01Speaker 00

  And that sort of started down the road of contextual vectors and inventing prompt engineering and trying to train a single model for all of NLP.

• 5:09Harry Stebbings?

  Given how steeped you are in the community, in the technology, we've obviously seen the recent hype cycle around AI. And I really wanted to start with a landscape evaluation of, with the historical context you have, is now a fundamental shift in AI? Or is it the result of hype cycles doing their work?

• 5:29Speaker 00

  It's a great question and it's actually so difficult to navigate for a lot of people who are not deeply in it because we are at the beginning of an exponential improvement in a lot of different capabilities. At the same time, some people think the exponential will just keep on going and we keep having these major breakthroughs. And I do think there's some inflated expectations also. People think they have to use a chatbot for every single thing out there. Just like they used to think we have to use speech recognition for every single thing out there. And, you know, back in the day, when speech recognition finally started to work, people thought, oh, I'm going to have like this restaurant recommendation engine in speech. That's actually probably not a good user interface, right? So, long story short, the entire level of AI capabilities is rising massively.

• 6:13Speaker 00

  But then on top of it, there's sort of small waves of inflated expectations.

• 6:17Harry Stebbings?

  You mentioned before something that was really interesting, and I want to start that. But you've been on a quest for a single pre-trained model. I just want to make sure everyone follows along with us in this conversation. For those that don't know, what do we have today? And why is that maybe inefficient?

• 6:32Speaker 00

  So today, it's finally changing. We've made progress towards that single model. But just even last year, or two or three years ago, the prevailing idea was that every task in natural language processing should have its own model. You have a sentiment analysis model that just classifies tweets as positive or negative. Then you have a summarization model that takes in some long input and then summarizes it in a fewer sentences. You have a translation model that just translates German to English. You have a question answering model that takes a context and says, who's the president in this Wikipedia article that's mentioned? And then you just give that. So there are all these different sub-modes that people have worked on.

• 7:15Speaker 00

  And some people have built their entire careers on just sentiment analysis models. And so the difference in something I've been very excited about for pretty much a decade now is to have a model that you keep making better, that you keep adding to, rather than restarting every new training run and so on. Imagine Wikipedia, and everyone just keeps adding to Wikipedia and keeps making one dictionary better, rather than everyone who wants to build a dictionary just starts their own dictionary company and then builds it from scratch. It just doesn't make as much sense. It makes sense when humanity and people and researchers work together and keep making a single model better and better.

• 7:53Harry Stebbings?

  Richard, I think one of the reasons why this shows success is because I'm not afraid to ask the slightly more basic questions. When you explain it like that, it seems highly logical that you would improve and improve and improve upon a single model versus start again. Why is that not obvious? And why has it not been that way?

• 8:11Speaker 00

  Oh, man, yeah, I had some funny rejections on that. Actually, the paper, when we submitted this prompt engineering paper, it got rejected. And the reviewer said, oh, this is such a misguided effort. There's not a single model in the world that could do any of these things together, to which I thought, the brain, it's like, we don't replace our brain. We use the same brain. So there is an existence proof for a single model for all of NLP. But it was just a very different way of thinking about the world. There's this debate, and people have done a lot of research and were very stuck in one way of thinking about it. There's sort of academic preconceived notions about how things have been and how certain people have worked on certain problems for a long time.

• 8:54Speaker 00

  And the notion that you should have a single model for each specific task was prevailing for a long time because it's also really, really hard. You needed the ability to train massively large models. You needed the ability to incorporate world knowledge through a large language model to really make that happen. And you needed attention mechanisms and fast GPUs and hardware. There's a lot of things to make it work to then eventually show that it was better. And when you had very small models 10, 20 years ago, it would have been impossible to make it work.

• 9:23Harry Stebbings?

  How important is the size of the model, Richard?

• 9:26Speaker 00

  It is super important. You just cannot train a single model for all of these different tasks with a small model. That's exactly why and how it would have always failed in the past.

• 9:36Harry Stebbings?

  I've had guests on the show before and they say the model size isn't so important, but it's the data size that is. Is that wrong?

• 9:44Speaker 00

  It's totally not wrong. It's just not mutually exclusive. You need a large model and you need a lot of training data for that model, either of them in isolation. Like, you know, just like imagine the simplest neural network will just predict a single one dimensional output line, right? It's like a regression analysis. You have some input X, some output Y, and you try to model where it goes. You can model that with a handful of neurons. And the simplest one is just like a line, right? A linear regression. And that model has even fewer parameters. Long story short, if you now give this linear regression model billions and billions of training data, it's not going to learn magically anything but a simple linear line.

• 10:25Speaker 00

  But if you give the model billions and billions of parameters, it can learn all kinds of very complex predictive functions and abilities. So concretely, the big breakthrough on top of this idea of prompt engineering of being able to have a single model. Was to also use language modeling as one of those tasks. It was actually on our to do list, but we didn't get to it before others did. And the idea of language modeling is you just predict the next word, which is very easy to get a lot of data for because you can use anything on the internet and so on. But it's actually incredibly hard. And to do it really, really well, you have to learn so much about the world, right? If I'm just, I have this sentence like, I'm in New York City and I'm driving north too, right? And now you want to predict what's the next word?

• 11:09Speaker 00

  Maybe it's Boston, maybe it's Montreal, maybe it's Yale. But you probably want to put more probability on the word being Boston than Yale, because there are more people probably driving to Boston because it's a bigger city. And so not only do you learn about geography, but you also learn just to predict that one word really, really accurately from that one sentence, you need to learn everything about the geography of the Northeastern United States. And now if you do this billions of times across the internet on all the chemistry articles and biology articles and so on, you learn world knowledge, you infuse world knowledge into that large language model, but only if you have enough parameters to learn it all.

• 11:46Harry Stebbings?

  I need your help. I think there's straight away about the access to this training data. And some people say, ah, this is where incumbents really thrive. They have the customer data, they have transaction, they have all this data, they can utilize that. And others say, no, there's so much open data today that actually access to data is highly democratized. Which side would you sit on? And how would you think about that question? Because I don't know the answer.

• 12:10Speaker 00

  It's so funny because it's another question where they're actually both right. So it's complicated in the sense that unsupervised data, just raw internet text is easily accessible. But there are still a lot of data sets out there that are not out there. They're actually stored in private databases. And indeed, if you want to answer customer emails automatically.

• 12:38Speaker 00

  It's very helpful if you're Salesforce and you already have all those emails. You already have labels that people, this knowledge-based article answered this email or answered this question, right? If you have that data, you can then train that particular AI for that company to answer its questions from its customers automatically. You can do that much more easily than if you're a small startup and you need to just get access to that data and to get permissions and so on. And so that part is true. At the same time, the state before large language models, and we call them foundational models because you can often build on top of them very easily. Before that, it was even harder. Like, it would have been impossible for us as a small company to build a search engine

• 13:20Speaker 00

  that understands all of these different things in many different languages. It would have been unthinkable, but now we can actually, because of these large language models and foundational models, we can have a general sense of understanding natural language. And you can, as a small startup nowadays, build in like 80% solution very quickly. You layer more and more specific data for your task on top as after you have an MVP, a Minimum Viable Product, and then you make it really, really good. Now, the big incumbents, they can make it really good much more quickly, but you could get to at least an 80% solution thanks to large LLMs like more quickly also.

• 13:56Harry Stebbings?

  A lot of people suggest in the venture community, so you can poo-poo this one, but like when they're denigrating kind of a lot of AI startups, they say, oh, it's a thin value layer on top of foundational models. And actually, really, the value accrues to the foundational model there because it's this thin line on top. Is that fair? Or do you think that's total bullshit?

• 14:16Speaker 00

  You're asking a very good question that often have a more subtle answer than what would fit in a tweet. I think there are some very thin wrapper companies out there that probably have very little moat, but there are also companies that people don't realize the complexity to make it really work. And they underestimate all of a sudden all the other stuff that companies need to get right to build a viable business. So concretely, you know, you could think about Instagram, right? Instagram is like, what's the moat of Instagram? It's certainly not their AI and their backend and the brilliance of engineering. It's just like a fairly simple photo-sharing app with some fun filters back in the day. None of that was rocket science.

• 14:58Speaker 00

  Turns out you can have moat other than your backend AI model, right? It's distribution, it's partnerships. Your sales funnel, processes and so on. So there's a lot. And then there are also areas where the default large language model will not do as well. So for instance, if you want it to be more factual, more up to date and have citations for the facts that it tells you, you need to have a search backend. And so at u.com, for instance, we've had to build this very complex search backend with a ton of data in it and knowing when to retrieve what facts from the internet so that your model, you can ask about Messi's Miami switch

• 15:39Speaker 00

  or something and it will just talk to you about that, even though the LM itself couldn't be trained. And if anything, you can think of these LMs as like reasoning engines, but you still need to feed them with the right facts and information so that they can reason over the right things rather than just sort of reason with what they remember. And their memory is a little bit like maybe your uncle who sometimes exaggerates some idea like stories from the past and doesn't remember all the details exactly. He often still gets it right, but not always. And so you want to infuse the fact into the LM and then reason over it. And that whole retrieval backend is also highly non-trivial, but it's something that some VCs don't appreciate and understand the complexity of.

• 16:22Speaker 00

  And then they say, oh, like u.com is just a thin wrapper around a large language model, which is very far from the truth.

• 16:27Harry Stebbings?

  You mentioned that about your uncle, sometimes kind of being hyperbolic or exaggerating. It was Iman at Stability who said on the show that it's like, I think it was like a crazy smart student who sometimes goes off their meds is what he described a model. And he said that hallucinations are a feature and not a bug. How do you think about hallucinations as a feature, not a bug?

• 16:48Speaker 00

  Yeah, you know, it depends again on the context. Your questions are perfect in that they help people understand it's neither black or white, but it's just some complexity in between. And the complexity here is that indeed the LM doesn't know your intentions and your background. And so it takes some time for them to adjust, to understand and for us to train them and then to balance out like what it can and cannot say. And I think some cases, we even overshot a little bit. You know, there are some folks who say, oh, you cannot tell any dirty jokes anymore. Or like a murder mystery is like unethical to write, so we should not ever write about a murder mystery. But no one says Stephen King or a Catholic Chrissie are like unethical people for writing murder mysteries.

• 17:31Speaker 00

  People just think it's great entertainment, but they're scared when an AI slash a large language model predicts a tokens given that prompt to, you know, write about that kind of stuff. And so in some cases, we almost overshot what we can and cannot say or hallucinate about. But hallucinations aren't a feature and are a problem for a search engine, right? We do want to be, most of the time when someone asks us about a fact, we need to bring in the right fact and then be very accurate.

• 17:59Harry Stebbings?

  You mentioned that they don't know your context, which I think is important. But we had another guest on the show that said, no models used today will be used in a year's time. Do you agree with that? And how do you think about the longevity and lifespan of model usage?

• 18:14Speaker 00

  That's a great one. That's sort of right in the sense that we're going to update all these models. Like we update our model every other week. And so it's not that exact model with those exact weights. But my hunch is we will still have a lot of large language models that are running in production that are that general model architecture. I mean, there'll probably still be some transformers, even though, you know, transformers are mostly amazing because they can be trained on GPUs. There are lots of other models that we're currently not exploring because they're not trainable on GPUs.

• 18:44Harry Stebbings?

  I'm like taking a collection of wisdom from other people and throwing it at you to hear your thoughts. I had Alex Nabl on the show and he said that actually a company's ability to transition between model is what will determine their success moving forwards. How do you think about companies transitioning between models as a differentiator and advantage mechanism?

• 19:06Speaker 00

  Interesting. You can almost start to, in the future, think about the LMs as akin in some ways to a database. You don't really care about which database people are using nowadays, right? If you want to go really big, you might use an Oracle database. If you just want to build a small Thing use some MySQL open source database and there's a bunch of others in between. And I think it won't matter that much which database you use, just like it won't matter that much which LM you use. But it matters what you do with it, how you tune it, what kind of train data you add on to it to fine tune it. How you retrieve facts into it so it can reason well over all of those things.

• 19:48Speaker 00

  How you maybe prompt it to run multiple chain of thought steps to get to a conclusion. All of those things I think will matter more and more. And so I would argue that it's even more important than switching between models is to be able to incorporate multiple different models. Cause you may have a predictive forecasting model and that is important to include into an LM. Like an LM won't be as good in doing a financial forecast. Cause that's not what they're trained on like LM's help a ton for all things natural language. Cause they understand so much about natural language and will have so much world knowledge. But that doesn't help you if you just have a huge sequence of numbers and you need to build a very, very accurate statistical predictive model for like forecasting or something.

• 20:32Harry Stebbings?

  Final one before we do open versus close, but on the foundational model side, do you think all the foundational model companies that have been created today are the incumbent layer? Do you think there will be a new incumbent foundational model layer or have the winners been created already?

• 20:48Speaker 00

  I mean, certainly you can't deny that OpenAI is ahead by a lot. I predicted that we'll have a GBD four equivalent model before the end of the year that's open source. Of course, GBD four keeps getting better and better. So my prediction was for the version we had like a few months ago. But I actually think that with models like Lama two from Facebook and everyone there. I do think open source will take over a lot of use cases, right? It's already getting close to GPT three point five. When there's this much excitement. In so many careers, depending on understanding these models, imagine all the researchers in all these universities, right?

• 21:27Speaker 00

  They're all of a sudden kind of out of a job unless they have an L M that works really, really well and they can do useful things with it. They're not gonna just say, oh, let's just from now on run our entire research agenda on some closed API that we cannot analyze and understand and improve and publish papers on. So they need to have a model to exist and those are all very, very smart people. Now they don't have as many resources usually, they can't train a single model for like 20 or 50 million dollars because they're in universities, but they're finding ways, they're collaborating and they're probably going to work on foundational models that are fully open source. And we now see this with like surprisingly, Facebook being at the very forefront of it.

• 22:11Speaker 00

  People will layer on top of that, make it better. And then there will be open source versions you can run on your phone and those will get better and better over time. And so I'm quite bullish on the L M's in particular, getting more and more commoditized. And yes, there will be a few foundational companies, you know, here in Anthropic also behind OpenAI, working very hard to catch up with them and they did raise a lot of money too. And it's good to have some competition in that space, but my hunch is a lot of people will be okay with an open source model too.

• 22:42Harry Stebbings?

  A duet on the show from contextual said that actually it was the lack of alignment between open ecosystems, which would lead to closed winning, lack of alignment around bluntly goals, timelines, decisions. And that actually to think that open would win would be naive. Can you see his point or do you think that's not right?

• 23:02Speaker 00

  I mean, I certainly see that there's a lot of complexity in open models, but you also have already seen a lot of open models, right? Like Lama two, this was probably like a week or two ago, Lama two came out, right? And Lama two is incredibly good and it made even more progress than the past open model. And so we might not see for a while until some really strong, smart engineering project manager comes along that can herd all these incredibly smart and ambitious cats to come together and build a single model. But that would be my dream is that we actually have a single model. And just like Wikipedia, the whole world collaborates and adds to it. And then we get ultimately.

• 23:47Speaker 00

  An AI that anyone on earth can talk to, just like anyone on earth can ask facts on Wikipedia. And you know, like Wikipedia takes a lot of coordination. These articles have lots of layers and hurdles you have to jump through to make an edit to that article. You have to build trust with folks and whatnot. But it has happened in the past that when there is enough interest and excitement and a strong structure, that people can collaborate on open models.

• 24:11Harry Stebbings?

  I love that vision and I love that excitement for that view of the world. Why would it not happen?

• 24:16Speaker 00

  Coordination is hard, right? Someone needs to come through, needs to get funding. There's a lot of complexity.

• 24:21Harry Stebbings?

  Do you think that's the role of governments to provide the funding?

• 24:24Speaker 00

  I think governments can help. The problem is it's very hard for governments to fund uncertain research and science projects. Because when it doesn't work out, some taxpayers will for sure complain. They're like, you guys wasted my taxpayer money. It's hard to build that. It will then inevitably have to become even more bureaucratic than we're like actually has some necessity for and so on. But I do think governments should fund science and especially open science.

• 24:53Harry Stebbings?

  When we chatted before, you said to me that search was the highest impact of NLP technology. Why? Help me understand that.

• 25:02Speaker 00

  Yeah, obviously it's a massive market, right? We have a trillion dollar plus company value in that. But more importantly, intuitively, we ask search engines questions every day to learn something. They ask in their phone search engines a lot. And so it's an incredibly impactful technology to help people learn. That's sort of another really big foundational block for the internet. Some people even think like Google is just sort of a Yellow Pages, right? I mean, there's so many examples and use cases where you can do so much better than what we had in the last 15 years.

• 25:37Harry Stebbings?

  Richard, how do you do this without killing those original providers? And what I mean by this is say, I type in a question into u.com. U.com scrapes it for the best information it can, answers incredibly well. I now don't go to the original site because you provided it for me instead. That site, often a news site or an information site or a content site. relies on clicks and traffic. How does the next generation business model of the internet work? When that attribution and that throughput is not to the provider, but it is to this chat search?

• 26:10Speaker 00

  Yeah, it's a great question. We need to have that search engine that we've built be an open platform where you can actually contribute your apps to and your content to. And then if it actually brings up the context from your app and we make money with it, then you can also participate in making that money. Or you have some subscription model, right? And you have some data that you show publicly, but then you have to subscribe. You can have that subscription happening right within your search engine by virtue of this being a much more open platform than Google. Now, we've launched this open platform last year, but to be honest, we haven't had a ton of really amazing apps being added to this platform because we just don't have hundreds of millions of users.

• 26:54Speaker 00

  And so if your app lets you book a kayaking trip in Indonesia or something, by the end, like you, you only get like five people. And so been slow in an uptake on that open platform. But I think that is philosophically the right solution. And I hope people will start collaborating with us. Because if we don't win and others win, then they'll be actually fully left out.

• 27:15Harry Stebbings?

  You mentioned slow on uptake. It just makes me think to a question that I ask myself so often in in this world, which is, will the incumbent acquire innovation before the startup acquires distribution? How do you think about that in this case here?

• 27:27Speaker 00

  Yeah, it is the main question. The truth is, distribution won't be ever fully solved and it's a constant uphill battle. You know, we just got de-ranked in all our sites from Google and saw like a drop right away. So we have seen that drop from getting delisted out of Google. We're working on a lot of different partnerships and yeah, constantly thinking of clever ways to do it that because we have some competition, small and and large, that, you know, have been copying many of our features in fairly quick succession. We have to be a little more careful there and not share all the details.

• 28:01Harry Stebbings?

  I mean, delisting, it's quite petty, isn't it? Where do you think value most accrues if you were to bet on one in the next five years? Does the next wave of AI create more value for incumbents or more value for startups?

• 28:12Speaker 00

  I think it'll be a mix. I think there are companies that have been fully activated. I see like Salesforce having launched a bunch of really incredible features and made incredible announcements in their AI day. That will make it harder for AI for service automation, for instance, because it's so core to their business. We've also seen Bing and Google copy what we have launched late last year with YouChat. They've copied us in the sense that we've launched it earlier and then they launched something very similar three to four months after. At the same time, we don't see Google change and become a chat for a search engine. They have some features somewhere else, but the main Google experience is the same.

• 28:50Speaker 00

  That kind of big change will be hard for Google too, because they make $500 million a day with privacy invading advertisements on that page. And so you don't just really nearly change most of that page and you get rid of the five, six ads that are on top of that page, followed by a bunch of SEO and microsites that are not as good as the ads, so people click more on the ads. Like you don't just replace all of that with a chat, right? You just lose hundreds of millions of dollars a day. And so there is still some innovator's dilemma that will not make them change their main experience overnight. It's been very carefully tuned. Every shade of blue has been A B tested to death. It's very hard to make a massive change and improve that entire revenue overnight.

• 29:34Speaker 00

  So they'll slowly move carefully in their default experience and that's sort of one opening that we have. But it's not going to be easy. You have to keep innovating and you have to keep working on partnerships too and try to stay clever and a little bit paranoid as a startup CEO in a space where you have these huge incumbents, trillion dollar something companies that would love to crush you.

• 29:53Harry Stebbings?

  Five hundred million a day, my word. Okay, final one before we do one final segment on AGI, which I'm super excited for, I have to say. Which incumbent do you think's done the best and which do you think has actually lagged behind?

• 30:06Speaker 00

  I'm a little bit biased because I still love Salesforce in my time there. I think they have done a phenomenal job. You know, when we invented prompt engineering, we actually did it at Salesforce Research when I was chief scientist there. And so they've been at the forefront for a while, thanks to the research. And so they've been very, very quick also in incorporating that technology into its products. So that's been really great to see. And then, of course, in my space in search, you can't deny the fact that we have evoked the giants, right? Like both Microsoft Bing. And Google are much more actively working and innovating than they had been for the last 10 years, right? Like since we've launched Uchat and had the first LM with a search backend and

• 30:47Speaker 00

  citations and web links and so on in the search context, which we launched last December before anyone else in the world. Since then, search has changed more than the last 10 years combined. That is exciting to see. And in some ways, at a very abstract level, our mission has succeeded. We wanted to improve the state of the art in search. We did it. But company wise, we have a lot more work to do to actually financially benefit from those changes even more.

• 31:15Harry Stebbings?

  I do want to discuss one final element, though, and we touched on it before the show, actually. But it is around AGI, because there's a lot of people who are very excited and optimistic about AGI. But you mentioned to me that people might be overly optimistic. Why do you think that people are potentially overly optimistic around AGI, Richard?

• 31:32Speaker 00

  I think it's very natural for people to look at a type of progress and then extrapolate it further and further. And I think there's a little bit of overly strong optimism because we have, again, made a ton of progress, right? Like, it's undeniable how much better AI has gotten. But just like with flight, for instance, human flight, we went from the first motorized human flight. And then literally 30, 40 years later, we could fly loopings with machine guns and full metal airplanes high up on the speed of sound. And you're just like, wow, I mean, at this rate of progress, we're gonna have vacations on the moon. And we're gonna have flying cars. And like, everyone will just fly everywhere all the time and so on. And then

• 32:17Speaker 00

  in the 50s, the whole thing just stopped. And we're flying slower often now than we did before. People realize all kinds of issues. It burned a lot of gas, like airplanes and so on, the way they're structured right now. And we're slowing down. And like, there hasn't been that much improvement. Like, if you think about the space shuttle, it used to be that you could land coming out of space. You land like a plane in the space shuttle. Now we just drop people in a bucket with a parachute and it falls somewhere in the ocean. It doesn't look like we've made that much progress in terms of returning from space. But progress is not as linear or as exponential as a lot of people think.

• 32:56Harry Stebbings?

  What in this sense do you think that is? Science and the laws of nature, physics, gravity versus human reasons? Skill, funding?

• 33:05Speaker 00

  It's all of the above, right? But maybe in terms of AI and the progress, right? You think like we have this, keep having exponential growth. But I'll give you an example where we already see the flattening out of that exponential growth into an S curve. And that is an image generation. It's photorealistic now. Look at some of these mid-journey images. You can just say, I want this exact thing and it'll just give you a photorealistic image of the scene you just described. Where do you go from there? You can't be like hyper, super duper photorealistic. Like people only can see an image, right? And that's how good it is. Now, you can, of course, still make a ton of progress in video generation and like make them moving. But image generation itself is basically maxed out now. It's hard to go.

• 33:48Speaker 00

  And then in some ways, people think, oh, the AI has this superhuman capability. And you could argue, yes, like one AI algorithm can translate decently well into 50 different languages. No human can do that. No normal human can translate that many different languages. But it's also hard to imagine what superhuman language is because language is a human construct. And if it was superhuman, it would just, AIs be talking to each other with like 50 parallel streams or thousands of parallel streams. But humans can only register and understand language sequentially, right? You can't have five conversations at the same time. You can't read five books at the same time. So language being a human construct and being how humans communicate thought and ideas has a limit on how superhuman it can be.

• 34:33Speaker 00

  It doesn't make sense to have superhuman language because humans couldn't understand it anymore to some degree, right? Again, you can produce it more quickly. You can generate a ton of stuff, but humans can only consume it at the rate that they have over the last several years.

• 34:47Harry Stebbings?

  It seems like there's this mismatch, though, between the asymptotic point of development there, as you said, with image generation, aligned to the distribution awareness. And what I mean by that is if you go to most people on the street in a normal town, not San Francisco, they've got no idea what mid-journey is. There's no question here. I'm just like, it's weird that we've reached plateauing technological development with zero awareness of it in 6.9 billion people.

• 35:13Speaker 00

  You're 100% right, and I'll just jam on that. But there's a famous saying, which is, the future is already here, it's just not equally distributed.

• 35:20Harry Stebbings?

  How long does that take?

• 35:21Speaker 00

  I mean, that's the interesting thing, whereas like I'm so bullish and excited about AI. I have been for over a decade and I think it will meaningfully change so many jobs. The more digital they are, the more data there is about a job, the more that job is likely to going to be automated and improved massively in its efficiency. But at the same time, there are still a ton of jobs where no one collects data. No one collects data about how my maid cleans and what are exactly all the inputs and the outputs and the actions and so on. Every house is slightly different. It's hard to constrain the environment. Full self-driving on off-road, dirt road, like small roads, nighttime fog, like all of these things won't work for a long time.

• 36:04Speaker 00

  You know, but the more constrained the environments are, the more the AI can do, the more data we can collect, the more the AI can help us automate and make things more efficient. But what that means is that the tasks that are physical are getting more and more expensive and they're going to become the new bottleneck, right? So you're carpenter, people like who built your house, all of those kinds of jobs are going to be more and more expensive. And then they're going to slow down overall progress so that the GDP can't like 100x because of AI. It can only increase less than we think it might when we're deep in the bubble.

• 36:37Harry Stebbings?

  Richard, I'm giving you a warning that this is a shit question, okay? So warning ahead of time. Are you concerned by the job displacement question? And just the awareness that when you look at industrial revolutions and technological revolutions, there's always decade-long actually transition periods. Whereas here it seems like the transition period is years, like a couple of years, not decades. Are you concerned or do you think we're overwiring about this?

• 37:02Speaker 00

  A hundred percent. I do think, you know, past industrial revolutions, they usually happened for an individual at a surprising rate. When you're a weaver and you got some big machine now and it's just like totally automates making clothes. You will hate that machine, right? And the Luddites tried to destroy those machines and and there are people who want to slow down and destroy certain kinds of AI that change their their jobs. And I feel empathy with those people. And it would be nice to have social systems that catch some of those people and help them learn new kinds of skills, help them incorporate those new technologies into their workflows. So that, you know, if you're an illustrator and you used to be able to charge a thousand dollars for an illustration,

• 37:47Speaker 00

  because it took you three days, now it takes you three minutes. If you don't use it and you still expect to get paid for three days, but now there are thousands of people who can do it in three minutes, that if you're not adapting to it, it will change your job landscape. And so to try to help people in that transition is incredibly important. I think it's also hard to say, let's not make things more efficient. Like, let's not have more art, let's not have in healthcare, like more healthy people cheaply, let's make more jobs. Feels a little bit like you'd want to slow things down. And some people feel like they want to go back to the mountains, right? And just live a simple lifestyle with no technology. You, you see that now come up sometimes, but I, I think overall a middle class person now lives in along many dimensions,

• 38:31Speaker 00

  a better life than a king did a thousand years ago, or even a few hundred years ago. Most of the time, people appreciate the end state of those transitions. Like, we can have more cheaper food now, there are fewer people who have to starve. You know, 150 years ago, over 90% of people worked in agriculture just to put food on the table. If you told them, hey, we'll have these massive machines, if you stand in their way, they will crush you to death. But like, they'll do all of this farming automatically. And now only 5% of people need to work those kinds of jobs. They would have said, that's really scary and what are we going to do? But people always find new things to do when there are efficiencies created in existing things.

• 39:09Harry Stebbings?

  I do have just one final thing before we do a quick fire. And it's just, we've mentioned before AGI and kind of why it's potentially overhyped or overexcited. What are the three barriers you mentioned to me before that prevent an AGI or will put a pause on it?

• 39:22Speaker 00

  We do have a lot of research breakthroughs we still need to make in order to achieve AGI. Just like it's very hard to know when someone would have been problem engineering, or when someone would have the idea to scale up language models and transformer networks just massively and not have. Other cute little models and modify those, but just scale up existing models massively like OpenAI and Ilya Sutskevra and others ideas. We don't know when those breakthroughs will happen. I find it hard to call something artificially super intelligent or generally intelligent. If it all it does is predict the next tokens and you say, oh, predict this next token, it'll predict that next token.

• 40:01Speaker 00

  I think an intelligent existence probably needs to have some of its own goals and like some of its own mind of what it might want to do. Maybe it doesn't want to generate tokens. If you just want to maximize the right token prediction, you just make sure everyone says AAA only ever. And then you predict the right token a hundred percent of the time, but that wouldn't be very interesting of fulfilling intelligent existence. And so because companies need to make money and governments want to have a productive economy, no one is working on AI just doing whatever it wants to do. Because that doesn't make any money. So no one's working on AI setting its own goals. And until that happens, like it's going to be hard to think about general intelligence of almost life form that it has some some intelligence,

• 40:45Speaker 00

  even if it is not a biological.

• 40:48Harry Stebbings?

  Richard, I'm going to do a quick fire with you. So I give you a short statement and you ping me a quick thought back. It's as fast as possible. Does that sound okay?

• 40:56Speaker 00

  I'll try my best. As you, as you may have noticed, I'm not really good with the short form.

• 41:00Harry Stebbings?

  Neither am I, my friend. So what do others not know that you know to be true?

• 41:06Speaker 00

  Knowledge similar to the future is already here, but not equally distributed. How important retrieval augmentation is for LMS. A lot of people underestimate that.

• 41:15Harry Stebbings?

  Do AI founders need to be in the valley?

• 41:18Speaker 00

  It certainly helps a ton.

• 41:20Harry Stebbings?

  What single element would you most like to change about the AI community?

• 41:24Speaker 00

  To have more folks talk to each other about real risks and let a small set of folks continue to think about existential risks, but not scare people so much with with very interesting sci fi, general fiction scenarios that would make fun of action movies, but are not the real issues at hand right now.

• 41:45Harry Stebbings?

  Ten years' time, what role does AI play in society then?

• 41:48Speaker 00

  And even bigger one than now.

• 41:50Harry Stebbings?

  Were you against Elon Musk's petition to pause development?

• 41:54Speaker 00

  I did not sign it. I don't think it makes sense to pause the training of models, just like I wouldn't want Tesla to stop updating the AI in my Tesla car. Because I would assume it gets better and better. And I don't assume at some point the car will just take off and watch the sunset by itself because it got too intelligent.

• 42:11Harry Stebbings?

  Final one for you. Ten years' time, where's you.com then? We have this conversation in 2033. Where's you.com then?

• 42:19Speaker 00

  Well, I think and hope it will be the default in many hundreds of millions, if not billions, of people's computers so they can find better answers.

• 42:28Harry Stebbings?

  Richard, I've loved this. We've gone off topic many times. It's been a fantastic discussion. So thank you so much for doing this, my friend.

• 42:34Speaker 00

  Thank you, Harry. You've been one of the most engaging podcasters I've ever talked to. That was super fun.

• 42:42Harry Stebbings?

  I mean, I have to say, I think that I showed that I actually know a little bit about AI in that podcast. I can do more than just venture. I'm quite impressed with that knowledge. I hope you enjoyed the show there. You can find out more and see everything from that episode on YouTube by searching for 20VC. That's two zero VC. Huge thanks to Richard for being such a fantastic guest. But before we leave you today, you know all those mind-numbing tedious tasks that seemingly take up half your day? Well, Coda is here with their new AI-powered work assistant that helps you and your team not just finish tasks but make progress. So your product team can bring a feature to market faster by using Coda AI to tag customer feedback, draft PRDs, suggest target audiences, summarizing product discussions and more.

• 43:27Harry Stebbings?

  Your sales team can engage more customers by bringing in data from sources like Salesforce and then use Coda AI to suggest action items, meeting agendas or lead scores. And your marketing team can drive an impactful launch with Coda AI summarizing user insights, creating briefs generated from notes and writing mock press releases to visualize taglines. With Coda AI, you can reimagine your to-do list and how you collaborate. So you're not only finishing tasks but really making progress. If you want a work assistant that lets you get back to work, you can get started with Coda AI today for free. Head over to coda.io/20vc. That's coda.io and get started for free. And speaking of amazing tools we cannot live without, travel and expense are never

• 44:12Harry Stebbings?

  associated with cost savings. Now you can reduce costs up to thirty percent and actually reward your employees. How do you do this? Well, Navan rewards your employees with personal travel credit every time they save their company money when booking business travel under company policy. Does that sound too good to be true? Navan is so confident you'll move to their game changing all in one travel corporate card and a spend super app that they'll give you a two hundred and fifty dollar in personal travel credit just for taking a quick demo. Check them out now at navan.com forward slash two zero V C. And last but by no means least, we need to talk cash. As of the twenty ninth of June, you can get five point five percent yield on your cash with twenty six week treasury bills.

• 44:56Harry Stebbings?

  But buying treasury bills, it's not that easy and you have to navigate a website that looks like it was made before I was born. Enter public dot com. Their treasury accounts make it simple to earn a high yield on your cash and it takes twenty seconds. Here's how it works. Sign up at public dot com, easily purchase twenty six week treasury bills that automatically roll over at maturity for a compounding yield. Plus, there are no minimum hold periods. You can access your cash at any time with the flexibility of a bank account. Of course, to receive the full guaranteed yield, of course, to receive the full guaranteed yield, you do have to hold to maturity. But here's the thing, these are T bills, which means your investment has the complete backing of the US government, making it one of the safest places to park your cash.

• 45:41Harry Stebbings?

  Go to public dot com forward slash two zero V C to lock in a historic five point four percent yield on your cash. As always, I so appreciate all your support and stay tuned for an incredible set of episodes coming next week.