Pod Insights The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch 20VC: Yann LeCun on Why Artificial Intelligence Will Not Dominate Humanity, Why No Economists Believe All Jobs Will Be Replaced by AI, Why the Size of Models Matters Less and Less & Why Open Models Beat Closed Models About this episode Yann LeCun is VP & Chief AI Scientist at Meta and Silver Professor at NYU affiliated with the Courant Institute of Mathematical Sciences & the Center for Data Science. He was the founding Director of FAIR and of the NYU Center for Data Science. After a postdoc in Toronto he joined AT&T Bell Labs in 1988, and AT&T Labs in 1996 as Head of Image Processing Research. He joined NYU as a professor in 2003 and Meta/Facebook in 2013. He is the recipient of the 2018 ACM Turing Award for "conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing". Huge thanks to David Marcus for helping to make this happen. In Today's Episode with Yann LeCun: 1.) The Road to AI OG: How did Yann first hear about machine learning and make his foray into the world of AI? For 10 years plus, machine learning was in the shadows, how did Yan not get discouraged when the world did not appreciate the power of AI and ML? What does Yann know now that he wishes he had known when he started his career in machine learning? 2.) The Next Five Years of AI: Hope or Horror: Why does Yann believe it is nonsense that AI is dangerous? Why does Yann think it is crazy to assume that AI will even want to dominate humans? Why does Yann believe digital assistants will rule the world? If digital assistants do rule the world, what interface wins? Search? Chat? What happens to Google when digital assistants rule the world? 3.) Will Anyone Have Jobs in a World of AI: From speaking to many economists, why does Yann state "no economist thinks AI will replace jobs"? What jobs does Yann expect to be created in the next generation of the AI economy? What jobs does Yann believe are under more immediate threat/impact? Why does Yann expect the speed of transition to be much slower than people anticipate? Why does Yann believe Elon Musk is wrong to ask for the pausing of AI developments? 4.) Open or Closed: Who Wins: Why does Yann know that the open model will beat the closed model? Why is it superior for knowledge gathering and idea generation? What are some core historical precedents that have proved this to be true? What did Yann make of the leaked Google Memo last week? 5.) Startup vs Incumbent: Who Wins: Who does Yann believe will win the next 5 years of AI; startups or incumbents? How important are large models to winning in the next 12 months? In what ways does regulation and legal stop incumbents? How has he seen this at Meta? Has his role at Meta ever stopped him from being impartial? How does Yan deal with that?
Transcript 11,337 words · 3 speakers
Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Speaker 00 AI is going to bring a new renaissance for humanity, a new form of enlightenment, if you want, because AI is going to amplify everybody's intelligence. Every one of us will have a staff of people who are smarter than us and know most things about most things. So it's going to empower every one of us.
0:16 Harry Stebbings? Welcome back. This is 20 VC with me, Harry Stebbings. And today I'm joined by one of the greats, Jan Laakun. Jan is VP and Chief AI Scientist at Meta and Silver Professor at NYU. He was the founding director of FAIR and of the NYU Center for Data Science. He's the recipient of the 2018 ACM Turing Award for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing. And I want to say huge thank you to David Marcus and Mathieu at Photo Room for helping make this one happen. This really was a huge honor for me to do. But before we dive into the show today, are you building enough conviction to outpace other investors in this changing ecosystem? Tegus helps VCs get under the hood of their investments quicker, map evolving markets with deeper research, monitor
1:00 Harry Stebbings? the financial health of portfolio companies, and even source new deals. Tegus is trusted by top firms like Redpoint and Spark Capital for everything from expert calls and transcripts to quickly surfacing insights across filings, comps, and benchmarking. Find out why a majority of the top venture funds are using Tegus on a daily basis to differentiate, consolidate, and accelerate their research. Head to tegus.com/growth for your free trial. And speaking of tools we cannot live without there, SecureFrame is the leading all-in-one platform for automated security and privacy compliance. SecureFrame simplifies and streamlines the process of getting and staying compliant to the most rigorous global privacy and security standards like SOC 2 and
1:43 Harry Stebbings? others. And SecureFrame's industry-leading compliance automation platform, paired with their in-house compliance experts and former auditors, helps you get audit-ready in weeks, not months, so you can close more deals faster. Automate your security and privacy compliance with SecureFrame. Schedule a demo today at secureframe.com. That really is a must. And finally, are you tired of using the same old tools and platforms in an attempt to manage your global team? Well, us too. I use 16 different tools when you could just use Deal. Deal is the only global first HR tool with unmatched automation, customizations and integration, making it the only platform for built-in contracts, visas, background checks, payroll, equipment, benefits and beyond.
2:26 Harry Stebbings? Are you ready to see how Deal can scale with your global and local teams? Visit deal.com/20vc and book your demo today. That's deal.com/20vc to book your demo.
2:44 Speaker 02 You have now arrived at your destination.
2:47 Harry Stebbings? Jan, I am so excited for this. I heard so many great things from our mutual friends, obviously David Marcus and then Mathieu at Photo Room. So thank you so much for joining me today.
2:56 Speaker 00 It's a pleasure.
2:57 Harry Stebbings? Now, I heard some of the early stories, but I want to start with one from David Marcus. How did you first enter the world of AI and make that first foray?
3:05 Speaker 00 I was still an undergraduate engineering student in France, and I stumbled on a philosophy book, which was a debate between Jean Piaget and the cognitive psychologist, and Noam Chomsky, the famous linguist. And they were arguing about nature versus nurture for language, whether language is acquired or innate. So Chomsky was on the side of innate and Piaget on the side of acquired with some innate structure. And on the side of Piaget was a guy called Seymour Papert, who was a professor at MIT. In his argument, he talked about something called the perceptron, which was an early machine learning system. I read this and discovered that people had been working on machines that could learn, and I was fascinated and I started digging the literature. Soon discovered that much of that literature was in the 1950s and 60s
3:50 Speaker 00 and basically stopped in the late 60s because of a book that they killed it. And Seymour Papert was a co-author of that book. And here he was 10 years later, actually praising the perceptron as an amazing concept. So I was hooked. I started getting interested in what was not yet called machine learning, but eventually became neural nets and now deep learning.
4:09 Harry Stebbings? Can we ask you, David asked this as well. How long did it take you to get to the major breakthroughs that you're at the origin of? When you look back over that time to get to those major breakthroughs?
4:19 Speaker 00 So the first one was. When I was still on undergrad, basically finishing my engineering studies, figured out that the way forward to lift the limitations of the old systems that were abandoned in the 60s was to find learning algorithms that could train multi-layer neural nets. And people had all but abandoned this type of research, except for a handful of people in Japan. And one guy I heard about called Jeff Intend, who had published a paper in 1983, so this was just the year I graduated, on something called the Boltzmann machine, which was clearly a method to go beyond those limitations. And so I had developed a method for training multi-layered nets, which was very close to what we now call backpropagation, but not exactly the same. It was closer to what we call targetprop, actually, nowadays.
5:01 Speaker 00 And then published a few papers in French and eventually met Jeff at a meeting in France in 1985. And we realized we'd been working on the same thing and were thinking alike. But I was in the middle of my PhD and he was an associate professor at Carnegie Mellon. So we started a discussion and then, you know, visited him at Carnegie Mellon for a summer school we organized. And then when I finished my PhD, I did a postdoc with him and then joined Bell Labs. And while I was in Toronto, I developed what was called convolutional nets now, convolutional networks, which is a major method for image and speech processing. And so that's what I'm best known for, but it started much earlier.
5:36 Harry Stebbings? I have to ask you, Yoshua described the hype cycles within AI and neural nets like deserts when you're not in them. And he asked the question, how did Yan not get discouraged when for a solid decade, we were in a desert where no one really cared about neural nets? How do you keep the enthusiasm, bluntly, when, as Yoshua said, no one really cared?
5:58 Speaker 00 Both Yoshua and Jeff and I had in the back of our minds that those methods would eventually come to the fore and that we would have to snap people out of their preconceived ideas about neural nets. So Yoshua and I were actually working together at AT&T, Bell Labs, in the early 90s. And then the interest of the community for those methods started waning around 1995 or so. And there was indeed about 10 years when not only nobody was interested in neural nets, but people were even making fun of it and talking about it in disparaging terms. Now, there is something, though. In 1996... I kind of changed job. I stayed in the same company. I stayed working at the TNT in the research labs, but I became a department head and this was the early days of the internet. And my group and I started working on something completely different that had not much to do,
6:42 Speaker 00 at least with machine learning, this image compression. I had this idea that with the internet coming up, we should have a way of scanning existing paper documents and then put them on the internet so that everybody could have access to them. And so I worked on this for five or six years and that project ended when all of us basically left TNT. That's when I restarted working on deep learning and Jeff also came back to Canada. He'd been in the UK for a while. Yashra, Jeff and I decided in the early 2000s to basically start a conspiracy to revive the interests of the community in neural nets by making them work, discovering new algorithms. And it took almost 10 years, but it succeeded beyond the wildest dreams, basically.
7:21 Harry Stebbings? So I'm going to ask you a range of varying questions in terms of depth, breadth and obvious and non-obvious. So forgive me if some are obvious. I just want to ask, when I hear the historical context there from you, over many decades, how do you feel today when we look at what's happening today? Are we at a new inflection point in development? Or is this merely the continuation of what we've seen for many decades?
7:47 Speaker 00 It's a combination of the two. On the one hand, a lot of what we see today, when you are down in the trenches of research, looks a logical extension. I was not as enthralled by the sort of recent progress as the public was because I've seen this progress happening over the last several years. Now, there are things that have been very surprising. The fact that self-supervised learning methods applied to transformer architectures work amazingly well and they work way beyond what we could have expected. The fact that we can do, basically train systems to understand language, translate language in multiple languages and then continue text if you train them to do this or answer questions if you train them to do this, works amazingly well to an extent that people didn't quite expect was going to happen by just making
8:30 Speaker 00 them bigger and training them on more data. So that certainly has been surprising for everybody, but that revolution occurred two years ago, right? So whereas the wider public has learned about it through That was made available for us, you know, it's been more continuous. And you see this in a lot of marking events in technological progress or in AI in particular are marked by kind of splashy events that the public pays attention to. But too many of us, like it looks like more like a continuous thing. And generally, what those progress require is a bunch of people to take the techniques that already exist, push them a little further, do a bit of engineering and then make a demo that demonstrates that it works. So that was the case for, you know, Deep Blue, the chess player that IBM built in the mid 90s that beat Garry Kasparov.
9:14 Speaker 00 You know, same thing with the DOPA Grand Challenge that Sebastian Thun team at Stanford won, you know, a car that could drive itself in the desert for a hundred miles. And then AlphaGo and the IBM Jeopardy. And there's a number of those things, right? And GPT just being the latest one. It looks like kind of jumps when you look at it from far away. But when you're in the field, it's more like a continuous evolution.
9:36 Harry Stebbings? Can I ask, has there been any other surprising on the positive side? Developmental things you've seen over the last year or so? You said about self-supervised learning and the efficiencies there. Is there anything else where you're like, I didn't expect it to go as well as it has done in the last year?
9:50 Speaker 00 Yeah, so I already mentioned it, the fact that merely training a language model to predict the last word in a sequence of words. If you do it properly, you get a system that has capabilities that are somewhat unexpected. And they emerge as you make those systems bigger and you train them on larger amounts of data. That's clearly been a surprise for everyone. Now, the thing is, as researchers and scientists, we're always looking for the next thing. So what I'm interested in at the moment is what goes beyond that. A lot of people are going to work on applications of auto-aggressive large language models, which is great. There's going to be a lot of products and new ways for people to do things and it's going to be wonderful. But I've already been thinking about the next stage for the last four, five years even, actually more, which is like what's missing from those systems.
10:32 Harry Stebbings? What are your thoughts on what's missing from those systems? In that logical next step, where does that lead you in your thinking?
10:37 Speaker 00 So those systems do not have anywhere close to human level intelligence. Okay, despite what you might think, we are kind of fooled into thinking it because those systems are very fluent with language, but their ability to to think, to understand how the world works, to plan are very limited and they're understanding the world is very superficial. And the reason for it is that they are strictly trained on language and language only contains a small proportion of all all human knowledge. Most of human knowledge is not linguistic at all and all of animal knowledge is non-linguistic. And we take it for granted. You know, this is the Moravec paradox, right? All the capabilities and abilities that we take for granted, planning a motion or something or very simple things that everyone can do.
11:20 Speaker 00 A 10-year-old can clear up the dinner table and fill up the dishwasher. Any 17-year-old can learn to drive. We still don't have so many cars. We don't have domestic robots.
11:28 Harry Stebbings? If they're non-linguistic, like the majority, I'm sorry for the base questions, but then what are they? And is that that we don't have able to be ingested by AI models and engines over time?
11:40 Speaker 00 So first of all, there is no question that eventually AI systems will understand the world in similar ways that that humans do. There has better ways, but there will not be auto-regressive large language models of the type that we're now talking about. They will be different for a number of different reasons. But to answer your question more directly, anything that has to do with sort of an intuition of the real world requires an experience of the real world or a simulated version of it, which those large language models don't have. They're purely trained from text. There's a number of questions that about the physical world that they'll be able to answer because there's a template for it in the or something very similar in the data that they've been trained on. But same for planning, you can ask them to plan a trip or something and they will adapt a template that they've they've been trained on.
12:23 Speaker 00 But they don't really have a mental model of how the world works that allows them to plan complex action sequences or use tools or things like that.
12:29 Harry Stebbings? Is that why you said that AI researchers face palm when they hear prophecies of doom?
12:35 Speaker 00 No, that's a different question. Those are kind of orthogonal concepts. So I mean, there is some some weak connection. There is a flaw in current autoregressive L L M's, which is that you can only control their answer in two ways. The first way is you modify the statistics of the training data that you train them on, possibly using a human feedback for specific answers. And the second one is you change the prompt. And the combination of the prompt that, you know, the question you you ask them, the form in which you ask the question and the statistics of the training data entirely determines the answer that the system will produce. So there is no persistent memory, first of all. But second of all, you cannot control the system. You cannot impose constraints on it, like be factual, be understandable by a thirteen year old.
13:18 Speaker 00 You can try to put this in a prompt, but then, you know, you rely on whether the statistics of the training data is appropriate for taking taking that into account. But there's no direct way to constrain the answer of those systems to satisfy certain objectives. And that makes them very difficult to control and steer. And so that creates some fears because people are kind of extrapolating. If we let those systems do whatever, we connect them to internet and they can do whatever they want. They're going to do crazy things and stupid things and perhaps dangerous things. And we're not going to be able to control them and they're going to escape our control and they're going to become intelligent just because they're bigger. And that's nonsense. First of all, because this is not the type of system that we are going to give agency to. The systems that will eventually be given agency that are going to be able to plan sequences of actions are systems that are going to have objectives
14:02 Speaker 00 that they're going to have to satisfy. And because of those objectives, they're going to be controllable. So they're going to be much more controllable than the current systems. Okay. So my prediction is that within a few years, nobody in their right mind would use autoregressive L L M's. They'll go away in favor of something more sophisticated and controllable. They can plan its answer as opposed to just produce one order to the other reactively. That's the first fallacy. The second fallacy is that there is this idea somehow that the desire to and the ability to dominate is linked with intelligence. So this is a statement that a lot of people are are making, including, you know, my friend Jeff Hinton recently, that somehow as soon as a machine becomes intelligent, it becomes uncontrollable because it's being smarter than us.
14:45 Speaker 00 It can influence us in ways that we can't even imagine. Now, I think this is a gigantic fallacy because even within the human species, it is not the smartest among us that want to dominate the others. Okay, to dominate other entities, you don't necessarily need to be smarter than them. We need to want to dominate them. This is not something that every intelligent entity is going to do spontaneously. We do it as humans because the desire to influence others was built into us by evolution because we are a social species. Okay, same as baboons and chimpanzees and wolves and dogs and etc. It's not the case for orangutans. Orangutans don't have the desire to dominate anybody because they are non-social animals. They are solitary animals. They are territorial, in fact.
15:29 Speaker 00 We need to separate those two concepts, the will, the desire and the ability to dominate on one hand and intelligence on the other hand. The fact that we're gonna have super intelligent machines at our disposal means that every one of us is gonna be like a business leader, politician or an academic with a staff of people working for them that are more intelligent than themselves. It's great. If you feel threatened by being the boss of other people who work with you but are smarter than you, you're not being a good leader.
15:55 Harry Stebbings? Can I ask then, how do we instill values within models where they don't have a desire to dominate?
16:01 Speaker 00 Okay, so let me describe the sort of architecture of future AI systems as I see it. We're gonna have AI systems that basically are going to plan their actions. And actions can include sequences of words that you tell someone. But they're gonna plan those sequences of actions or words so as to optimize a series of objectives that we set them. Okay, so what objective is, does this answer the question I just asked? Okay, another objective might be, while you're talking to a 13-year-old, make that answer understandable by a 13-year-old. Another objective might be, I ask you to answer a question about the world, so be factual. Or it's a question about yesterday's political event. Can you kind of be compatible with everything you've read in the press this morning? Things like that, right? You'll have those systems that have a series of objectives
16:43 Speaker 00 and their output, their answer by construction is going to have to satisfy those objectives. And some of those objectives would be hardwired to make those systems safe. Like if it's a domestic robot that can cook dinner. And can wield, you know, kitchen knife in its arm. There's going to be a term in there that says, stop moving your arm when there is people around because, you know, you might hurt them. So that's going to be an objective that the system cannot violate because by construction, we're going to have to satisfy them. So that's the way to build safe AI system. You make them produce answers that by construction have to satisfy objectives and you design those objectives so that their actions are safe. Now, how precisely to do this is not a completely solved question, but you try it, you deploy it at a small scale, you see what the effect is and you correct it when it doesn't work.
17:25 Speaker 00 And you fix it progressively, not like if you get it wrong, it's going to destroy humanity.
17:30 Harry Stebbings? How do you determine who's able to set the objectives? Because there could be right or wrong depending on who sets them.
17:37 Speaker 00 That's true. So there's going to have to be a process by which we allow people to do this, like some vetting process. The same way that there's a vetting process for people to take care of your health or cut your hair or fix your plumbing for your car, right? So there's some vetting process, certainly some testing and market deployment procedure with regulating agencies for things that are potentially dangerous. Probably not for all applications, but for many applications, certainly in healthcare, transportation and things like that. And then let's take the example of intelligent assistants. Let's imagine a future where everyone can talk to their intelligent assistant. That system will have pretty close to human level intelligence, probably more accumulated knowledge than most humans. You know, they could translate in any language and give you a quick summary of yesterday's newspaper and things like that, right?
18:22 Speaker 00 Explain mathematical concepts to you, things like that. So people are probably going to use this almost exclusively in the future for their interaction with the digital world. You're not going to go to Google or Wikipedia, you're just going to talk to your assistant. And the only way to do this properly is for the base infrastructure for those assistants. They would be so pervasive, so much will ride on those systems that I don't think anyone will accept that those assistants be behind the event horizon in a private company. They will insist that the infrastructure is open. They will insist also that the vetting process by where those systems are trained be something maybe like Wikipedia. We tend to trust Wikipedia, sometimes with a gun assault, but we tend to trust Wikipedia because there is a vetting process so that whenever an
19:05 Speaker 00 is modify some editor kind of check on it and then the changes are accepted on that things like that. So you can imagine that the sort of common repository of all human knowledge that will be our assistance will be constructed through some sort of cross sourcing process, perhaps similar to Wikipedia, where you're going to have a bunch of people training those systems and fine tuning them so that whatever the answer they produced are correct.
19:25 Harry Stebbings? It's so funny you say about that, kind of the benefits there of the open approach over the closed approach, because that's where I've been stuck, which is like, where does value accrue? Is it to the closed model or the open model? And then we had the leaked internal memo today from the Google employee who said, we're not ahead, OpenAI are not ahead, there's this third being, which is actually far more significant and we haven't taken notice of, I'm summarized.
19:48 Speaker 00 That was triggered by Llama, which is the model that was put together by my colleagues at fair, which was the code was open sourced, the model sadly was distributed only for research and non-commercial purpose. And the reason for that is basically complicated legal issues of what's the status of the data that the system had been trained on and things like that. It's not a lack of desire from meta to open source, it's more kind of complex legal issues that go beyond my head.
20:15 Harry Stebbings? Why does Open win against a more controlled, tight knit, well-funded OpenAI or other large corporate with a big balance sheet and a very rigorous but streamlined team?
20:27 Speaker 00 It's very simple. It's because no outfit as powerful as they may be has a monopoly on good ideas. If you do it in the open, you recruit the entire world's intelligence to contribute to things and having ideas and ideas that you may not have thought about, which an outfit with 400 people has no chance of thinking about, or even a large company with 50,000 employees may not want to devote any resources to because they may not think it's useful in the long term, or they have more urgent thing to take care of. So you give it away. And then you have tons and tons of people, some of whom are undergraduate students or people, you know, in their parents' basement. So coming up with amazing ideas that you would never have thought about or willing to spend the time to crunch down the seven billion way Llama so that it runs on a Mac on a laptop. I think that's why
21:13 Speaker 00 open source projects succeed. When they concern basic infrastructure. So if you think about it, the early days of the internet, there was a battle between Microsoft and Sun Microsystems to provide the basic infrastructure for the internet, operating system, the web server, things like that, right? So on Sun Microsystems, there was Solaris and whatever web server and Java. On the Microsoft side, that was Windows with IIT or whatever, you know, and ASP, which was their kind of server and client side protocol. Both of them lost. In fact, Sun Microsystems pretty much went bankrupt and was sold for parts to Oracle. One was Linux and Apache, which is completely open source. And you might ask why? The entire internet and the entire tech industry runs on Linux, right? And your phone probably runs on Linux too, if you have Android.
21:56 Speaker 00 So that's three quarter of the phones, you know, in the world. So the reason for this is that it's just a much better way of gathering competence and talent around a common project, even if it's not motivated necessarily by profit.
22:09 Harry Stebbings? Yeah, I agree and I love this. You work with Meta. My question and David Marcus's question was, how does Meta win then?
22:17 Speaker 00 It's been the case that Meta in the past has open source pretty much everybody, everything that it's ever produced in terms of basic infrastructure, right? So you have, you know, React for the framework for web and mobile apps. You have PyTorch. PyTorch is not even owned by Meta anymore. The ownership was transferred to the Linux Foundation because it's so essential to the AI R and D infrastructure nowadays. ChatGPT was developed on PyTorch. Okay, OpenAI runs on PyTorch. The entire world, in fact, runs on PyTorch, except Google, because they have their own thing, right? But it goes beyond that, right? Meta, open source is its hardware, server, backplane design, so that hardware manufacturers can build to its specifications. And pretty much everything, aside from legal issues
23:00 Speaker 00 that are sometimes due to recent laws or court decisions, pretty much everything has been open source. It is not because other people can use your technology that you can't exploit it to the same extent, right? Who can use smart NLP systems for translation or content moderation on Facebook other than Facebook? It doesn't matter if other people have access to the same technology.
23:21 Harry Stebbings? I totally agree with you. And this kind of led to my next question, which you actually tweeted. About which comes to the size of data modes and size of data availability. Is it simply a case that the largest model wins? And how do you think about value in small models as well?
23:36 Speaker 00 Yeah, so it's not the case. And this is really what Lama has demonstrated and really kind of shown people. They've demonstrated that you don't need those models to be very large to to work really well. And I think it caused a bit of an epiphany for a lot of people realizing, oh, okay, maybe you need a thousand GPUs running for a couple of weeks to train it, the base system. In fact, that number is going down too, because people are figuring out how to do this more efficiently. But once it's pre-trained, you can use it for all kinds of stuff and you can find your net really easily. And then at the end, you can run it on your laptop, right? That's going to be amazing. Or maybe on a desktop machine with a GPU in it or a couple of GPUs. So I think it opened the minds of people to the fact that there is like enormous opportunities that really weren't thought to be possible before.
24:18 Speaker 00 And I think it's going to make even more progress because if we go towards the design of AI systems, perhaps along the lines of what I described with objectives and planning, I think those systems could actually be even smaller to some extent.
24:30 Harry Stebbings? How would they be even smaller?
24:31 Speaker 00 Because for them to work, you have to train them on gigantic amounts of data, way more data than any humans has ever been trained on. So the amount of data Lama is trained on, for example, is something like 1.4 trillion tokens, which is a quarter of the internet or something. So it's something absolutely enormous. It would take someone reading eight hours a day at normal speed about 22,000 years to read through that. Okay, so obviously those systems can accumulate a lot of knowledge from text, but they don't do it the same way humans do it because we don't need that much time to be that smart and to learn that much. So obviously, we are much more efficient, our brains are much more efficient than those models at learning things. Like, how is it that a teenager can learn to drive a car in about 20 hours of practice? We still don't have level five self-driving cars. So obviously, we're missing something really big.
25:14 Speaker 00 And what we're missing, I think, is ability for AI systems to learn how the world works by observation mostly. And then this ability to plan so as to satisfy objectives. And then beyond that, the ability to set sub-objectives in the satisfaction of a bigger one. Okay, that's hierarchical planning. And we do this, humans do this, some animals do this to some extent. Every animal, mammal and bird is capable of some level of planning. Autovertical LLMs, basically, don't do planning, or a very simple form of it.
25:42 Harry Stebbings? Yeah, and you mentioned the efficiency that can come from actually smaller models than expected, and how actually size of models isn't everything. The other thing, we spoke about open and closed. The other thing that I've been thinking, and everyone's been thinking about, and I've interviewed many leading AI experts, and they say the value will accrue to the incumbents. Startups, they don't have the data, they don't have the models, it'll accrue to the incumbents. Is that right? Will the value accrue to the incumbents? Or do you believe that, given what you just said about size not being everything in terms of models, it could be startups as well?
26:13 Speaker 00 So it depends on which scenario you believe in. So the scenario I think will happen, and I'm certainly waiting for, is the scenario I described earlier, where you have some sort of open platform for base LLMs. So base LLMs basically would be seen as a basic infrastructure, TCP IP, Linux, Apache, essentially, completely open. And then there would be an ecosystem of companies building stuff on top of it, which for vertical applications for specific things, right? To specialize those systems for particular application, to offer support, to make it customized for enterprise applications, for personal things. There'll be like a whole economy around this, which will create jobs, by the way, not make them disappear. So this is the scenario that I believe will happen. And the reason I think it will happen is because there is essentially a need to use essentially millions of contributions
26:57 Speaker 00 for making those systems factual and correct, et cetera, so Wikipedia style. So I think the proprietary approaches will actually fall behind. So that's one point, okay? The second point is, you can ask yourself the question, how is it that the companies that were best positioned to produce something that ChatGPT, namely Google and Meta, didn't? Why is it OpenAI, a small ad sheet with 400 people? And the answer is, it's not because Google or Meta did not have the competence or the technology. It's just that they didn't have the pressure to produce completely new products that had a lot of risk attached to them. And we know what the risks are, because a few weeks before ChatGPT, my colleagues at FAIR produced a large language model called Galactica, which was an experimental system.
27:39 Speaker 00 So Galactica was a large language model trained to, trained on the entirety of. The scientific literature and it was basically designed to help scientists write papers. So you would start writing a paragraph or something like that to describe the topic of a paragraph. And then Galactica would basically complete the paragraph and it wouldn't be factually correct. You would have to fix it, but it was like you would ask it to build a table result and it would just put the LaTeX commands to build the thing and populate it with the known results on the literature about the topic that you're working on. Or you would type a chemical formula for something and it would turn it into an actual name for it. Very useful for scientists. As soon as the demo was put out, it was murdered by the social network Twitter sphere.
28:21 Speaker 00 People said, oh, this is going to destroy scientific publication because now any random person can write a positively sounding scientific paper that is nonsense. And there was so much material thrown at the system that the people at Meta who built it couldn't take it. They took down the demo because they said, we can't sleep at night. So here is an example of a very useful system, a system that could have been extremely useful, particularly for writers of scientific papers who are non-native English speakers, that basically was destroyed by AI doomers. People who just did not think about the risk benefit analysis. The risk of flooding the literature with nonsense is ridiculous because scientific publications are vetted and things like that. There was not a significant danger.
29:03 Speaker 00 And then ChatGPT came two weeks later and was welcomed as the second coming of the Messiah. What does that tell you? And then a few months later, Google came out with Bard and in the demo Bard made a tiny minor factual mistake about some astronomical fact and Google's stock went down by eight percent. Now, what it tells you is that when something is produced by a large company that has a reputation, particularly a reputation to defend, they can put out things that show nonsense, but it's okay for a small company. So that's the landscape of what happens now, which is why I think there's a bit of a paradox, which is that the companies that have the best technology basically can have difficulties putting it out because of those legal issues and public image.
29:44 Harry Stebbings? Do you not also think there's this core business model challenge there, which is it's the classic innovator's dilemma? Like, why didn't Google do this? Because it would have killed that absolute cash cow of Google Ads, the cost to service a query. Versus the costs of this is so significantly different. You'd be killing your core cash cow with this with unknown upside versus retaining what is a great business.
30:06 Speaker 00 You don't have a choice. There's no question that it could take a while, but there is no question that people will interact mostly with the digital world using AI assistants. And they may run into your augmented reality glasses, okay, or something of that type, like in the Spike Jonze movie, Her. That's not a bad depiction of what the way things could develop. And so if you if you take the assumption, you make the assumption this is going to happen, you have to build it as quickly as you can. And it might cannibalize your newsfeed algorithm or whatever, or the case of Google, your search engine, but you have to do it. As Meta has been known to make those choices in the past, like the move to mobile, for example, and the move to short form video, for example, which obviously TikTok has been very successful at.
30:45 Speaker 00 Meta has entered the business in kind of a big way, despite the fact that the amount of revenue derived from it is lower than traditional newsfeed. Because it's hard to put ads in videos, basically.
30:54 Harry Stebbings? You mentioned the job creation element there. I do just want to touch on the job side because it's the classic AI doomer that's we're all going to be unemployed. And we're going to have universal basic income and an optimistic world. You said about job creation there. We don't hear about job creation through AI. How do you see what jobs will be created through this new ecosystem and what that world of employment could look like?
31:16 Speaker 00 So 100 years ago, or maybe 120 years ago, most people in most of the world worked in the fields in food production, pretty much the majority of the population. Today, in developed countries, it's between 1 and 2%. And that has caused the migration of people into the cities and the development of service business. Same thing 20 years ago or 20, 30 years ago, there was a big movement towards automation of manufacturing. And a lot of manufacturing jobs disappeared in developed countries, but they were replaced by other things. So 20 years ago, who would have thought that you could make a living with a podcast?
31:50 Harry Stebbings? I just didn't think I could five years ago, Jan. I'm as surprised as everyone else.
31:55 Speaker 00 Right. A lot of jobs appeared thirty years ago. There was no such thing as web designer. And now we have engineers in the world basically do this, right? The number of economists that I have talked to, which is pretty large, where I asked that question, we tell me we're gonna run out of jobs because we're all gonna be replaced by, I think, is exactly zero. No economist believes this. No economist believes we're gonna run out of jobs because no economist believes that we're gonna run out of problems to solve or requirement for human creativity and human communication and stuff like that. This is gonna create as many jobs as it makes disappear. And those jobs, by the way, are gonna be more productive. So overall, technology makes people more productive. In other words, for the same amount of hours worked, you produce more wealth, okay? But every technological revolution, unless it's accompanied by political changes and social changes,
32:40 Speaker 00 generally profit a small number of people, at least temporarily, right? That happened in the industrial revolution in the late nineteenth century, where a few people became extremely rich and a lot of people were exploited. And then society changed and there were like social programs and income tax and high tax for richer people and stuff like that, which the US has backpedaled on this, but not Europe. So there is a question of how you distribute the wealth, if you want, okay? How do you organize society so everyone profits from it? But that's a political question. It's not a technology question. It's not new. It's not caused by AI. It's just caused by technological evolution, right? It's not a recent phenomenon.
33:14 Harry Stebbings? This is so unfair of me to ask, but what do those jobs look like? Are they creative oriented? But what does that actually mean? Sorry, I know it's a really hard question, but I'm just trying to understand how we actually spend our time and my children, which I don't have, by the way, Ian. What do they do? Like sculpt or paint? I don't know.
33:30 Speaker 00 I don't know. That's a good question, but it's not because I don't know that it won't happen. You look at like how many people exercise their creative juices today, right? With all the tools that are available that weren't available ten, twenty or thirty years ago. Like 3D artists or something like this, you know, game designers, all kinds of things. So there are two types of jobs that have a bright future. Creative jobs, whether they are scientific, technical, educational or artistic. It has to do with communication, right? And communication of human emotions, which is intrinsically human. So that's one category. And then the other one is personal services, so where you need actual people. People to interact with you.
34:05 Harry Stebbings? The only thing that I worry about is the speed of transition. Like when you look at past industrial revolution, when you look at even the introduction of PCs into kind of working environments. These were multi-decade introductions. Bluntly, what AI feels like in some industries today, we use it at the media company and it's cutting our employment. Like the speed of transition is much more compressed in this timeline, which will lead to short-term significant high unemployment. Do you concede that or do you not concede?
34:34 Speaker 00 So this is something I used to be very worried about, that the speed of progress of technology was going to leave a certain number of people behind who cannot be basically retrained fast enough or maybe they are too old to retrain themselves for the new world. I was worried about this. And then I talked to a bunch of economists and they say, oh, not really, because the speed at which a technology disseminate in the economy is actually limited by how fast people can learn to use it. A good person to talk to about this is Eric Bernyovson at Stanford. And what he says is that when a new technology is introduced, let's say the PC, right, with graphical user interface, the mouse, et cetera, right? In the mid 90s, how long did it take to have a measurable effect on productivity? Which is the amount of wealth produced by per hour worked.
35:19 Speaker 00 Typically, it's 15, 20 years. And the reason is that's what it takes for people to learn to use that new technology, basically.
35:24 Harry Stebbings? But do you buy that here? Like, people are pretty good at prompts. Social media content managers are using prompts very efficiently to produce content plans, to create content ideas in under half an hour after watching a couple of TikToks.
35:36 Speaker 00 Yeah, but what is going to be the effect of this on, first of all, on measurable productivity? Second of all, on the job market, is it going to make people lose their job like right away? And no, it's going to take a while. It's going to take 10, 15 years, possibly more. It depends when you start counting, right? Because the AI revolution maybe started 10 years ago. So if you start counting then, then it might only take another 10 years. But I don't think you want to underestimate the degree of conservativeness of the business world, right? Things tend to change not that quickly. But if it's that easy to learn, like people will learn it and then invent new professions out of it or be or become more productive themselves.
36:11 Harry Stebbings? Why do you think we love the doom, Jan? I love your approach. And mindset and I agree with it, but why do you think we are magnetized? Oh, we're all going to be unemployed in the doom.
36:20 Speaker 00 Because I think we're hardwired to pay attention to things that occur or may occur that could be dangerous to us because it means that there's something about the world that we don't completely understand and we have to pay attention to it and be careful about it. For example, take a young child, five months old, and show a scenario to this more child of a little car that is sitting on a platform. And then you push the car off the platform and instead of falling, the car appears to float in the air. The five months old will barely pay attention to it. But if you show this to a 10 months old, the 10 months old will look at it with huge eyes and stare at it for a long time, wondering what's going on. Because in the meantime, babies around the age of between six and nine months learn about gravity. They learn that objects that are not supported are supposed to fall. And so their mental model is that an object is not supported, should fall.
37:04 Speaker 00 And they see this object that appears to float in the air and they say like, this can't be. Like there's something, I don't understand about the world. I need to look at this and investigate. Okay. So we're hardwired for this because that's the way we learn our internal mental model of the world. That allows us to predict what's going to happen, allows us to plan. That's what makes us smart. That's the basis of intelligence, the ability to predict. And so we naturally pay attention to stuff that is surprising or dangerous or both. Which is why you see a outrageous piece of clickbait at the bottom of some website. And like you have to convince yourself not to click on it.
37:36 Harry Stebbings? Can I ask you a couple of direct questions? I'm just too interested and we can take them out if needed. What did you say to Jeff when you heard that he was obviously making moves that he did? I'm sure you had a conversation with him. What did you say to him?
37:48 Speaker 00 We haven't spoken yet, actually. We're going to speak to get each other's opinion on it. I don't think he knows my opinion on this because I don't think he follows what I post on Twitter or whatever, even though he is on Twitter itself. So I think we have a discussion to have. I've had this discussion before with Yotra Benjio, but not with Jeff. And to me, the fact that he left Google is not particularly a surprise. The fact that he leaves Google to be able to speak his mind, I think is not surprising. So I have a very different deal at Meta, which is that I say whatever I want. Okay, I'm not under the tight control of the communications department or anything. I just say what I think.
38:23 Harry Stebbings? How did you get that deal, Jan? Seriously, many of my friends at Meta in very high positions, as we're mutual friends, they don't have that deal.
38:34 Speaker 00 There is a particularly sweet spot because I have quite a bit of following people who trust me or believe me or want to hear what I have to say, even if they don't trust me at all. And at the same time, I'm not an officer, so it's not, there are things I can't say because of legal issues, you know, financial, blah, blah, blah, right? I'm a vice president, but I'm just below the level where you had to be really careful and control your message. And I think there is a cost benefit trade off here of AI is such a complicated, fast evolving issue that you basically, you need someone to be able to speak freely. And I think Jeff didn't feel like he had that option at Google, maybe for various reasons. So I understand why he might have wanted to leave, but I don't agree with him at all with the whole probability of human extinction or whatever.
39:19 Harry Stebbings? Have you ever felt your role at Meta has impeded your ability to be impartial?
39:24 Speaker 00 I don't believe so, no. There are certain things that I would post on social media that are popping up the work of my colleagues. I'm obviously biased about this because I know about the work and they are friends and or colleagues. And I think it's interesting probably because I followed the projects. So yeah, for this kind of stuff, I might be biased, take this with a grain of salt. You don't have to believe me, things like that. But it's given me a vision also of how things are built, what the problems are. For example, there is a narrative, a very common narrative that AI is the culprit for a lot of the bad side effects of social networks in the past. And in fact, it's completely backwards. AI is the solution to those problems. Let me tell you, backpedal 12 years ago or something, even before I joined Meta, where Meta started experimenting with the newsfeed.
40:09 Speaker 00 And the newsfeed was an algorithm that would pick which piece of news to show to everyone. And originally it was decided by how friends are you with the person making the post and things like that, right? How many interactions you have with that person. Eventually, a bit of machine learning was put into it shortly before I joined Meta. It was very simple. It was something like logistic regression, something like. The simplest method you can imagine with a lot of engineering behind it and a lot of hacks by hand and and special cases. But basically, it was something like logistic regression. Some big vector that describes for what you click on, like how much time you spent on a particular piece of content and blah, blah, blah. And then it would it would decide to give a rating to everything. That was deployed and people ended up spending more time on on Facebook. But then also it created problems that were quickly identified, like information bubbles in the context of political discourse.
40:54 Speaker 00 And the fact that what I was talking about earlier, that people tend to click on things that is more outrageous, right? So it caused the appearance of clickbait companies that basically were just like farms of teenagers in Montenegro or someplace making false news to get people to click on them and get money from the ads that they show them. Okay, so then this was realized there were like big groups at Facebook at the time studying the what the effect of those things are. And this was corrected. So that's the way you may go to some work, right? You try it on a small scale, you see what the effect is. If there is bad side effect, you correct it and then you compare to two systems. And then sometimes something unexpected occurs and you have to backpedal and completely change the way you do things. That's what happened in 2017 after the presidential election, American presidential election in 2016.
41:36 Speaker 00 The main UC algorithm was completely changed so that there was no clickbait anymore. There was no like news outlets that could like push their content that was propaganda, basically. You know, much more effort to take down false accounts and attempts to corrupt the democratic system and stuff like that. You correct it. And then what the progress of AI over the last few years basically allowed systems to be deployed to do things like take down hate speech relatively reliably in hundreds of different languages, which was basically impossible to do before.
42:04 Harry Stebbings? You mentioned correct it. Elon Musk said on with Tucker Carlson, the trouble with AI is you can't release and then correct. Unlike all prior technological developments, once released, it is too powerful to be able to bring back into the box. It cannot be amended in that way. Is that not true?
42:20 Speaker 00 That's not true. That's completely false. It makes an assumption which Elon and that some other people may have become confused of by reading Nick Bostrom's book, Superintelligence, or reading some of Eliezer Yudkowsky's writing. So this is predicated on an assumption that is just Falls, which is the existence of hard takeoff. All right. So the fact that the minute you turn on a super intelligent AI system is going to take over the world. And it's going to escape your control and it's going to refine itself to be even more intelligent. And the world will be destroyed. And that's just ridiculous. It's just completely ridiculous because there is no process in the real world that is exponential for very long. Those systems will have to like recruit all the resources in the world. They would have to be given limitless power agency.
43:05 Speaker 00 Like, why would we do this? And what's more, they would have to be built so that they have a desire to take over. Systems are not going to take over just because they are intelligent. Even within the human species, it is not the most intelligent among us that want to dominate others.
43:18 Harry Stebbings? So his desire and many other leaders desire to prevent any further development and to regulate intensity right now and stop all progression is BS, basically.
43:28 Speaker 00 It's obscurantism. It's like people who wanted to stop the printing press and the diffusion of printed books. Because if people could read the Bible for themselves, they wouldn't have to talk to priests anymore and then would have their own idea about religion. And that's exactly what happened. People read the Bible for themselves and that created the Protestant movement in Europe. And that created 200 years of religious conflicts. But it also brought to us the Enlightenment, science, rationalism, philosophy, ideas of democracy, and then the French and American revolutions. And then you can compare this with the Ottoman Empire, which for reasons of being able to control their population, forbid the use of the printing press. It started 300 years of decline. They were dominating science in the Middle Ages, which is why every star in the sky has an Arabic name.
44:10 Harry Stebbings? I love this. I'm going to do a quick fire round with you now. So I say a short statement and you give me your immediate thoughts and then we'll rock and roll. Does that sound okay?
44:18 Speaker 00 That's good.
44:18 Harry Stebbings? Which regions most need to change their modus operandi when it comes to the practice of scientific research and incentive mechanisms?
44:27 Speaker 00 Pretty much every region, I'm afraid, but for different reasons. You start with China. So China has a bit of an epidemic of bad science. There are a lot of very smart people in China, a lot of very good researchers, a lot of very good work coming out of China, particularly in AI, particularly in computer vision. But a lot of absolutely terrible work that has to be retracted a few months later is being published. Partly because of the incentive mechanisms in the academic system in China. So there's a problem to fix there. I can move to Europe. So in Europe, there are good things. The education system for like undergraduate education in Europe is great because it's partly free. So that allows talented people to go to good schools, even if they're not rich, right? Which is not the case in the US, for example. So that's that's good for Europe.
45:10 Speaker 00 European engineers and scientists are great, a top, best in the world. But then what are the opportunities for people who want to go into science and research? And there, most European countries actually don't have systems that really encourage this and motivate the most talented people and students to go into science. And so some of them go to North America, like me, thirty four, thirty five years ago. There are opportunities now that are really good in research labs like FAIR in Paris or Google also has labs in Paris. Actually, my brother works at Google in Paris and various other outfits. So that gives opportunities for people who really want to be productive and don't think that they can in the public research and academic system in France and the rest of Europe.
45:51 Speaker 00 The only European countries that rival the US in terms of quality of job for an academic scientist is Switzerland.
45:57 Harry Stebbings? What do you think they do to rival that? What is it about their incentive mechanism structure that gives them that ability?
46:03 Speaker 00 Two things. They pay people better. Second thing is they give them resources for research. They can get extra resources through grants and stuff like that, but they're good. And then they also attract some of the best students in the world. So you get the ideal combination that you only get in the top thirty universities in North America.
46:19 Harry Stebbings? So we've got China, we've got Europe. What about the US? What could they do differently or improve?
46:24 Speaker 00 There's a lot that the US does in terms of of research, which is to a large extent a bit of a partial explanation for the success of the tech industry in the US. Partly because the US devotes a significant amount of resources to fundamental research through NSF and NIH and various other outfits, probably more than Europe. Universities pay their faculty pretty well. Now, this comes with a downside, and the downside is that studying in the US is expensive and it's a trade off, right? So can you do one without the other? Switzerland figured out how to pay academia pretty well while. Actually offering free education to their students. So there is a way to do it. Canada also figured out a pretty good trade off as well. So a lot of things the US does right, but one thing that the US system or like the rough does right also is
47:10 Speaker 00 the willingness to take risk and invest on ideas that seem a little crazy, but basically the sort of vibrant startup scene in Silicon Valley and other places in the US, in New York and in the Boston area is leading, leading the world. Now you start seeing a similar thing in Europe now. There's been like an enormous growth, for example, of tech startups in Paris, the Paris area, in France more generally, continental Europe more widely and in the UK as well. And so I think that's a good thing, but it's still more difficult to have access to investment money in Europe than it is in US.
47:41 Harry Stebbings? That's why I'm here, Jan. I'm happy to provide. I'm going to do a penultimate one for you. When you think about what you'd most like someone listening to take away, what would it be? When they hear this, what do you want them to take away as the number one thing?
47:55 Speaker 00 AI is going to bring a new renaissance for humanity, a new form of enlightenment, if you want, because AI is going to amplify everybody's intelligence. Every one of us will have a staff of people who are smarter than us and know most things about, you know, most things and most topics. So it's going to empower every one of us. It's going to make us more creative because we're going to be able to produce text, art, music, videos without necessarily having all the technical skills that are currently required for doing those things. And so exercise our creative juices. So that's the positive side. There are risks, there's no question, but it's not like those risks. Don't believe the people who tell you that those risks are inevitable or that they will inevitably lead to catastrophe. That's just not true.
48:37 Speaker 00 Face yourself in 1920, who would have thought that a mere 50 years later, you could cross the Atlantic in a few hours in complete safety, you know, at near the speed of sound? And would people seriously want to ban aviation? Or call for regulation of jet engines before jet engines existed? I mean, that's kind of insane. So I'm not against regulation. There should be regulation of AI products, particularly the ones that involve making critical decisions for people. But regulating or slowing down research is complete nonsense. It's just obscurantism.
49:09 Harry Stebbings? Who's incumbent team do you most respect and admire when you look at Amazon, Facebook, Google? In terms of their approach and talent internally, outside of Meta, obviously.
49:22 Speaker 00 So this is changing a lot. And the reason it's changing is because a lot of people are leaving large companies and large labs. And the reason they're doing this is that until recently, a lot of AI research was very exploratory. And now there is a path towards commercialization for a lot of things. And so people think that they are better off just leaving large companies and doing this on their own, doing a startup and things like that. So you see a relatively large motion of applied research engineers, a few scientists, basically leaving those labs to do startups. And that's across the board, right? So you look at the original paper from Google about BERT or transformers, right? The thing that revolutionized NLP, all of them have left. Okay, they're all in startups. Some of the people who produced Llama, the open source slams from Meta.
50:06 Speaker 00 So the key people have left already, okay, to do startups.
50:08 Harry Stebbings? I saw their companies.
50:10 Speaker 00 Yeah, one, yeah, one called Mistral. Yeah, there is an insane amount of money in days. So more power to them. But I'm sad that they left Meta, obviously, but.
50:19 Harry Stebbings? But who's existing team, you're like, yeah, they're good.
50:22 Speaker 00 Yeah, yeah. But I think like in terms of the basic competence and the people who are going to push the science forward, because what we need now is not to work on applications of LLM. There's a lot of people who are capable of doing this and they're going to do a good job. What we need to do, people like me, who are really working on research, is coming up with new concepts that will allow us to get machines that basically have common sense, have an experience of the real world, have basically human level intelligence. And in my opinion, the outfits are in our best position for this are FAIR from on one side and the new DeepMind now, which is DeepMind plus Google Brain.
50:55 Harry Stebbings? Yeah.
50:55 Speaker 00 There are a lot of people there who are interested in that question. And I think they are probably the best, together with Meta, FAIR, they're the best position to really have an impact on this, something all of us have been working on for quite a while.
51:05 Harry Stebbings? Jan, if we do this again in 10 years' time, where is Jan in 10 years' time in 2033?
51:11 Speaker 00 Well, I'm 63, 10 years old now, but 12 years old now, I'll be Jeff Intens's age, okay? And I'm excited like a teenager now because I see the opportunity of the next step in AI and the opportunity perhaps to get to the goal that I set myself, so that I imagined for myself when I started working on AI many years ago, which of course I was very naive about. At the time of understanding intelligence, first of all, it's a scientific question. What is intelligence? What is human intelligence? And one good way, as an engineer, a good way to understand intelligence is to build a widget that actually reproduces it, right, to some extent. So I'm excited about this right now. I'll find the substrate, the landscape, the location, the position where I can make the best
51:55 Speaker 00 contributions to this. And currently, that just happens to be, to be fair, at Meta. I keep a foot in academia because I think it's very complementary and also important. There are projects of different types that you do in academia and industry that are complementary. So I like the combination of the two. As long as my brain keeps working, that I think I can contribute and that I've given the means to contribute, I keep working. And then there's some point where my brain will turn into white sauce or I'm totally out of it or something and I'll stop.
52:24 Harry Stebbings? Yeah, I want to say personally, thank you so much. I speak for many, I'm sure, when I say we've learned so much from you in terms of your public speaking and discourse and willing to speak. I think if you're willing to speak as openly as you have been. So thank you for educating so many of us and thank you so much for joining me today, Jan.
52:40 Speaker 00 Thank you so much for having me, Harry. This was, this was fun.
52:45 Harry Stebbings? I mean, what an awesome show. If you want to see the full video, of course, you can check it out on YouTube by searching for 20 VC. But before we leave you today, are you building enough conviction to outpace other investors in this changing ecosystem? Tegus helps VCs get under the hood of their investments quicker, map evolving markets with deeper research, monitor the financial health of portfolio companies and even source new deals. Tegus is trusted by top firms like Redpoint and Spark Capital for everything from expert calls and transcripts to quickly surfacing insights across filings, comps and benchmarking. Find out why a majority of the top venture funds are using Tegus on a daily basis to differentiate, consolidate and accelerate their research. Head to tegus.com/growth for your free trial.
53:30 Harry Stebbings? And speaking of tools we cannot live without there, Secureframe is the leading all-in-one platform for automated security and privacy compliance. Secureframe simplifies and streamlines the process of getting and staying compliant to the most rigorous global privacy and Security standards like SOC two and others and Secureframe's industry leading compliance automation platform paired with their in-house compliance experts and former auditors helps you get audit ready in weeks, not months, so you can close more deals faster. Automate your security and privacy compliance with Secureframe. Schedule a demo today at secureframe.com. That really is a must. And finally, are you tired of using the same old tools and platforms in an attempt to manage your global team? Well, us too. Why use 16
54:13 Harry Stebbings? different tools when you could just use Deal? Deal is the only global first HR tool with unmatched automation, customizations and integration, making it the only platform for built-in contracts, visas, background checks, payroll, equipment, benefits and beyond. Are you ready to see how Deal can scale with your global and local teams? Visit deal.com/20vc and book your demo today. That's deal.com/20vc to book your demo. Now stay tuned because on Wednesday we have Emad at stability.ai joining us in the horse. He's such an incredible show coming.
Previous 20VC: Why The Future of AI Is Open Not Closed, Why We Are Years Away From AI Being Autonomous, Why AI Founders Do Not Need to Move to the Valley & Why Founders Should Not Meet Investors in Between Rounds with Clem Delangue @ Hugging Face Next 20VC: Why the AI Bubble Will Be Bigger Than The Dot Com Bubble, Why AI Will Have a Bigger Impact Than COVID, Why No Models Used Today Will Be Used in a Year, Why All Models are Biased and How AI Kills Traditional Media with Emad Mostaque, Founder & CEO @ Page updated September 29, 2026 · transcript and summary are machine-generated
