AI in 2025 – A global perspective, with Kai-Fu Lee

Kai-Fu Lee joins me to discuss AI in 2025. Kai-Fu is a storied AI researcher, investor, inventor and entrepreneur based in Taiwan. As one of the leading AI experts based in Asia, I wanted to get his take on this particular market.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Azeem Azhar?

  Hi, happy holidays. While I'm away this holiday season, I wanted to drop here a few of my conversations with leading experts in AI. I discussed how AI might surprise us in 2025 with Ethan Mollick, Dylan Patel, Nathan Benet and Kai-Fu Li. The conversations were initially released for members of Exponential View. Here's my discussion with Kai-Fu Li. I'm delighted to be chatting to my friend Kai-Fu Li, a long-term friend of Exponential View. He's a storied artificial intelligence researcher, inventor, operator and investor based in Taiwan. He runs Synovation Ventures and has really been busy in the last few years with a series of generative AI efforts. Kai-Fu, thanks for joining us this evening.

• 0:43Speaker 01

  Yes, hi, Azim. Glad to be here again.

• 0:46Azeem Azhar?

  Now, generative AI has really swept the US and Europe. The term generative AI, it's become synonymous with artificial intelligence, much to the annoyance of many longstanding AI researchers who see the world, of course, as broader than that. Has that been the same in your part of the world?

• 1:04Speaker 01

  Yeah, I think overall in Asia, specifically in China, I think generative AI is generating a lot of waves. It's probably not at the same height as the US, partly because I think when ChatGPT came out, it was a one of a kind. And it was not believable that one could have that kind of user experience and quality. So that kind of became viral. Whereas in much of Asia, I think there are a number of competitors by the time the term became hot, both American and Chinese and from other origins. So people are feeling, oh, a number of companies can build it and they are free. So I think it's not created. It didn't have

• 1:47Speaker 01

  that ChatGPT moment. But certainly more slowly, it's kind of taking off or people are aware of it. Certainly in the research community and in the product investment community, it is the most interesting and exciting area.

• 2:02Azeem Azhar?

  Right. I mean, I agree there was this incredible ChatGPT moment. It must be the most commonly used headline on slides when people give their presentations on AI in the last year and a half, the ChatGPT moment.

• 2:17Azeem Azhar?

  The data, the market research data showed very, very rapid uptake of the tools by, by employees right across the broad board. I mean, particularly, I saw data coming out of India where the rates of use within a year were extremely high. OpenAI has said 305 million people are using ChatGPT every week and Menlo Ventures, which is a venture capital firm, had some data showing that the amount that US enterprises were spending on GenAI products, mostly on foundation model access, had gone up sixfold to about thirteen and a half billion dollars at their forecast for 2024. I mean, do you have any way of quantifying something similar

• 3:00Azeem Azhar?

  in, in your parts of the world and in China in particular? What do you think the numbers look like there?

• 3:05Speaker 01

  Well, it's probably not those numbers, not at that level yet. The market penetration is still smaller. And I think the enterprise spending is smaller, but some interesting phenomenon we, we see in, in China. First is that because the access to good models came a bit later, because OpenAI didn't make their models or ChatGPT available in China. So there was a latency in the beginning. But now the Chinese models are good enough and they're actually very cheap. And so that's enabling a lot of applications to coming out, to come out. And we see more and more interesting products going in, in the consumer space,

• 3:47Speaker 01

  both going in China and also going overseas. The other thing we see is that the businesses in China, it's a double edged sword. There isn't as well established a layered approach to the SaaS and enterprise software. So companies tend to roll their own, which is of course bad in. Entrepreneurs want to develop a new space in the SaaS. But the good thing is, these enterprises are getting their hands dirty. Dirtier than the American companies, right? The American companies rely on stacks of APIs and and technologies and products. And the Chinese companies are going deeper.

• 4:28Speaker 01

  And actually, at this formative stage of the generative AI technology, one probably has to go deep to extract the most value and to develop the greatest economic benefit for the company. So we're seeing interesting implementations that are helping individual companies, but maybe not as easily expandable throughout the the space.

• 4:51Azeem Azhar?

  Well, I mean, the ramifications of those two points that you've made, which is cheaper and cheaper models that are getting up to GPT-4 capability and the fact that enterprises have to roll their own are really, really significant. So we're definitely going to come back to those two two points. I think they are super, super interesting and we should get there. Turn to how you and I are both using these tools. Has your, how has your day to day work changed in the last eighteen months or two years now that these tools are a little bit more mature?

• 5:24Speaker 01

  Uh, yeah, of course, I don't do anything without generative AI tools, but I want to make a particular emphasis on AI-first tools. That is to flip on its head where AI becomes central. Human is on sort of behind the scenes, backseat driving, but AI is driving.

• 5:43Azeem Azhar?

  Can you give an example?

• 5:46Speaker 01

  Yeah, absolutely. One example is with Microsoft Copilot. It is it is able to have you create documents. It's a wonderful product, but the human is still in the driver's seat. And you've got a really nice coach sitting next to you saying, oh, I can fix that. I can improve that. But I think the right tool should be AI is writing better than the most of us. Not not better than you because you're a well-known author, but AI writes better than most of us for most type of things we write, from letters to poetry, to business, to applications, to research reports, to PowerPoints. So we've invested in the company that build

• 6:25Speaker 01

  a product called Pop AI, P O P A I, and it is an AI-first application in the sense that when you go in, it doesn't expect you to write anything. It expects you to continuously to prompt, improve, feed in documents, and then inform AI how to make a better document for you. When you're all done, of course, you can edit it, but it's an A. The key difference with Microsoft Copilot is AI does the writing, human does the tweeting.

• 6:53Azeem Azhar?

  That's a that's really interesting. And one of the of the implications of what you've said is that when we think about writing, we often think about the words that just get produced one one after another. And I noticed a significant difference between GPT three and my go-to model now, which is Claude three point five new Sonnet. By the way, they need better product names than they currently have.

• 7:16Speaker 01

  Yeah.

• 7:17Azeem Azhar?

  Is is that it has a conceptual understanding because writing really, as as any creative endeavor starts, right, starts with this, the structure, the infrastructural ideas.

• 7:29Speaker 01

  Yeah.

• 7:30Azeem Azhar?

  And the words that represent those ideas flow from that structure. What I found of GPT three and three point five was it was pretty superficial. When I now work with with Claude, I think it's getting deeper to the fundamental ideas and how they they in interact, which tells me something about an L L M being feeling to me as more than just something that produces the next token. Does that make sense to you?

• 7:56Speaker 01

  Yeah, I think Claude three point five is special in that way, both in its better writing, in the more complex scenario and better code. One other interesting aspect in the Pop AI product I mentioned is that it has a feature called humanize. It's used to address an issue that people are starting to able to detect, hey, this is written by chat G P T. And teachers would say, check with students, hey, did you write this with ChatGPT? So an interesting feature called humanize is a special model that turns any LLM output that can check it for whether it's likely to be written by an LLM and then modify it so that it feels more human written.

• 8:38Speaker 01

  So I think, you know, we're starting to figure out ways, two years after the launch of ChatGPT, to fix a lot of problems, like recency was a problem, hallucination was a problem, and the recognizable non-human-ness is a third problem. Now we're sort of patching each one, so it's getting to higher level of usability.

• 9:00Azeem Azhar?

  And, and so in your, you know, your everyday, how would you use, you know, how would you use Pop AI?

• 9:07Speaker 01

  Yeah, so I would, if I want to study something, like is scaling law reaching, you know, this limits? I would find a bunch of documents that are from credible people and I would feed all of them in, not one, but all twenty, and then I would converse with it and understand it. And then I would use my search tool to find out more stuff. And then when I'm done, I would have it summarize and write something for me with my proper prompt. And then I would tweak the prompt until the result looks good. Then I would humanize it, and now you know the secret of how I do my writing now.

• 9:43Azeem Azhar?

  Well, that, I mean, I think, I think that what you're doing as well, that is the, the kind of consistent interaction with, with the system. And something that struck me about the distinction between these AI tools and pop.ai sounds like a good example of this, is that with something like Google, Google Search, the difference between someone who knew how to do it well and the average person was not very significant. But with the generative AI tools that we're starting to use, whether it's a more traditional LLM chatbot interface like Claude or pop.ai, which is obviously more developed, it feels like the gap between a ten X or a hundred X human and an X human is more significant, right?

• 10:27Azeem Azhar?

  It's a combination of the domain expertise you have and also your understanding of how, how the tool works.

• 10:35Speaker 01

  Yeah, and it's interesting you mention Google because the other example of some the tool I use that I've introduced to you is called Beagle. B E A G L, another one of our venture build companies, and it's a search engine that gives you one answer. Because I remember fondly when I went to Google, Larry Page said, Google's not the best way to do search. Because when you search, you don't want a bunch of words and get a bunch of links. You wanna ask one question and get one correct answer. Of course, that's been elusive because getting the correct answer has been difficult. But I think we're very close now. There are a number of search overviews by Google and others, but Beagle has found an interesting way

• 11:19Speaker 01

  to significantly improve factuality and with a very high content rich flow and also interspersed with images. So brings kind of brings a different kind of a search experience. And I think at a at a grand level, it's a small step towards Larry Page's vision of one question, one answer. So now, when I want to know something, especially something complex, right? If you want to do something simple, any search engine will do. In fact, other search engine will do better for things like navigation. But if you want to ask something complex, like, why would Elon Musk want to serve in the government? When he's such a brilliant business man and an inventor?

• 12:04Speaker 01

  What does he get out of it, right? These are the exact words you type into a one question, one answer, answer engine, we should call it, search engine. It gets you get an elaborate answer with all the possibilities, combining multiple sources. And and again, you know, one of the problems with early, why couldn't ChatGPT have done it? Because they didn't solve the recency problem. That was solved by the recent advance of RAG and use a vector database. And and also, if you could have a search engine that has a, you know, up to the minute crawl of all news and then a full repository of truth. Like Wikipedia, WebMD and things that are held to be true.

• 12:48Speaker 01

  And a bunch of social stuff that's not necessarily true, but recent and interesting and provocative. If you very quickly crawl these things and then combine it with a full search engine and an LLM, that's how you build a product like Beagle. So it couldn't be built a year and a half ago. But now, you know, also ChatGPT has a search capability. Perplexity is pretty popular. So I think, I think in, uh, I would bet, um, regardless of who wins in two or three years, we will mostly move out of the current search engine into what I would consider answering.

• 13:26Azeem Azhar?

  Yeah, so it's interesting that both of the examples that you've, you've given are, um, more sort of sophisticated applications, right? The, we think about the foundation model, you access it via an API, then we threw a chat interface over the top. But we, we haven't perhaps outside of, um, you know, image and video generation, seen a lot of, of, of, of applications. My sense of twenty twenty five is that one of the words that will be common will be the idea of agents, right? Agent-based systems emerging. But I, I'm curious that you've said applications a few times. So when you look at a twenty twenty five, do you think it's more about applications?

• 14:04Speaker 01

  Well, I think we will have mature text tools, like the tool I gave. I think we will have near mature multimodal tools. I think, uh, text to image is now mature, getting mature. So I would say, you know, text and text to image, we're going to see lots of applications in lots of applications and products. They will come out everywhere because inference cost is getting lower. Text to video, that's still a bit expensive. It's still got rough around the edges. So I would say that's maybe a year behind. Agent, I think we'll find some really exciting vertical applications. But the pervasive use, I think it's still got rough edges to come out, right? If ChatGPT was an indication, it would shock the world, but didn't change.

• 14:50Speaker 01

  People in the way they use tools until the main problems like hallucination, recency were fixed or non-human-ness were fixed. So my prediction is that we've taken time to fix text and text to image. We'll need to take time to fix text to video and we'll need to take time to fix agents. But for those two technologies, we'll see vertical applications that are less vulnerable to the current problems of these technologies that just came out are exciting, but either are too expensive or still have rough edges to be smooth over.

• 15:26Azeem Azhar?

  So the, so the point being that for a broad application, right? You need a tree of robustness because a broad application is all about edge cases. Whereas in a vertical industry application, you know, everyone's uses a bit more constrained.

• 15:38Speaker 01

  Yeah, I'm also very, very excited about agents. I love the, the, the anthropic demo of controlling a PC. There's a GNI company in China that's built something that controls a phone. And, and they're both very nice demos, incredibly impressive because they're the, the, the extent to which they can control things. But a couple of issues, I think one is, you know, still the speed and then the rough around the edges that would take time to fix. But the more fundamental issue is, hey, these two products were designed not to be driven by voice. Why are you gluing a voice interface to something not designed to be voice driven?

• 16:18Speaker 01

  But this was something I discovered like three decades ago when I tried to build speech into the Mac when I worked at Apple. And then later I've seen other people try to build speech into the phone. And even today, they're not that common because the, the, the, on the PC, people have a big interface called the keyboard and the trackpad or the mouse. So that is often the default way that people use this. But more importantly, the apps were built to be driven by keyboard and mouse. So now you're using voice to drive.

• 16:57Speaker 01

  The PC, when there is a keyboard and mouse sitting in front of you.

• 17:00Azeem Azhar?

  Yeah, yeah, it's very inefficient, right?

• 17:01Speaker 01

  Yeah, you start with a handicap, right? With a phone is better because the phone, I think, has not as good an interface. And it's something that has a microphone very close to you. So driving a phone is better, but much better would be driving something like, you know, a task, like planning a trip or something like that. That was by definition voice driven.

• 17:25Azeem Azhar?

  I mean, what I love about the voice apps, I use Claude on the phone when the voice mode where you, and I can just in my commute home when I'm dropping my, after dropping my kids to school, for 10 minutes talk into this with the, anything from how I have to respond to the insurance company to the new idea I've got to notes on a company I'm looking at investing in. And I get home, open the app up and it's structured into four sections of there were my thoughts. Now, what it can't yet do is turn those into actions, right? I want to turn it into the letter to the insurance company and start to sketch out my diligence document for the investment, but it hasn't got that far. But I can absolutely see that, see that happening. So I think, I think voice is, is pretty interesting.

• 18:08Azeem Azhar?

  I want to turn to, to the, the core technologies at this moment. So a couple of weeks ago, you know, absolute grandee, even though he's a young man of deep learning, Ilya Sutskevich was talking at the big flagship conference, and he declared, made some comments about the scaling laws. So the scaling laws have been the thing that really sort of driven the investment levels. Throw more data, throw more computes, make a more complicated model, get a better model at the, at the end of it. And he says, look, pre-training, which is a type of scaling, as we know, will unquestionably end. We've reached peak data and there will be no more. And we're going to have to deal with the data that we have. Do you agree with him?

• 18:50Speaker 01

  If I were working at OpenAI, uh, I would surely agree with them because they have their most advanced. If they say so, it must be so. And we also hear issues about GPT five scaling as much well as before. So it's definitely reaching diminishing returns if not reaching its limits. But I would say because we're not OpenAI, so I wouldn't fully agree that we're done, right? OpenAI may be done, we're not done. Because what these giant models do is that they set a ceiling below which you can basically fine tune your model to to be to be close to that ceiling. And if I'm not OpenAI, my ceiling's lower. So I want to raise the ceiling so I can fine tune towards it. That's the first point.

• 19:34Speaker 01

  Uh, the other is when you have a great model, it becomes a teacher model. So whether you call it reinforcement learning with AI feedback or using better model to improve smaller models, you still need a bigger model. It doesn't have to be yours. But I think what he states is definitely true, especially knowing what he knows. But I would also add a few more minor points. First is that we haven't fully exhausted the data. There's a little bit more because my experience at zero one dot AI is that when we added Chinese to English, not only did our Chinese get really good, but our English got better. And our Chinese data is very, very, very high quality.

• 20:17Speaker 01

  Because we're a poor little startup, so we can't afford to train on a lot of data. So we find really good ones. And and we've done it for Chinese, a number of other Chinese companies have done it for Chinese. But I bet nobody has done Japanese, Korean, Spanish, Arabic. So and when you go deep, there's the internet data, there's also the book data. So I think we can probably still increase the high quality data by two X. Of course, that's not the next mortar of magnitude, but it's also not reach the limit. That's the one one difference. And the other thing which I'm sure he would agree with is that there are other scaling laws. Like, you know, O one scaling on the inference.

• 21:00Azeem Azhar?

  This is this is chat GPTs. Yeah, yeah, that's right. So so he's talking, I think, about the the big pre-training scaling where the the engine. Chops up and processes material and produces this final product. But what we've seen now is what people call test time scaling or inference scaling, where you know, you ask the machine the question and it generates lots of different answers and evaluates them after the fact, goes through an iterative process and then presents a final answer to you. And I actually just had sometimes with ChatGPT's O1 Pro have had it sync for a few minutes before it gives me a response. And in that time, that's that's a computational problem. That's a lot of scale that's being being thrown to it.

• 21:45Azeem Azhar?

  So my interpretation is that the demand for computing capacity is still going to go up, but the ratio moves in in favor of compute that's required for inference, not just for the training upfront.

• 22:00Speaker 01

  Yeah, yeah, I think that's right. So the amount of compute is still a lot. Jensen will still be very happy. He can sell more inference chips. Um, one other point I would make is that super fast inference becomes really important, right? Before, I think super fast training was important. And one of our at zero one, our special results was that we were able to train, pre-train a very good model uh for just three million dollars. And and that was, you know, two thousand H one hundreds uh running for a month. And the model is currently ranked number seven in the world. So that was something we focused on. We're glad we did it. Otherwise, we would have spent hundreds of millions, perhaps, or at least a million, hundred million,

• 22:44Speaker 01

  which we don't have in the bank. So we're glad we did that. But I think and also about nine months ago, we said, okay, making inference fast was more important because inference was too expensive. If you make it fast, it also becomes cheaper. Uh, and that was important. Our other accomplishment was our inference is around 10 cents per million tokens, which is pretty cheap.

• 23:07Azeem Azhar?

  Okay, let's put some, let's put some framing around this for for for people. Zero one is one of your startups focusing on building foundation models.

• 23:15Speaker 01

  Yeah.

• 23:16Azeem Azhar?

  Yeah. And you said two thousand H one hundred. So the H one hundred is one of Nvidia's GPUs. I think the general view is that some of the big models were trained on forty or fifty or a hundred thousand hundreds. And Elon Musk is building this hundred thousand cluster, has built this hundred thousand cluster. The other thing you said was inference cost per token is ten cents per million tokens. So a million tokens is roughly eight hundred thousand words. It's what I might say in a year. And at ten cents per token, you've just forced me to update one of my databases. Because we had twenty four cents per million token as the cheapest GPT four quality model.

• 23:59Azeem Azhar?

  And to contextualize that for listeners, we were up at two hundred dollars a million tokens a couple of years ago. Right? So the price has absolutely dramatically come down.

• 24:09Speaker 01

  I I think, you know, if people don't know a million tokens, I would say if you were building an AI search, a million tokens will give you about a hundred searches because we built an AI search.

• 24:20Azeem Azhar?

  Right.

• 24:21Speaker 01

  And and so if you think you're building a search engine, so ten cents per hundred searches is point one cent per search, which sounds really low, which it is. If you were back in the early days of GPT four, it would have been seventy five dollars per million tokens. That would be seventy five cents per search, which would bankrupt any company, including Google, especially Google, if they were to.

• 24:47Azeem Azhar?

  Yes, because they have the most searches. Yeah.

• 24:49Speaker 01

  Yeah. So that's a good comparison. But the other point I want to point to is because we're now focused on inference time or test time scaling, which means think longer and longer and longer on the test time. So the inference cost becomes so important because when I'm talking about, you know, ten cents per million tokens in in the GPT, the O one search, it actually does a lot of tokens because it charges you for it because it does, but it doesn't show you that those are its chain of thought tokens. So that's kind of one aspect. The other aspect is when. Stinking for three minutes, it's crunching on GPU. So if you have a faster inference engine, it would crunch less, right?

• 25:32Speaker 01

  If your inference engine were twenty times faster, you wouldn't need three minutes, right?

• 25:39Azeem Azhar?

  How do you speed up that inference? Is it architectural? So is it about the chips you end up using? Is it about designing particular chips that are good for inference? Or is it things that you do algorithmically? Distillations, optimizations, other things that you can do in the model?

• 25:56Speaker 01

  Yeah, I think about a third of it is algorithmic. Like, we use a very unique M O E model, that's mixture of experts.

• 26:05Azeem Azhar?

  Mixture of experts, yeah.

• 26:06Speaker 01

  We have, you know, many, many mixtures. You know, some models like Mistral, they have eight experts. We have, you know, more than ten times their number of experts. So they're finer, granular, so able to capture information better. And then each at the runtime, we activate fewer, a fewer parameters total. So our total average total actively active parameter is twenty two B. So roughly, it's.

• 26:33Azeem Azhar?

  Twenty two, so just to clarify, twenty two billion parameters that you activate through the network. Think that, people think that the Claude model is about a four hundred billion parameter. Model G P T three was a hundred and seventy five billion. So you're activating only a small portion of that, which reduces the computational cost.

• 26:51Speaker 01

  Yeah, our total parameter size is also over two hundred. But because our active is smaller, it's faster because of that. That's a third of the reason. Another third is we optimize our inference engine to build on a special cache memory architecture. Because GPU is expensive, so how do you do, do you save money? You have less GPU computing. How do you have less GPU computing? You use cache. Where's the cache? There's the H B M on the GPU, but there's also RAM and S S D and potentially SRAM you could have on the CPU. So while the GPU is crunching, the CPU is orchestrating, orchestrating data to the right places every.

• 27:35Speaker 01

  Remembering pre-computed numbers so they don't have to compute it again. So.

• 27:39Azeem Azhar?

  Right, I'm gonna, I'm gonna, I'm gonna pull out some acronyms again for the audience. So HBM is a high bandwidth memory on the Nvidia GPUs, SSD solid state drives, SRAM is a RAM that's close to the chip. The CPU is the non GPU chip and for most of us, it's what we have thought of as a computer chip. And if you're watching this on video, over the back of my shoulder is a BBC B micro, which has got a six five oh two uh CPU sitting, sitting in it. I think what's interesting about what you've just described, of course, is how the mighty CPU has fallen. Now all it does is orchestrate data flows from different memory, different latency for a GPU. Whereas, you know, five years ago, it was a thing that mattered.

• 28:24Speaker 01

  Yeah, yeah, exactly. Right?

• 28:25Azeem Azhar?

  Yeah, yeah.

• 28:26Speaker 01

  So building that memory cache architecture and designing a special computer with the right memory and, and, and parts was critical for about the third. Now, the computer is not a custom computer. It uses standard parts, but it just has a ton of memory and runs our software to, to use CPU to orchestrate the, the data. So that's about the third. And the last third are just a bunch of engineering tricks, like how many bits do you need to represent it? How do you basically deal with context windows and can you not deal with um uh twenty two B? Can you sometimes answer the question with four B, sometimes ten B, sometimes two B?

• 29:09Speaker 01

  The fewer you use, the more money. So just basically, basically penny pinching and squeezing everything out possible. So three things.

• 29:15Azeem Azhar?

  But this is, this is super interesting because what you're describing to me here is the layer below what we talked about earlier, which was AI native applications. Here, what we're understanding is that the engineering of the infrastructure and the, and those architectural choices is really, really important and is a new set of disciplines that perhaps we've only got two or three years experience of. And then it makes me think about twenty twenty five and beyond, about the class of applications that we might have and the choices that we might, we might have. So if you speed up inference and so instead of having to wait three minutes, you get the same quality of response in nine seconds. As you said, you twenty, twenty times improvement.

• 30:02Azeem Azhar?

  That sounds great, except what if I really want the best quality response I can and I'm willing to wait three minutes? Right? So don't we end up having a kind of a perito frontier, a set of trade offs where I can say, listen, this is important enough. I'm willing to go off and make a cup of tea, think about it for three minutes. And then I'm sitting there thinking, how do you make that easily accessible as a choice for an end user? Otherwise, I'm constantly faced with the cognitive load of where I want to put that slider. So, so there seems to be, actually, that feels like it's a UX, user experience and design problem more than an, an, an engineering problem that we need to, we need to address.

• 30:44Azeem Azhar?

  Cause there's any number of tasks I want responses in a second, but there are ones which are so important, I'm willing to wait half a day if necessary. How do you think?

• 30:53Speaker 01

  Yeah, yeah, I, I think, yeah, clearly you lose a little bit of precision in doing the speed improvement that we have. It turns out not all that much, but they do lose some. But in some cases, if you're trying to win the Nobel Prize and you have, you're the human who's having the O one pro being your research assistant. You'd rather it take longer to get the right answer, right? But if I'm doing a search, I would rather get an answer instantaneously than wait a minute. So you have to really give that flexibility, I think, to the application developer. So they can have super fast and not perfect or really slow or somewhat slow, but as good as they can do.

• 31:36Azeem Azhar?

  Yeah, and, and I think some of this ends up being about perceived time for the, for the end user. One of the things I noticed now with, with Google searches is Google searches are very fast, but I may have to do six or eight to get the result I want. But the total time ends up therefore being a few, a few minutes and. In some sense, I might be happier with a response that takes two minutes. But the trouble is, what am I going to do during those two minutes? While I, while I sit around and wait, right? That feels like an eternity compared to just doing another Google search. And this, this all, you know, is great opportunities for UX designers and, and, and, uh, you know, product, product people by the sounds of things.

• 32:19Speaker 01

  Well, as you just made a very insightful point, I have to point out. It's something we weren't ready to publish before, but I talked about our Beagle search.

• 32:28Azeem Azhar?

  Yes.

• 32:29Speaker 01

  And that's in fact what it does. It, you ask a long question, it decomposes it into multiple searches. So it calls Google Search not once, but a number of times. And then the results come back and are aggregated. So, so then you can get the answer pretty quickly, but it's actually mentally modeling your process. But on your other point about the user, the interface designer, having to really understand the user to, to design the right interface. And I think it's going to be hard to be a UI designer who doesn't understand something about the technology. So now they have to know more about that as well.

• 33:10Speaker 01

  Um, and, and, um, give feedback to the people who are building technologies.

• 33:15Azeem Azhar?

  I want to come back to the two really interesting points you made at the start of our discussion. I said, I'm going to hold them back, uh, so that people kind of get through to this far. So you, you said a couple of things. You said that, um, you know, models are getting cheaper and cheaper. And particularly we're seeing in, in China from, uh, and, and the region both from zero one, but also deep seek and when models that are extremely good and extremely cheap. And it, it, it suggests a path. And one of the beauties of the, the, the cost of these models being cheap is it's easier for people to, you know, experiment and make mistakes.

• 33:52Azeem Azhar?

  The second thing that you said was that, you know, Chinese enterprises are not used to the kind of clean layers of SaaS and infrastructure that American firms are. And so they're having to get their hands a lot dirtier.

• 34:08Azeem Azhar?

  And getting your hands dirty is really about experimentation and about learning, right? That's what it involves. And the reason I find these two trends so interesting is that as I've talked to enterprises across Europe and the US and some parts of Asia, the thing that I've said is you have to build your learning muscle because the technology is changing so quickly. You know, you can't just pick up the phone to Salesforce and the thing is delivered. And the only way to do that is actually to get close to the technology and to build and to experiment and to make mistakes. And that is where commercial velocity will ultimately come from. So when you made those two points, I was sitting there thinking, wait, does this actually

• 34:51Azeem Azhar?

  somehow benefit an ecosystem that doesn't have a really mature SaaS environment in the SaaS?

• 35:01Speaker 01

  It's, I'm hesitant to reach that conclusion, but it's a plausible consequence. To give you an example, we're working with a very large patent team, patent company in China. They have a lot of patent experts who are engineering plus legal experts who understand the invention disclosure, works with the inventor, write something and file a patent. That, as you know, is a very expensive process that very highly paid people get paid doing. It's very hard to get in the door. So if you could somehow improve the efficiency of that job, you can either make a lot of money, more money, or save a lot

• 35:41Speaker 01

  of money, or get more patents done. So in a typical American case, I think people would say, okay, well, I would use the Claude API, the G P D four O API, and I would bring in some, you know, Accenture or whoever to come and implement something for me. But I think, you know, the system integrators aren't going to be domain experts. They're not going to be the best at the chat, the L L M stuff. And then the patent lawyer people don't know the technology either. So you have three people with discreet skill set that may take a Long time to reach the right solution.

• 36:19Speaker 01

  And then they insist on building it on top of whatever they have. They may have Databricks, they may have SAP, they may have Salesforce, whatever combined together. So in China, this team just called us this company. It's a rather large company with thousands of these lawyers and engineers. They just says, well, we have, you know, some. Can't disclose the number. We have, you know, hypothetically, two thousand lawyers who are engineering people. Uh, can we, can you help us do four thousand lawyers worth of work? And we go in and say, well, we can, but it's going to take a lot of tweaking. Says, well, you tweak it with us. Sure. And we say, if we tweak it and you get four thousand lawyers of output, would you give us X percent? Let's say ten percent of your savings. And then they would say, yeah.

• 37:02Speaker 01

  And then we might even say, I'm hypothetically speaking again, maybe we can form a joint venture and we can develop a software that will give you the hundred percent improvement in efficiency. Then we can sell it to other patent firms who can get similar benefits. And then this company might be able to go public. So it's a completely different business model. So we feel if we do the joint venture, there's a lot of upside. So we're happy to send five experts and ten engineers to do this task. You know, and OpenAI would never do this, right? So we'll see how it works out, but it's a very different business model, very different type of upside. And with open minded traditional companies like this patent firm,

• 37:46Speaker 01

  we might be able to hit something interesting. I would not quite be ready to say that, um, you know, it would disrupt the American default. I think that's going to be difficult because if you ask me today, would I rather play in the American ecosystem or the Chinese ecosystem? I think I'd pick American because there are many, many low hanging fruits to build a good business.

• 38:07Azeem Azhar?

  Yeah, yeah. I mean, definitely I'm seeing a number of startups that are taking around ground level up approach of the kind that you're you're you're talking about in the in the US US market and.

• 38:22Azeem Azhar?

  You know, I think that you've got a classic innovator's dilemma if you're a traditional SaaS company, because the way in which you build, the way in which your products work, the way in which your teams are organized, doesn't necessarily sit nicely with an AI, you know, an AI model. But I think they can market more aggressively, of course, than any of these startups.

• 38:41Speaker 01

  So I was just thinking of a paper from, I think Sequoia wrote the paper. They said it used to be software as a service, now it's service as software.

• 38:50Azeem Azhar?

  Right.

• 38:51Speaker 01

  Right. So when we go in and help this patent company, it's flipping, flipping on its head.

• 38:57Azeem Azhar?

  Yeah, and you're charging like a share of the kind of combined increase in value, which is means everyone's aligned. And I think finding AI alignment, so pricing for these tools over the next year, 2025, is going to be, going to be quite interesting. You know, should you be charging per API call, which feels a bit naive? Should you be charging per seat? Well, that also doesn't feel right in an autonomous sort of self-running system. So I think that that'll be interesting to explore in 20, in 25. Let me, let's understand, let's come back to where we think we might end up in 2025 by looking back at 2024.

• 39:36Azeem Azhar?

  Would you, if you can cast your mind back to the Kaihui Lee of December 2023, and you look at where we are in 2024, have you in general been surprised to the upside? Things have gone faster, deployment has been quicker, there have been more breakthroughs? Or are you, is it where you thought it would be, or are you surprised to the downside?

• 39:58Speaker 01

  I think the, um, the technological pace is faster than I thought. The barrier of entry was lower than I thought. The fact that a number of Chinese companies are now reaching the, entering the first tier on the shoestring, the fact that a number of startups like Mistral, Cohere are able to build interesting models and technologies, and the fact that Anthropic could challenge OpenAI by taking away its market share is really more dynamic than I had. Have seen. So that's a bit of a surprise. So the total amount of innovation piecemeal, I think in aggregate is very good.

• 40:39Speaker 01

  Maybe roughly equivalent to what I thought. The number of killer apps is probably a bit a bit lower than I thought. But then I would attribute that to the inference cost becoming what what has been an inhibitor. Because the cost was prohibitive for apps to be built on expensive APIs. But now that has changed. Yeah, and I'm a little disappointed that multimodal is still more of a demo. Than adding real value with the text to image being one notable exception, but I think that will change next year.

• 41:14Azeem Azhar?

  Why? Why is multimodality important?

• 41:18Speaker 01

  Well, it's clearly important ultimately because we live in a world that is three dimensional. And we don't learn closing our eyes and just, you know, input text and tokens. We learn faster, better, because we have a world in which we uh live in. And we have functions well beyond cognition. We also manipulate and move and reason spatial reasoning. So and and ultimately, if we had AI that was really smart. It needs to understand the world around us. It needs to move around. It needs to manipulate things. It needs to get physical tasks done.

• 41:59Speaker 01

  So it's like when those things are possible, it's hugely beneficial, maybe even bigger than text. But I think we are limited in a number of ways. I think one is just we have a history of text data, right? Thousands of years of texts we've kept. We didn't have thousands of years of images, certainly not video. And what little video we have is either movies or TikTok. And those don't represent videos of the real world. So our data is is less good. So it's understandable why it took longer and and also, you know, video understanding has come a long way. But where are the apps? So I I think hopefully next year we'll find a lot more apps because you can't let researchers.

• 42:45Speaker 01

  Come up with the apps. It's like, you know.

• 42:48Azeem Azhar?

  Good to be said. Yeah, yeah.

• 42:49Speaker 01

  Yeah.

• 42:50Azeem Azhar?

  Okay, so coming back to twenty twenty five, we talked about multimodality in this multimodal world we live in. That has an implication on robotics. I have changed my mind this year on self-driving cars. I had a chance to chat with people from the Apollo project. And of course, I went to San Francisco and I could not get out of Waymo cars. I just wanted to get in them. And so I really realized that they they're changing. I think Apollo in Wuhan is going to be profitable this year. Waymo is is already as big as Lyft in San Francisco and is getting access to more and more cities. So I think on self-driving cars, twenty five will be a story of the start of that part of the S-curve where things really take off.

• 43:30Azeem Azhar?

  The other thing that I'm I'm starting to think about with humanoid robots is I didn't really believe the thesis of humanoid robots in industry. Because I think that, you know, as JD dot com has with their warehouses and Cardo in their warehouses, you you just redesigned the warehouses to be roboticized and automated.

• 43:52Azeem Azhar?

  And then I was, and with that, I was uncertain about whether humans, people would want, consumers would want humanoid robots. But even there, while I haven't changed my mind, I'm open to the possibility that they would start as luxury products. I'm looking at what's happened with Unitree's robots coming out of China and then Figure and Sanctuary and so on in North America. And so my mind changed slightly in in in twenty four. Thinking about robotics, self-driving cars or or or humanoid, what's your view of where it is? And how might your mind change in in future couple of years?

• 44:28Speaker 01

  Uh, yeah, um, actually, my VC, Sanovation Ventures, invested in about five autonomous vehicle companies. We're happy to see they're all going IPO now, so that's that's great. I think what you describe is very accurate. I'm I'm quite bullish, but I think we still have a couple of hurdles to overcome. One is basically they need to go from their current technology to more of an end-to-end technology. That is, they're still glued to. Together with old AI technologies. They sort of work, but even Waymo, I don't know if they're end to end. Tesla's close to end to end, but then they don't have all the sensors. So I I think a lot of these smaller autonomous vehicle companies have to basically revamp their technology stack.

• 45:13Speaker 01

  That will be a challenge. And also, I think we're probably as this technology matures. It may be the first one where the professional drivers may have various complaints, right? So so jobs being displaced in large quantity in this case. Truck drivers, for example, are highly paid and not easily able to take on any new job. Because they're so good at what they do and they haven't done anything else perhaps for their careers. Because it's a highly paying job. And also, I think accidents and deaths may create more issues. So we'll have to see. Those are not technical. The only technical issue is upgrade to the end to end or L L M like capabilities.

• 45:58Speaker 01

  So that's how I feel generally bullish. Embodied AI, I'm also very fairly bullish, but a little bit cautious about immediate applications. Embodied AI means, you know, the basically the AI is able to to capture data and integrate into the real world. It doesn't have to be humanoid. So that aspect I'm bullish. And I think it's critical to taking us to the true multimodal application. Now, the humanoid, I still reserve some conservatism about that. Because I think the difficulties with mechanics are not necessarily solvable by these fast improving AI algorithms. There's still mechanical parts. Also, I think with humanoid robots, there's one thing very special about it.

• 46:44Speaker 01

  Is because you're now not you're not benchmarking it with a robot dog or a robot car. You're benchmarking with a human. So where it falls short to the human, people are gonna get disappointed. And you set a very high bar, which is, you know, why early efforts to deploy natural language processing for customer service was has been difficult. Because people assume it's got to masquerade as a human. And when you can't, people are unhappy. When, you know, your first generation, you know, smart speakers from Amazon start to talk to you. You're very happy when it can just. Playback a song. But once you start, once they entice you to talk to it, you think it's human, then you realize, hey, you're not human.

• 47:29Speaker 01

  You don't know all these things. Now, L L M's changed that. But now the question is, you build this human that dances, oh, that looks cool. And then it enters your home, then you're saying, hey, what, why are you falling down here? Why can't you pick up the clock like I told you? Or why can't you shut off the T V like I told you? Why can't you cook me a meal? Aren't you a human? So that high bar may end up, and the high price that you pay, so you expect a high bar, which is human-like. So that might create a wave of disappointment as well.

• 48:00Azeem Azhar?

  Interesting, interesting, yeah, I I can imagine that that high bar, that set of expectations. Um, is pretty tough for the humanoid robot. But we are in the expectations game during this conversation. So let's turn in the last couple of minutes to twenty twenty five in general. We we'll go for quick answers because we're at the end of the call. What do you most hope will emerge in the field in twenty twenty five? What is the thing that you would would make it a dream year for you?

• 48:30Speaker 01

  Agents, real multimodal app and seeing every app being replaced with an A I native app. And that would be, those are the three I I look forward to.

• 48:41Azeem Azhar?

  You've got a big wish list there. I mean, I know we're near Christmas and I've got a beard, but I I'm I'm not as good as Santa Claus when it comes to to to presents. Um, when you look out for for twenty five, are you more likely to be surprised to the upside as we get to the end of twenty twenty five? Uh, or to the downside?

• 49:00Speaker 01

  I think on apps, we will, I won't be, but I think most people will be surprised with how fast and furious these will come to the upside. I think technology and model, it's hard to say because like Ilya just turned a whole world optimistic about scaling law to not so optimistic. So I think there will be disappointments. I also think on the technology side, Open A I, I respect them greatly as a company. They're brilliant, they're industry leading, but.

• 49:32Speaker 01

  I think Sam makes a lot of promises. And when those promises are not delivered, that would cause a huge downside. I hope that doesn't happen. But you know, Elon Musk somehow has had the ability to make big predictions. Some come true, some don't. He's still hugely respected as he should be. I think Sam's built up larger expectations. And just like Ilya's comment, I think really hurts OpenAI's set expectations. I worry that more of those promises driven by their need to grow may cause some downside in the technology side.

• 50:08Azeem Azhar?

  Either way, it's going to be another year of a lot of reading, a lot of rewriting, rethinking, redrafting. I mean, it's an amazing time for us to be in this space. Kai Feili, thank you so much for your time today.

• 50:21Speaker 01

  Yeah, thank you, Azim.