AI in 2025 – The great normalisation, with Nathan Benaich

Nathan Benaich, Founder and General Partner of Air Street Capital, joins me to discuss AI in 2025. From runaway consumer adoption to evolving enterprise moats, from still-elusive AI-driven drug breakthroughs to the renewed vigour in robotics, several core themes stood out.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Hi, happy holidays. While I'm away this holiday season, I wanted to drop here a few of my conversations with leading experts in AI. I discussed how AI might surprise us in 2025 with Ethan Mollick, Dylan Patel, Nathan Ben H and Caifu Lee. The conversations were initially released for members of Exponential View. Here's my discussion with Nathan Ben H. Enjoy.

• 0:22Azeem Azhar?

  Today, I'm super excited to have Nathan Ben H, who is the founding general partner of Air Street Capital. He is a man who is committed to backing AI entrepreneurs for, I would say, coming up to close to a decade now, and very, very well known for his annual report, The State of AI. And Nathan, it's great to see you. You're in a wonderful glamorous location today as well.

• 0:45Speaker 02

  Yeah, thanks for having me. It's gonna be a fun discussion. I mean, so much of what we've written in the report is probably due for update already.

• 0:51Azeem Azhar?

  Yes, well, I am, I'm giving a speech tomorrow and I'm thinking, how much do I have to update because of Google's last announcements, which happened 24 hours ago?

• 1:01Speaker 02

  Yeah, hopefully you're not making any like fancy graphs that you have to go back in there and like update everything with painful formatting.

• 1:08Azeem Azhar?

  Yeah, we've done a little bit of that. Now, I, I always grade my predictions. I do a horizon scan at the end of every year, and I know that you and your colleagues with the State of AI report also grade your prediction. And you know what? We both graded ourselves pretty well. So, almost too well. So are we being too unambitious with our predictions of the state of the AI world? Or is it just more predictable than we let on?

• 1:37Speaker 02

  Or we're just insiders, playing inside baseball. I don't know. I mean, there's certainly been predictions that we've made that have just not hit for several years. I mean, the main one has been around competition around AI semiconductors. And like, there's been so many, you know, new chip companies that have been started over, over the years. And I was originally excited about this as a prospect to compete with Nvidia, but it just looks like it's a losing battle over and over again. Which sort of prompted that slide we put in the report this year of trying to like stylize a portfolio of, you know, six billion dollars invested in all these chip companies versus six billion dollars in Nvidia at the same time points that the startups got the money.

• 2:19Speaker 02

  The delta would is basically like six billion in startups would transform into thirty one billion today versus two hundred billion if you would have put it in Nvidia.

• 2:27Azeem Azhar?

  It's in video.

• 2:28Speaker 02

  Yeah, it's great. So like there we got wrong, but I mean, some other stuff we, we got right. That was like pretty punchy.

• 2:35Azeem Azhar?

  Yeah, that semiconductor one is, is also one that I, I got wrong. I got really excited back in twenty fifteen and twenty sixteen with companies like Groth, Core, and then when Jonathan spun out of Google to set up Groth, had Mythic, Mike Henry's business, Cerebras. And I thought these companies are going to do really, really incredible things. I think really only Cerebras is in the game now with this big, single chip. And as you say, everything has flooded back to, to Nvidia, which I think is something we definitely got, got wrong. This has happened against the, this remarkable landscape of scale and adoption.

• 3:15Azeem Azhar?

  But when we look at consumer adoption, I wouldn't have guessed three hundred and five million weekly users for ChatGPT if you'd asked me at the start of this year, eighth busiest website in the world. And then we see this with these revenue numbers. Menlo Ventures just released their survey of enterprise spending on gen AI, generative AI. It was running at thirteen billion for twenty twenty four from about two and a half billion, three billion the year before that. And at the same time, Andreessen Horowitz are saying there's no money in foundation models. It's a race to the bottom. So when we look at this question of scale, speed and, and the business model.

• 3:56Azeem Azhar?

  Bundle together, which is the shape of the industry in a way. What do you read into what has happened in 2024? What do you think is emerging? And were you surprised with where we've ended up as we come to the end of the year?

• 4:09Speaker 02

  Yeah, I'm not gonna lie, like, I was surprised with the the revenue prints of OpenAI and Anthropic. Because I remember making these slides like a year or two years ago and OpenAI launched them. They said they're going to be billions of revenue. And I don't know, anybody predicts they can go from zero to billions like that quickly? It's hard to hard to put your your money against that. But like, they've done it. So I think I've been surprised with that. I've been also surprised with how open source has competed with closed models. So, you know, in some sense, I was like enthusiastic of, oh, there's going to be a big universe of open source models specialized for all sorts of things. And it makes pragmatic sense for customers and users to like not use the Ferrari for every task.

• 4:52Speaker 02

  But, you know, have a more specialized power for the job. To some degree, that might still be true in enterprise, but I'm not sure if consumers care.

• 5:03Speaker 02

  On the consumer side, it seems to really be like chat GPT equals AI. And that's just it. And in enterprise, for those who really care about privacy and self hosting, maybe they can do open source. But but at the same time, like the devil's argument here to me is if open source is so great and is really kind of growing in adoption, why is it that OpenAI and Anthropic and others are having their revenue grow so much?

• 5:29Azeem Azhar?

  Right, right.

• 5:30Speaker 02

  So then it sort of leads me to these analogies, which perhaps are helpful, hopefully. It's like, at the moment, I think, because because the market structure is so nascent, these products are so nascent. There's experimentation in both, in the same way that when smartphones were launched, you had iOS. And then at some point, you had Android, and Android was cool for quite a while because you could do funky things with it. You could fork it, change things, whatever. People like this open and permitted garden, but. As you kind of roll the years forward, it seems to me like iOS just increasingly crushes Android. And at least for like the segment of people I work with, even in or outside of technology, I have, I don't think I've added a phone number recently that's not an iPhone. In like the last year and I had a lot of phone converse based on the people that I'm meeting all the time.

• 6:16Speaker 02

  So I think that at the limit, users and businesses just want a service that's dependable. It's powerful and is, you know, updated whenever you think things happen and is that a good price?

• 6:28Azeem Azhar?

  I mean, it feels like it's much deeper than you don't get fired for buying IBM. It's deeper in the sense that you have an account manager, you have a roadmap that's visible. You have a developer community that is emerging, you have well-run developer days.

• 6:43Speaker 02

  Yeah.

• 6:44Azeem Azhar?

  And all of these things come together and if you're an enterprise, that is really beneficial for you.

• 6:49Speaker 02

  Yeah, yeah. And then I think you have a bit of this challenge if you're going to work on open sources, like which, which cloud is going to reliably update and serve these models?

• 7:00Speaker 02

  Because in one sense, okay, you have like Google, Microsoft and Amazon, but all those companies have their own internal large model divisions and probably biased towards serving. Towards serving those. I don't know. I can't remember if Google serves open source models, but they sure as hell don't promote it in Google Workspace.

• 7:17Azeem Azhar?

  Right, right.

• 7:19Speaker 02

  So. So as like a large enterprise buyer of Google services, I think you're just going to end up nudging towards their proprietary stuff. And then, then you're left with, you know, maybe players like Databricks or Snowflake that are sort of neutral grounds. Cause neither of them are really committing to building their own like large, large systems. So maybe they can be the preferred partner. To serve open source. But then they have different business incentives.

• 7:46Azeem Azhar?

  So you know, look, this feels a lot like the, the traditional enterprise. Cloud market, right? The decisions that people make, the buying is often, you know, we have a great MSA, master services agreement with. AWS, Amazon's cloud. And therefore it makes sense for us to use these models because they are just good enough. But I feel that underneath that, there are some. Real remarkable differences emerging. I mean, I've been surprised. By.

• 8:17Azeem Azhar?

  Two things in about two weeks that have really changed the way I already work. So of course, OpenAI released the Pro version of O1 at $200 a month. And then I started to move lots of my workflows onto O1 Pro. And then this morning at 8 a.m., one of my colleagues got me access to Gemini Deep Research. And I've now done half a day's work using Gemini Deep Research, which has been available for 36 hours. I mean, we'll dig into each of those, but there is something that is quite exciting and it feels like there is momentum in the core technologies beyond the sort of standard play out of of of plan.

• 8:58Speaker 02

  Yeah, I think because they're quite general purpose and therefore like the utility is quite diverse. They're less like fungible. It's not like you're buying a deterministic API that does authentication with text messages and it doesn't matter like what type of company you are, you just you need that and you just pay for volume and it's is what it is. Here, I think like this competitive advantage by creativity and what you won how you formulate your tasks.

• 9:27Azeem Azhar?

  Yeah, so let's let's look at those models from those those big companies. And I think the three that really stick to mind are Anthropic's Claude, OpenAI's, what is it? One model or two, right? GPT four and O1. And then what Google has done with Gemini, because each of them feel to me to be distinctly different qualitatively and quantitatively than GPT, which was the best that we had at the start of the year. Do you see it in a similar way that they are qualitatively different?

• 9:59Speaker 02

  They certainly have different product experience vibes and they're going for different things. I think, I mean, the the main the main sort of line in the sand seems to be at the moment like the coding abilities, where developers love Anthropic systems for coding. Now it looks like Google Service is going to rival it, but that's one thing. And then with OpenAI, you get voice and voice mode. You don't really have that in in other in other models. I still think the Google experience is terrible, like that I dev. I can't even remember what the URL is.

• 10:29Azeem Azhar?

  Visual Studio or something?

• 10:31Speaker 02

  Yeah, AI Studio, but then there's Vertex if you're an if you're a developer like, oh, sorry, in enterprise. It's just very confusing.

• 10:38Azeem Azhar?

  Right.

• 10:39Speaker 02

  Pick a domain and stick with it. So and then and then the other probably big difference is you're like anthropic systems are mostly consumed via API, or at least that's where most of the revenue comes from. And then OpenAI is more a consumer company.

• 10:53Azeem Azhar?

  Yeah, and we don't know what's happening with with the Google consumption there. So if we if we look at the voice mode.

• 11:01Speaker 02

  Yeah.

• 11:02Azeem Azhar?

  I don't really get voice mode on OpenAI. Is this something that you use? I mean, do you have a personal experience? I don't understand what to do with it. How do you use it?

• 11:13Speaker 02

  Yeah, I mean, for but but but really for like consumer use cases, like I'm in the kitchen or something and like, you know, stuff's going all over the place. I'm like, how do I do this? And it's so much better to like, listen to the agent, like, talk to talk to her than than write or read.

• 11:30Azeem Azhar?

  Well, personal use cases are are valuable. I mean, the iPhone is a, you know, multi hundred billion dollar business based on sending pictures of cats to strangers, right?

• 11:38Speaker 02

  Shit, shit, shit. I still haven't gotten me to convert to paying. I'm still on a free. Like, I think these things are just kind of fun to talk to, to be honest. They're, I mean, companion is the wrong term, because that sort of implies that you that you're like lonely or something. But I think just getting advice is pretty cool. Like, I was I was reading, another mutual investor like tweeted recently that they had this long discussion for hours with their colleagues about something and they fed the audio transcript in and then the agent like told them what the logical flaws were with different people in the room. That was like pretty sick.

• 12:10Azeem Azhar?

  It's gonna make us, isn't that gonna make us insufferable at Christmas parties? We'll have our little earpiece in. Wrong there.

• 12:15Speaker 02

  But it but it's also, yeah, true. But it's also cool to like, use this as a way of saying, you know, how would how would this person react if I said this? Or what's what's this person's vantage point without like offending them and asking, you know? So I think that's very powerful and interesting. Plus, like, I just love voice. I think it's super uncanny valley right now.

• 12:33Azeem Azhar?

  Oh, yeah. Yeah, yeah. It's really interesting. I am. I mean, I use the Claude voice mode, and and I will show you my my power trick here and for listeners as well. And with Claude voice mode, you press the button and you talk into it and you can talk for up to 10 minutes. So when I'm dropping my my daughter's Into school and I'm on the way back, I'll just hit voice mode and I'll say, hey Claude, it's me, it's Azim Azar, A Z E M for Exponential View. It doesn't have memory yet, set this up. I'm gonna, I'm gonna talk to you about what I'm thinking and I'm gonna, I want you to then give me notes for when I get back to my computer and I will sketch out an email I need to send. I will talk about a company, maybe I'm looking at investing in and I

• 13:14Azeem Azhar?

  want to be reminded about things I don't know. Um, I might have a note for my team and I'll just have this Ulysses like James Joyce stream of consciousness. Then when you get back, I get back to my computer, I will fire up Claude and, you know, I mean, the second after I've stopped talking, it's processed all of this. And I have these structured notes and I've made use of that, that ten minutes that would otherwise have been quite dead time. So that's been a real shift in my behavior. It's something that I now do every day, actually to the detriment of um various podcasts, which I used to listen to uh on my, on my drive back. That has been pretty surprising.

• 13:51Speaker 02

  That's sick. I, I just like how the Apple Store has like, you know, genius, geniuses and like little bootcamps to teach people how they could use like service. I think we really need that. It's like, unless you have the volition to really go and read it yourself or like learn it or come up with this as inspiration, like how does somebody even think about using this tool in that way?

• 14:12Azeem Azhar?

  Yeah, I, I agree with you. I mean, I felt, I felt really out of touch when about three months ago, I started to play around with one of these agentic workflow tools. This one's called, called Lindy and it requires a little bit more, not so much coding, but computational thinking than just talking into a text box. But it's really powerful. And one of the, one of the things that we built, uh, we use is we've built persona in Lindy of four of our different reader types and another persona who's an expert editor. And so we'll put essays through that and get them to argue until they believe that the thesis is really strong and distinctive.

• 14:52Azeem Azhar?

  And then the editor will sum up their recommendations and it'll, it'll come back to us. And something like that, I felt, wow, I need to spend four or five hours figuring out how to do this. Now, I'm quite lucky, like you, I can get hold of the founder. So I phoned up Flora, the founder. And I said, explain to me how I get this to work. And he did. And you're right. You know, we need, we need kind of geniuses to, to help us make, make sense of the, the tools. But then I wonder, like you've, you've invested in these founders quite a lot.

• 15:20Azeem Azhar?

  What extent do you look for the ones who have got that insight of how do we make this application actually really, really work and sing for that for the customer?

• 15:29Speaker 02

  It's tough, like, because I think the default is yes, you do. But when you look at the N of one or N of two, the creator's artifact, it was absolutely not. So I think the future is bright for a new AI products, because the ones that we have today are basically quite naive. Their models exposed as products. And I think recently, models are not products because people need help and they need guidance and they need some affordances in the product and the UI to get things done and prompts and things. And then B, there were all the products are designed by AI research people were like, you know, not any detrimental way, just generally not that much of like a user affinity compared to somebody who's been designing mobile apps their whole

• 16:11Speaker 02

  life. Those individuals who built consumer internet companies and e-commerce and all the rest of it, media have been flooding into AI. And so they bring these like vignettes, like the one you described, and can make beautiful product experiences out of it. So I think there's so much more that's gonna, that's gonna come as a result of the phenotypes of contributors that are moving into AI. But similar to you, I think, once you've experienced some workflows that really work, I don't think you can go back. Like, for example, before, when you wanted to generate a narration for text, you would read it and maybe you would use a recording app and then maybe like the next step was a speech to text and you could edit your audio file as words. And that was amazing unlocked, but it still takes you an hour for like a few thousand words and

• 16:56Speaker 02

  the ums and ahs and things. But once you have like a voice clone, I mean, this is game over. Like, why would you ever read a piece of text?

• 17:05Azeem Azhar?

  Are you using voice clones?

• 17:07Speaker 02

  Yeah.

• 17:08Azeem Azhar?

  Yeah. How do you use them?

• 17:10Speaker 02

  Well, any piece of text I write, I think it's like worth generating audio for. I take 10 minutes and I generate the audio for it. And then I shove it in the in the same medium that we share the for the text. And you know, if people want to consume it, they do. I even tried to do like discussions, like for example, if we did this one over writing, you have it's like written interviews. You can have my voice clone and your voice clone, and we just go mine, yours, mine, yours.

• 17:36Azeem Azhar?

  So one of the things that you you said in your, I think this is, this was, yeah, I read it on one of your essays. Was that you thought that the consumer market around AI has incredible potential over the next year. Could you explain why you think that is?

• 17:51Speaker 02

  Well, I think I think the most exciting technologies are ones that unlock new user experiences and new product experiences that you couldn't do before. And I think in enterprise, we're still stuck in this mindset of there's a lot of unstructured data. It needs to be structured, it needs to be summarized, you need search, and you need workflow automation. This theme has generally been the case for like a decade, basically. And yeah, the capabilities are getting better, but everybody knows what the problems are. Whereas I think in in consumer land, you have these

• 18:23Speaker 02

  sort of uncanny valleys that once crossed unlock entirely new use cases. And it's not like

• 18:28Azeem Azhar?

  So just explain the uncanny, just explain the uncanny valley for for people who might not be familiar with it.

• 18:34Speaker 02

  I mean, the way I the way I interpret it, maybe it's not the textbook definition. It's like you have some text, you have some tech capability, for example, voice cloning, which is not new. You know, it first kind of came out in 2017, 18. There was, you know, a business called Lyrebird in Canada that was allowing you to clone your voice. And they had this like Obama voice clone. It still sounded a bit robotic. It sounded like him, but a bit robotic. It's not good enough that that's going to represent you in your

• 19:01Speaker 02

  business persona, you know, reading your newsletter or like giving a talk or something. So it's good, but not good enough. But then, once you get like the quality of audio cloning, that if you were to share it, 98% of people would think it's you. And maybe there's one or two things around the edges. That is good enough that it's basically indistinguishable and therefore it is now usable and I can deploy it.

• 19:22Azeem Azhar?

  So coming back to why you thought consumers were going to work, you were saying that your voice cloning was one of the things that was getting across the uncanny valley. But just let's understand a bit more what the where you see these opportunities playing out.

• 19:36Speaker 02

  Well, I think then people realize that there's for voice, there's a lot of scenarios where you are speaking things that you already wrote, or it's better to write them, but you maybe want a bigger audience that can consume it in terms of languages and that's painful. Or the post-production of changing something that you've recorded is pretty excruciating.

• 19:57Azeem Azhar?

  Right.

• 19:58Speaker 02

  Video and like I record something, I make a mess up and then now it's dark and it was light before, like that's game over. Or as uh, you know, now you have this this capability of like, oh, I said something wrong. I just go into the transcript, rewrite it and generate it and it's it's job done.

• 20:13Azeem Azhar?

  I'm quite excited about about the prospect of things that are simply too hard to do economically without without AI. And that lends itself very much to the the consumer market. And some of the things that we all, small business market, some of the things that we now do, and I'm sure the same is true in air streets, uh, are exceptional notes for every meeting, like a conversation with my insurance broker about my home insurance. I get great notes. And you would never have invested in doing that previously. And I I think that does change our experience as individuals of the world because we have got that layer of quality.

• 20:52Speaker 02

  Yeah. And I and I think it's gonna expand a bit further. Like, for example, we have a company that's building like an AI secretary over the phone for small businesses. And there I think it's quite cool because, you know, frequently you're calling your service provider, whether you're doing like home remodeling, or it's your lawyer or your electrician or whatever, they're not there and you get the voicemail and like, please call me back, whatever. And now it's like an AI system can talk to you, multilingual, resolve your query, probably pass your message to the individual. But then the cool thing I think is you get accountability because you could get the transcript back with the key points. You could maybe even like get text messages saying, hey, like the electrician said you should call him back in forty eight hours. And it texts you in forty eight hours saying, hey, it's been forty eight hours. You should call back.

• 21:32Speaker 02

  So I I think people are gonna gonna really like this stuff because, yeah, more transparency, more accountability, they get the information they need. And I think people's like reaction to these systems are getting better because they realize they're not dumb anymore. Like it's not.

• 21:46Azeem Azhar?

  Right.

• 21:47Speaker 02

  Like in so many cases, I would prefer customer service, frontline customer service to be like an AI agent or an L M or whatever, because it's way better. And then if I actually need to talk to a person, then a person can get on the phone and not behave like a robot, which is what they currently do.

• 22:05Azeem Azhar?

  Which is what they currently do. The other problem you have is that people are rushing these agents out and they're not, they're not good enough. And so I've got a couple of consumer products. Because I'm sufficiently frustrated with them, I wouldn't normally do this, but I'm going to name them. Ura, the Life Ring and SoundCloud, the music service which have replaced for their premium customers. Any support ticketing with a bot, neither of which work.

• 22:30Speaker 02

  Yeah.

• 22:30Azeem Azhar?

  At all. In fact, Ura was so bad, I had to get a friend who could get his chatbot or Ura support to work to put my support query in for me and he's now handling it, which is a weird delegation, but it was the only way through. But I, I think that there is, um, there is this sense actually that as these products get better, the customer service can improve and what you can then do is free the human app, as you say,

• 22:55Speaker 02

  Yeah.

• 22:55Azeem Azhar?

  to have much, much more discretion. And I, I wonder, I mean, do you look into that space? Because within customer service, the reason people don't have discretion is because they are working through decision trees. And it's a principal agent, agent problem. They're given no freedom to go left or right. Could you imagine that with, when you support an agent like that with an L M, it actually gives them much more discretion in how they could behave? I mean, is that, is that possible, given what you know of C S technologies?

• 23:24Speaker 02

  I think, uh, I think the benefit is like the, the language model just has access to a massive corpus very, very quickly. And therefore the human doesn't need to somehow realize or like figure out how like some weird thing on your phone works to go fix it. And, and then also you don't have the multilingual problem. I think that's quite big. You can potentially handle a bunch of people at the same time.

• 23:46Azeem Azhar?

  Mm hmm.

• 23:48Speaker 02

  But I think because because language models are quite good at outpouring like bullet bullets and stepwise plans, this like aligns quite well with customer service, which usually involves some kind of stepwise plan. And did you try that? Did you try that? If you didn't do that, try this.

• 24:01Azeem Azhar?

  But it could be more dynamic. So let's turn this into a 2025 prediction. What do we think? What do you think we will see with consumer meets AI for 2025 then that we haven't seen getting any real traction or success this year?

• 24:17Speaker 02

  I still want to see these like, uh, real time speech to speech translation. Like where we're doing this, but I'm talking in French and you're talking in English or something. And then it's like instant latency, like it's with with zero latency translation. That would be so epic.

• 24:35Azeem Azhar?

  And that would be, I mean, and do you think that that's going to be something that we could get in 2025? Or there's still technical hurdles to get?

• 24:41Speaker 02

  I think we can get there. Yeah. For example, OpenAI recently hired the guy who who did WebRTC.

• 24:49Azeem Azhar?

  Right.

• 24:50Speaker 02

  To do.

• 24:50Azeem Azhar?

  Which was the foundation of Google Talk, Google Talk Talks, the voice back in the day 15, 20 years ago.

• 24:55Speaker 02

  Yeah.

• 24:57Azeem Azhar?

  Yeah. Yeah. Yeah. Okay. So they're, they're, they're thinking about that. Um, they're thinking about that, that issue.

• 25:02Speaker 02

  What? I think so.

• 25:03Azeem Azhar?

  I'll give you one of my, my thoughts on the consumer app. So there's a lot of noise around consumer companions. So Mustafa Solomon from Microsoft Inflection, DeepMind X has talked about them. And the new Google, uh, L L M's have very large context windows, a million tokens, which is, you know, 800,000 words, roughly 900,000 words. A human speaks about a million words a year. So you have now got the capabilities to run something that is, is recording your, your life in some passive way. So I, I've seen a few of these things on Kickstarter, their little devices tendons that you wear and, you know, they record your life. But I, I think that you might start to see people

• 25:48Azeem Azhar?

  figuring out some kind of product like that from the ground up. Maybe it's, it is, it is what we used to use Yahoo for back in the mid 90s and what we use our home screen for today, which is the way into the services that we, we, we need. I mean, I think we might see something like that come out in 25.

• 26:07Speaker 02

  I think relatedly, there could be thinking there could be something cool around like a personalized like generative autobiography.

• 26:14Azeem Azhar?

  Right.

• 26:16Speaker 02

  Cause like you could, you know, you're speaking all these thoughts into like some model or you certainly could do that. But but based on all the artifacts that you leave online and things that you could tell it every day, you know, I can figure out gaps and stuff. And then, you know, over, over the next few years, it can write like the story of your life. And you know, guess what? It could probably predict like, I think it's gonna go in the future if you continue on the same course.

• 26:37Azeem Azhar?

  I find that really compelling. I find it compelling as an idea because it sounds so kind of outlandish and slightly ridiculous. But so too is I'm going to take 500 photos a day and share photos of doorknobs, cats and cheese to my friends.

• 26:52Speaker 02

  Yeah.

• 26:52Azeem Azhar?

  And yet I can, I can see that moment where people can have that sense of self-reflection because it's, it's easy to do. It can be presented in lots of different ways. And as you say, it could start to deliver.

• 27:04Speaker 02

  Yeah.

• 27:05Azeem Azhar?

  Predictions. It's a cool idea. Are you, are you looking for investment for that?

• 27:09Speaker 02

  I might just end up having to incubate it because I pitched it a bunch and no one wants to do it. But like relatedly, you kind of have this already for kids, like they have these, you know, here's like your two-year-old kid book or your four-year-old kid book with like pictures of your life and things that happened and whatnot. So, I mean, I'm not saying you should have like an L M for a four-year-old, but like, it's kind of cool if, you know, they wake up and like when they're six or something and they're a bit more human, like they can talk to a system and it can tell them about their early life. Like I would love to know some of these things. And you only learn these little artifacts through random conversations with your parents or picking up the picture books of the holidays or something.

• 27:47Azeem Azhar?

  The people who are not as excited about these prospects as you and I are, it's worth hopping in that there are obviously lots of edge cases where there are bad, bad counter examples, the kind of the, the, the kid. I think it was with character AI where there was a, a, you know, horrible situation and what was happening with Replit. But there are also some good examples. There's a, um, a robot toy called Moxie, which is, has been designed by a combination of child psychologists and sort of. Idi educationalists as a, an emotional companion for kids who have autism spectrum disorder and is sort of kind of present and patient the whole time, actually before the L L M wave really, I saw it three years ago.

• 28:27Azeem Azhar?

  And I, you could see they had pretty good results from their, that some evidencing of its positive impact on child's well beings. There are, there are anchor points which says, hey, this can really work. What will happen, of course, founders will just rush out there and figure out what drives engagement and, you know, what drives you.

• 28:43Speaker 02

  Yeah.

• 28:44Azeem Azhar?

  Maybe something will emerge.

• 28:45Speaker 02

  Yeah. And then everything I said is probably all wrong.

• 28:48Azeem Azhar?

  Everything else is wrong. Yeah. Yeah. Um. Okay, great. So those are, that's consumers. Let's flip to some other spots. I am curious about, you know, you are a scientist by, by training, you have PhD in, uh, you know, biology and, and, and, and machine learning. And so much promise about science and, and AI. I mean, Dennis and, and his collaborators got the Nobel Prize for the work on, on Alpha Fold. And every day there's a new thing. Hey, we've accelerated materials discovery. Hey, we are doing something in multiomics predictions and, and so on. Has there been any beat delivered from all of this wonderful work?

• 29:26Speaker 02

  Hitting me with the tough questions. I think yes, but like the, the problem is like the bar is just so damn high. At least in biopharma, like until, until we see a drug that can kind of unanimously be claimed to have not existed without some advanced AI technique become approved and really work and ideally be a blockbuster. I think you'll still always have the Debbie Downers, basically, um, that'll poo poo this space. And I think it's maybe because like the narrative has been framed as like AI is this magic thing for drug discovery. And, you know, these 10 year development cycles will be gonna, gonna become far shorter. We won't have to spend billions of dollars, et cetera. Like maybe that was too much of a grandiose like framing.

• 30:12Speaker 02

  Because I think similar to our discussion about how, you know, once you discover this workflow that can be better solved with AI, you're kind of not going back. I think the same is true in biology and just empirical science. Because when I look at 10 years ago or so, 11 years ago, and I was doing my PhD, you know, if you were, for example, taking pictures of cells with a microscope, and then we wanted to like enumerate the number of cells that like glowed or whatever are expressed in certain protein, we'd still have to take the image and then basically go click, click, click, click, click. Like, there was no automated computer vision software that I could just show this to you and it would just be done. And that's like pretty nuts. Like now this is trivial problem. And why I think this stuff is important and actually quite impactful is because

• 30:59Speaker 02

  like the enumeration of experimental data in science is such a big scale problem that right now, a lot of humans are doing the analysis for and humans make mistakes and reproducibility is a real issue. That's probably one of the biggest issues plaguing most of science, particularly in biomedicine. Most of the biopharma companies or biotech companies certainly are spun out of academic research, which has largely been done by humans, which are doing work that is not reproducible and then, you know, go figure why a lot of biotechs don't work out.

• 31:31Azeem Azhar?

  Right.

• 31:31Speaker 02

  So I think if you start like automating experimentation, automating data analysis, having it be far more robust, I think you can make quite a big dent into the irreproducibility of science. And I don't think you can go back. Like, why would you ever go back to going click, click, click?

• 31:47Azeem Azhar?

  Yeah, absolutely. And, you know, I have spoken to a number of scientists and read a lot of papers that have come out, even, you know, peer reviewed journals. And there is, I get a sense that once people start to move their labs onto these tools, they don't really, they don't really come back from them in the same way that once you start recording something in a Jupiter notebook or you do your analysis in, even in Excel, frankly, or SPSS, you know, you don't go back.

• 32:14Speaker 02

  Yeah.

• 32:15Azeem Azhar?

  And it just, it takes a bit of time.

• 32:17Speaker 02

  Yeah, the other fun one I heard was uh was around grant writing. So I remember there was like this study in Nature, I think it was like Nature reviews or something 10 years ago or 15 years ago. And it was basically like the academic grant writing process is completely broken because the majority of applicants spend more time writing the grant, which overall costs more money than the amount of money they're applying for because of reviews and just how bloody slow the whole thing is. I have a friend who's in still in academic science, but he's using ChatGPT to help brainstorm grants with him.

• 32:50Azeem Azhar?

  Right.

• 32:51Speaker 02

  So he's like, yeah, this idea like gene therapy, and I think I could take this approach by, I don't know, like considering this pathway and this this way that a molecule could bind and, you know, what's the prior art? What do you think? Like, do you think this could work? Blah, blah, blah. And then like half a day, the guy is like a nice grant that he can go apply.

• 33:09Azeem Azhar?

  Okay, so that is acceleration of science, right? Because he's spending 90% less time grant writing, he's getting the grant earlier, and he is uh able to get working faster. So that's that's already a return.

• 33:23Speaker 02

  Yeah, yeah, yeah, yeah.

• 33:24Azeem Azhar?

  I think, what what do you think? What might we see? What would need to happen in 2025? Is it a is it a is it a technical breakthrough, a model breakthrough? Is it a company's products just crossing the chasm that would say, oh, wow, this AI science thing is now turning from, you know, promise to real evidence?

• 33:48Speaker 02

  I think for bio, you'd need to have a really good readout, which readout being industry parlance for like clinical performance. Probably like phase two or, I mean, phase three is not going to happen based on the drugs that are in pipeline, but you need to have a really good readout that people like, whoa. And and that this molecule is like substantially designed with AI.

• 34:09Azeem Azhar?

  So we we have to just be patient, I think, is the answer there, because that process just takes the time it takes, right?

• 34:15Speaker 02

  Yeah, yeah. I mean, the problem is like, you know, the kind of OG companies here have been at this for 10 years, you know, that's recursion and exientia, and they are making good progress, but. It just takes a lot of time because also I think you know a lot of the AI companies in bio have been focusing on the early discovery like I don't know hey we have like a billion molecules in the universe. How do we sift through them and find which one binds to this protein that causes a disease? Far more than they've been focusing on the clinical side of things. So in a way, we have like this engine to generate lots of molecules, but you still have to feed them people and run human trials. And yeah, even that is like not very, not nearly as advanced as what it could be in terms of like, who's the right patient to feed this drug to?

• 34:58Speaker 02

  Where are they? What do I measure when they are actually in the clinic?

• 35:03Azeem Azhar?

  Right. And there's even a phase before that. There's the process of synthesizing at the first time and then figuring out whether, how you can deliver it is what you've synthesized soluble. If it isn't, you have to go back through that process again. It's another thirty days and hundred thousand dollars each time. So there are lots of places where you could squeeze out some time once you get these tools working, but it won't be a single model to end all other models. Okay, we have about ten fifteen minutes left. I want to kind of whip through some other thoughts for you. So what about one area where I have, I hugely revised my view of the world on self-driving cars in July this year.

• 35:42Azeem Azhar?

  I looked at the data, both coming out of Wuhan and coming out of San Francisco and wherever else Waymo was operating. And the disengagement trends were significantly improved and actual customers using this regularly was going higher and higher up. I know the chairman of Apollo, which runs Wuhan, I spoke with him and got a sense of the fact that they're very, very close to, or if not, have already passed this break even point there. So I changed my view. I then sat in a Waymo in October and in San Francisco, and then I was just Waymo-ing my way around the city and, you know, loving it. It's an incredible experience, which I'm sure you've done as well.

• 36:21Azeem Azhar?

  So that then got me to think about humanoid robots, which I also had thought of as being just a little bit technology seeking an application. But I've started to shift on that and next year I will likely shift further once I've done more thinking. How do you interpret the robotic space and where it might go in twenty twenty five?

• 36:41Speaker 02

  It's been amazing to see the vibe shift in robotics because for the last decade, it was where you went to burn a lot of money and get nowhere.

• 36:49Azeem Azhar?

  Right.

• 36:50Speaker 02

  There were, there were some businesses that got to scale like AutoStore, Cardex, Berkshire, that is back and then tanked and a few others, but it's been really hard. And I think generally speaking, the mobile robotics space and manipulation space was gate kept by the fact that models were not robust enough to be able to adapt to all these customer sites and you had to like build basically a custom system for a customer. But like, I think coming out of COVID and then be all this like general purpose AI system like really changed that because now you can have basically a single system. Now nobody wants to work in facilities and you had like a decade worth of companies throwing money at trying to tell businesses that they need robots and now they kind of believe it. So it's, it's magical to see this like vibe shift.

• 37:34Speaker 02

  Now there's this renaissance of robotics, which is really only happened last like year during half of those. So I think it's going to go, I don't know, fast, but it's going to go faster in warehousing, logistics, manufact, probably more warehousing and logistics because those segments kind of serve online commerce and this big pressure there. Manufacturing, I think maybe is a bit more challenging, but then as it comes to humanoid, I think I probably have to think a bit more about it, but at the moment I feel like it's going to be self-driving cars. Like it's going to take a while because I keep hearing, you know, that there's a lot of teleops, a lot of like demonstration where it's not robust. Many of the cases that are shown are better done with just norm, like quote unquote, normal robots.

• 38:18Speaker 02

  And while I'm sympathetic to the argument that the world is designed around humans and therefore a humanoid factor makes the most sense, I do feel like quite a lot of the economically useful tasks don't really need a humanoid form factor.

• 38:31Azeem Azhar?

  Yeah. I'm not thinking about humanoid robots in in industry.

• 38:36Speaker 02

  Okay.

• 38:37Azeem Azhar?

  Think that Ocado Warehouse, Ocado is a big online grocer in the UK. The warehouse is essentially a pixel system and these little robots on tracks run around and and get pack what they need. You know, you you'll do much and JD dot com's warehoused in China, you know, similarly. You don't need to, you know, weaken the capabilities of the machines by fitting them in a human world. But actually, where I was slightly persuaded of humanoid robots, what what would the idea of them being a luxury product initially in in homes or maybe in offices where there there are there still are people? I was actually persuaded by one of my readers who said, look, I bet you spend like me five hours a week loading and unloading a dishwasher.

• 39:19Azeem Azhar?

  Right? How much would you pay for a robot to just do that overnight while you're you're asleep? And I'm thinking, you know what? I'd probably pay something for that, you know, maybe not 10,000 bucks that the unitary Chinese humanoid robot goes for. But if it was half that price, you know, maybe I'd be thinking this is five hours of my week that is worth coming back. And I can imagine there being some people. So I've gone from completely not believing that people would want them in their home to starting to think, hmm, maybe they would, because maybe that would solve the dishwasher thing.

• 39:50Speaker 02

  Yeah, but it builds character.

• 39:51Azeem Azhar?

  It builds character my kids who are not doing it at nine p.m. at night. I mean, to be fair, they do it on the weekends, but yeah, I I'm I'm generally the loader.

• 39:58Speaker 02

  Yeah.

• 39:59Azeem Azhar?

  Let's switch to a sort of broader, the last few minutes and a prediction source for the next year. And and and I have just sort of three sets of questions here. So set number one really is, what are your top three predictions for the AI space for for next year? We've covered a lot of ground. You can go, we like chips, China.

• 40:18Speaker 02

  Yeah.

• 40:19Azeem Azhar?

  Foundation models, reasoning, AGI, risk, wherever you like.

• 40:24Speaker 02

  I'm I'm quite excited about these like fully generative world games.

• 40:31Azeem Azhar?

  Right.

• 40:31Speaker 02

  I think there's gonna be a pretty cool video game that's based around interacting with Gen AI elements, like the whole thing is generative. And you can see where this is heading with Genie two, this works from Google Deep Mind, where you can kind of throw a picture of an environment and then the thing will just generate.

• 40:45Azeem Azhar?

  Explore it for infinity.

• 40:47Speaker 02

  Yeah, it'll just kind of keep generating and it and it has like some intuitions around like game worlds and levels and characters that you can move around. I think this is pretty epic. I'd love to see some like blockbuster game that happens there. I think they'll similarly, there'll probably be like an app or website that's created by somebody with no coding ability that actually goes viral and like hits maybe like the Apple Store top one hundred.

• 41:10Azeem Azhar?

  What's your third prediction?

• 41:12Speaker 02

  I think, I mean, I feel like there's gonna be some like national security implications, or there is like a line that, I mean, governments are gonna have to take with regards to these like big labs. And I mean, there's not really any other country than like the US that has these big labs, where it's like, okay, the next step is we're gonna have to raise, I don't know, tens of billions or hundreds of billions. It's probably gonna be from some nation states or sovereign funds. And like, is that, is that okay? Is that a line we want to take? Or do we want like the US government to fund it as it relates to like critical, yeah, critical assets?

• 41:44Azeem Azhar?

  And they're critical assets because we don't want the weight and the know how to be exfiltrated? Or is there some other thing that makes them critical and the required that requires that protection?

• 41:55Speaker 02

  I mean, I think realistically, you know, even if you have a foreign government that wires $10 billion to a big lab, like if they have no special rights, how are they gonna, how is that $10 billion gonna give them any ability to extract information other than soft power? So I do think it's a bit of a, a soft power vibes positioning kind of thing.

• 42:15Azeem Azhar?

  Right.

• 42:15Speaker 02

  Where, yeah, if that, if the company just needs, is like on a, on a lifeline for massive sums of money, and then the only player that's willing to front the bill is from a nation state that like is now adversarial, like what are you gonna do?

• 42:28Azeem Azhar?

  Yeah, okay, that's great. So let's go to the second section. Let's try to understand what you think the most determinant thing is that will impact the whole AI ecosystem. And we'll play this in a game, which is if you can ask the Nathan of December twenty twenty five, one single thing, and that version of future you can only send you back a short message. What would you ask them to best understand what will happen in twenty five?

• 42:52Speaker 02

  That at the end of the day, like if you're not a frontier model, then nothing else matters.

• 42:59Speaker 02

  But yeah, I think that's the biggest question, you know, I think it's heading that way. Like I originally thought it wasn't, but it sort of feels like it is.

• 43:07Azeem Azhar?

  Yeah, yeah, the sense that it's the big frontier models and they'll just expand and construct their capability. Okay, that's a great one. And then the the the final question, which is, when you look out to twenty twenty five and we think about both the growth in capability that we've seen in twenty four and the growth in usage, consumer and enterprise. Will that rate be faster, slower, or the same for the next year?

• 43:34Speaker 02

  Gonna say same because I can't choose. The reason for that is, um. Like, we sort of know what a ten thousand GPU cluster gave us. It gave us like GPT four O Omni, etc. We don't yet know what a hundred thousand GPU cluster has given us. I think we debated this briefly on Twitter.

• 43:50Azeem Azhar?

  Yeah.

• 43:51Speaker 02

  So let's see. But then I also think that there there is so much more to run. However, I don't know what I have to see next to be in this like holy shit moment again, like we've

• 44:02Azeem Azhar?

  Right.

• 44:03Speaker 02

  We've we've normalized so many of these capabilities, which is probably a good thing, like society has to like adjust and adapt and like figure this thing out. So that's probably good. But at the same time, like the bar is so high now, like, oh, cool. You have a better voice model. You have a better thing. Yeah, I've seen that already. Who cares?

• 44:20Azeem Azhar?

  Right.

• 44:20Speaker 02

  Whereas I think when when you and I were like, I mean, you you were kind of earlier than me, like with your startup and machine learning, but like, jeez Christ, I mean, if if if you back then saw what you had today, like, you wouldn't believe it.

• 44:32Azeem Azhar?

  I wouldn't believe it. I wouldn't. I sometimes I barely believe what I have today today.

• 44:35Speaker 02

  Yeah.

• 44:35Azeem Azhar?

  I'm sitting there using it. Yeah.

• 44:38Speaker 02

  It's literally magic.

• 44:39Azeem Azhar?

  It is magic. I I and I I agree with you. I think it's very difficult. Once you have, and as we've seen, there's one benchmark, which is PhD level science benchmarks. And in eighteen months, the best models have been have gone from as good as flipping a coin random to better than a typical PhD human expert. And that's in in fifteen months. Once you're at that level, I'm not actually qualified to judge whether those models are the future models are better than the previous ones, because I don't have that expertise. So so I I think that it gets harder to to judge. And my prediction on this question will be that the fact that it's harder to judge and the benchmarks won't move in as big lumps will create a little bit of negative media media pressure. But I think that usage will grow faster

• 45:25Azeem Azhar?

  next year than it did this year, because for exactly the reason you said, we are starting to normalize and come up with a language around this. And I'm more likely to be surprised to the upside of my predictions than than to the downside. So in other words, I think twenty five is going to be a pretty exciting year. And I slightly fear for you and your colleagues producing the next state of AI report. And I do hope you enlist some AI to support you writing it.

• 45:53Speaker 02

  We tried. We tried this year. It turns out it was it was challenging to get them through reason through papers. But yeah. If more PhDs get paid for data annotations, probably to our benefit.

• 46:04Azeem Azhar?

  Nathan, thank you so much for taking the time

• 46:06Speaker 02

  Thanks, Azim.

• 46:06Azeem Azhar?

  to join me today. Wonderful. And happy new year to you.

• 46:09Speaker 02

  Thanks, you too.