The difference between early and late AI adopters

Physicist and entrepreneur Steve Hsu, whose startup Superfocus tackles hallucination problems in large language models, joins Azeem to discuss AI agents, hallucination challenges and what happens when technology meets labor markets.

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  We have agents that are capable of replacing, you know, something like eighty percent, maybe ninety percent of the calls that come into a call center.

• 0:07Azeem Azhar?

  So what is it like when you go out there and you show this technology, particularly to those frontline operatives?

• 0:14Speaker 00

  I think they feel about it the way that someone who was a blacksmith or a buggy whip maker felt when they saw their first automobiles rolling down the street, right? They could tell something was happening and they don't like it, but what else can they do?

• 0:27Azeem Azhar?

  I think every person who's developing software knows that getting to the first seventy percent is trivially easy. It's the climb up the last thirty percent.

• 0:34Speaker 00

  You don't just want to turn this thing on and then discover like, oh, overnight it got into some bad loop and pissed off a hundred thousand customers, right? One of the things that I think the general public doesn't understand is like how much rigorous statistical testing is required to know what are the tail risks associated with a deployment of autonomy? That could be automated customer service, that could be a driverless vehicle.

• 0:56Azeem Azhar?

  Let's talk about China. There is this tournament-style competition between mayors of cities. They act a little bit like, you know, mini Y Combinator bosses or mini venture capitalists.

• 1:06Speaker 00

  Combined with the people, there are just more pro-technology. You say like, oh, we're gonna have AI at the hospital. The average Chinese person is not thinking like, oh, what about my privacy or what if the AI makes a mistake? The Chinese people are more like, this is awesome, man.

• 1:19Azeem Azhar?

  In a future that is full of AI agents, what's something that you might end up doing more of?

• 1:25Speaker 00

  Again, I'm gonna tell you something incredibly crazy. You can break the news here.

• 1:31Azeem Azhar?

  Today, we are joined by physicist and entrepreneur Steve Shu. The most loyal exponential view listeners may remember, he was on my podcast back in two thousand and nineteen. Feels like a century ago, so much has happened. We talked about biotechnology and the promise of genomic sequencing. And I actually had to go and buy a maths book after that podcast to make sense of some of the answers. So today, we're talking about something else, AI agents and what might come next. Steve, it's great to have you here. I want to start with what you're building. You've co-founded a company called Superfocus, which you've described as solving some of the key limitations in today's LLMs.

• 2:14Azeem Azhar?

  Give us a quick tour. What are you building and what is the deeper shift that it represents?

• 2:19Speaker 00

  So I think everyone who's experimented with the large language models knows that they hallucinate. So they will sometimes give you a very confident but completely wrong answer. And that answer will always be typical of the kinds of things that it saw in its pre-training. So if you ask an LLM, is there a flight that reaches Paris from London that lands around four o'clock? It will give you an answer, but it might not be real. It will seem like a real answer, like, oh, Air France has one that lands at four twelve, right? And maybe in the past it did, but maybe right now it doesn't, right? Typical large language model hallucination. And so our startup realized early on that in order to make practical applications from these AIs, one would have to solve this

• 3:04Speaker 00

  problem. One would have to control both the behavior of the AI agent, if you want to call it an agent, and also control its fact base, the sort of core knowledge base that it uses to answer questions or conduct operations. And that cannot be drawn from the pre-training data. There's just too much junk in the pre-training data. Or even if it's not junk, it may not be relevant to the specific problem that the AI agent is trying to solve for you.

• 3:31Azeem Azhar?

  Right. So you're tackling that fundamental limitation in the way that LLMs are built. I've had that experience. The way I think about this is that they are often accurate at a conceptual level, but not necessarily accurate at the word or token level. So they'll sometimes say of me that I went to Cambridge University, which if you're an alien sitting on Neptune, I went to Oxford. Well, Cambridge is kind of the same thing. But they don't ever say, I went to West Point and I did SEAL training. I mean, I've never had as wild a hallucination as that. And that really is about the way in which they represent the world in this complex, high dimensional space, isn't it?

• 4:15Speaker 00

  Yes, so there is something called the embedding space. The models are actually working in this abstract space, which is literally a space of concepts. Oxford and Cambridge are very, very close in that concept. And maybe there's no other school that's exactly in that space. But you know, sometimes the details matter. So if I'm a fundraiser for Cambridge, I actually care whether you went to Cambridge or Oxford in deciding whether to contact you, right?

• 4:39Azeem Azhar?

  Right.

• 4:40Speaker 00

  Yeah.

• 4:41Azeem Azhar?

  Yeah, so the details matter, but in some sense, they've still got a useful model of the world that perhaps loses some of its resolution if we push really, really hard for it to be precise. So how do you go about resolving that?

• 4:57Speaker 00

  Right, so we actually build systems in which we embed the language models in a larger software platform. And the language model itself, we generally are using mainly only for its language and to some extent reasoning abilities. But the knowledge base is stored separately. The fancy way we describe it is as a kind of attached memory for the AI. The AI can rely on that attached memory. And the way we program, we use sort of old style programming in this platform. We sort of force the model to only use that knowledge base in answering the fact part of questions. And so that that extra constraint, it solves the hallucination problem

• 5:42Speaker 00

  and makes both the behavior and the knowledge base of the AI reliable.

• 5:47Azeem Azhar?

  How different is that to a traditional RAG retrieval mentored generation type of approach?

• 5:53Speaker 00

  So if you look at the architecture, it is actually, a piece of it is RAG. And interestingly, we actually, when we founded the startup, we actually filed a patent. The company filed a patent on our architecture. And that was actually before the word RAG was in wide usage. So it is possible, who knows how the USPTO Patent and Trademark Office operates, but we might be issued a patent on RAG. Of course, it's not referred to RAG in the patent filing. It has a lot of similarity. Another thing you might do is you might have multiple models involved in the generation of the response, in which some models are just error checking. The proposed response of the big model against what the little models can see in the knowledge base.

• 6:37Speaker 00

  And all that, if you're doing voice, which we do, all that has to happen in a latency time of less than two seconds. So humans, if I stop speaking and I'm waiting for you to respond to me, if it goes more than a couple of seconds, it's kind of strange. And so all of the stuff that I just described to you is engineered down so that the latency is between one and two seconds, so it sounds natural.

• 6:59Azeem Azhar?

  One of the things that I'm curious about is whether the improvement in models really starts to tackle the hallucination question. So I've been using Gemini Pro 2.5 or Gemini 2.5 Pro for the last couple of weeks, both through the app and via the API and some applications I built. And I've been really, really impressed with its ability to be factual, be precise and hold that over quite long context conversations. That was often a problem with LLM models, which was they were very accurate for the first 500 words. And by the time you got to 5,000 or 10 minutes into your discussion, they're just going all over the place.

• 7:41Azeem Azhar?

  And I found that wasn't the case with Gemini 2.5 Pro. So in a sense, isn't a lot of this going to be solved by the investments in bigger and bigger models that can model more precisely this concept space? And that's the direction that the Googles and the OpenAI is going?

• 8:01Speaker 00

  Well, it is improved. The situation is improved if you're using a model that has reasoning capabilities. What's going on in the reasoning is the model has been taught as it, as it sort of talks to itself in trying to solve, generate a good response to your query. It has been taught to double check facts or components of the reasoning. However, if the model doesn't really have access to the actual ground truth, it can still go off the rails because it can, it can think X is true. It doesn't have a way to independently validate X and it can then use X in its reasoning pattern, even though it knows it's supposed to check X, it doesn't really have a way to falsify X. The other problem with reasoning models for the application that we're specifically

• 8:46Speaker 00

  addressing in our market segment is people want for voice, again, fast response. So if you, if you rely on a whole bunch of tokens being generated for reasoning, that's way over the, you know, fraction of a second that you have for that generation's, you know, the, the speech to text and text to speech, all that stuff is wrapped in there. And then the reasoning has to be, you know, even smaller than a second, like a fraction of a second. So, so you would need a reasoning model that generates all of its reasoning tokens that fast and then you might be able to use it. Yeah.

• 9:17Azeem Azhar?

  I, I mean, when I, we talk at this level of detail, of course, it, it puts into con- into context what this path to super capable AI really is because you're working in a very constrained environment where not only do you have to, you know, take the sound wave, you've got to break it into phonemes, you have to put it into a voice model, you then have to do, turn that into text, you then have to reason across it and you've got a really limited amount of time to, to do all of that. You know, it's not like O1 Pro going away to think for, for, for 10 minutes. And this is sort of the reality of how these products are going to be built over the next few years. With, with Super Focus, I saw quite an impressive demo where essentially your, the, your AI

• 10:00Azeem Azhar?

  is, is having like a support conversation with, with a real person. So, so where, where is that now? Where's that technology now? And, and was that a shiny TED style demo or is that the reality of customers using, using this?

• 10:15Speaker 00

  It's interesting. So and this gets into a very, I think, important point even for the broader question of AGI and what's going to happen to human society in the future. So the technology for, you could call them customer support agents is very advanced now. So, so we have agents that are capable of replacing, you know, something like 80%, maybe 90% of the calls that come into a call center. So if you're ordering a pizza or you're, you're changing a delivery address for a package, you're wondering what happened to your package, you know, all of those things, AIs can actually handle pretty well now. However, in terms of what fraction of labor that used to be done by humans, entirely

• 10:57Speaker 00

  by humans in these call centers, has been replaced thus far by AI agents, it's still miniscule. It's very tiny and it's, it's, it's a lot of it's held up by human decision making. A lot of it's held up by sunk costs in old systems and, you know, old ways of doing things. And so one of the lessons that people don't understand, like the, the real pain that startups have to go through isn't just inventing the technology, it's actually diffusing the technology and getting customers to understand how it works, to adapt their workflow to it, to buy it, make a bet because, you know, you're taking some risk to let some, some guy come in and say, hey, my black box AI is going to replace those 100 people over there in the Philippines that are in the call center. So, so it's just slower than

• 11:42Speaker 00

  what people think.

• 11:44Azeem Azhar?

  I, I was going to ask about that because you, I do remember you were going off to the Philippines quite regularly a couple of years ago where BPO, Business Process Outsourcing is 2 million people in, in the country employed there, about 10% of GDP. It's obviously bigger in, in India, 6 million roughly working BPO and, and support by the end of this year. Those are big numbers and they tend to be above average in terms of their education, human capital and above average in their, their income. So what is it like when you go out there and you show this technology particularly to those frontline operatives who? In a sense, you are producing a technology that does quite a lot of their work.

• 12:28Speaker 00

  I've been in many, many meetings now where in the room are usually it's managers, more senior people at a company, maybe a company that does BPO work or a company that hires outsourced BPO providers. But people who are familiar with the customer service problem and have maybe even worked, some of these people have worked their way up. So they have worked in a call center themselves or they spend time in call centers all the time. And what's amazing is how big their eyes get when, you know, we might show them a video demo, which is a recording of maybe me or somebody else talking to one of the AIs. And, okay, that's impressive enough, but then like some of them are a little more suspicious. So like, well, let me talk to the AI. Can I talk to your AI?

• 13:11Speaker 00

  And, and I'm like, sure. And I turn my laptop around and, you know, hit a button and they can talk to the AI. And when they realize that the AI is actually, you know, quote, understands what they want and can make changes in the system, generate a return label, you know, can do lots of things that we've, you know, functionality that we've equipped it with. It's kind of shocking because you see the wheels turning in their head and they say, wait, I employ a thousand people who do mostly this. Like surely there are some corner cases where you really want the AI to pass it off to a human. But 90% of the labor that those thousand people do is being done by this AI. And if I choose to power it with DeepSeek or something now, I can do it for, you know,

• 13:56Speaker 00

  it, we used to think in terms of one tenth the cost per hour of labor versus the human. But you could go down another order of magnitude. I mean, that sounds insane, but it's actually true.

• 14:06Azeem Azhar?

  It's crazy. And at one one hundredth of the cost, you could run this across five agents at the same time. And if four of them agree, you just move to the next step, right? You, you can construct a sort of a voting mechanism to reduce the number of edge cases you can't handle. That, of course, though, let's talk about what that means in those employment circumstances in those companies in, in that country. You said that deployment is slower than you expect because buying and decision making is slow. So that is from the bosses of the companies over in, in say, the Philippines. But what's your sense for the extent to which people have understood what this might mean?

• 14:48Azeem Azhar?

  Once the technology is being bought and and deployed and ultimately copied, the deployed to being copied by everyone's competitors in, you know, in the various various suburbs where this gets done.

• 14:59Speaker 00

  So we are right in the middle of one of the greatest examples of the future is already here. It's just not evenly distributed typically like if you're, let's suppose you're a growth stage startup. So you you found a particular niche and you're growing like gangbusters, but you're still very functional. You're a small company, relatively small, and you've got very good startup people, agentic people, et cetera. Those companies are going to or are deploying AI very fast. And so some of those companies, when you deal with customer service there or you you have to do something in their system, you literally are dealing with an AI and and and they've embraced the technology and it's what all the other companies are going to look like five or 10 years from now.

• 15:43Speaker 00

  On the other hand, I I have to go to these like trade shows and expos, this is founder led sales, right? So I I'm I'm often at a meeting where everybody at the meeting is somebody who owns like I own the customer service function for, you know, Hilton Hotels, I own the customer service function for XYZ Bank, I own, you know, right? Those are the people there and those people want to deploy this. They're being told by their CEO and CFO that they should deploy it. But if you look at their incentives, okay, imagine you're someone who came up, you're you're 50 years old and you came up for, you know, you've been working for 25 years in a call center environment, you worked your way up. You know how to manage people, you know, how to deal with, you know, contractors in the Philippines or in India.

• 16:28Speaker 00

  There's a bunch of like specialized knowledge that you've built up over the years. Okay. Some crazy professor startup founder guy comes to you and says, oh, you see this little black box? This is going to, this little black box, which isn't even really here, it's in the cloud, is gonna replace all of those people that your core competency is managing and provisioning those people. My little box is going to handle that for you, my little cloud thing. I just give, let me just give you a cryptographic key and then you can you can hook your system. Most of these people are incredibly threatened by that because they think ahead one year and they say like, what do they need me for? They need me to manage the now reduced by an order of magnitude set of people who just handle the edge cases.

• 17:10Speaker 00

  And this young whippersnapper guy, his thing is now the whole thing I used to manage. One guy who's a friend of mine, who owns the entire CX function for a very well-known entity, sort of similar to DoorDash or Uber Eats, or what's the one you guys have, Deliveroo?

• 17:30Azeem Azhar?

  We have Deliveroo, yeah.

• 17:31Speaker 00

  So this guy owns the whole CX function for a company like the ones I just mentioned. And he told me, most of these guys you're going to deal with in my space are going to pretend they're interested in installing your system. They're going to try to learn from you as much as possible, but they're dragging their heels. They actually don't want to install it for the reasons I, the incentives, because of the incentives that I just described to you. If you're trying to get the factory to switch from using steam to electricity, all the people who build steam and fit steam pipes are like against you. They're actually against you at some level and you have to fight through that.

• 18:09Azeem Azhar?

  So, I mean, I think that that's very reasonable assessment of the incentives and the pressures. It is a, the difference between the long term and the short term that we see with new technologies. And in a way, it goes to answer the question of what happens to the people who are working in the call centers. But I'm just, I'm quite curious just to come back to that idea, which is when you meet the frontline workers who have access to Google and YouTube and ChatGPT, I mean, they know what the trajectory of the technology is and they know what the promise is. And virtually every discussion about AI and the future of work puts BPO and call center workers right at the tip of the spear. How do, how do they feel?

• 18:54Azeem Azhar?

  How are they thinking about it? Is there a sense of, I don't know, what is the sense that is amongst those people in the conversations you've had?

• 19:02Speaker 00

  I think they feel about it the way that someone who was a blacksmith or a buggy whip maker felt when they saw their first, you know, their first automobiles rolling down the street, right? They, they could tell something was happening and they don't like it. They're vaguely worried about the future. But what else can they do? Now, I will tell you that in some contexts, again, I have some of this is proprietary information, so I can't say it, but in some contexts, companies that we are working with have said to us, we cannot say anything publicly about this because it's very bad optics for us to be shown to be replacing all these humans with robots.

• 19:43Speaker 00

  And furthermore, we're actually negotiating a labor agreement. Like, like, our whole industry maybe is negotiating a labor agreement. And the unions are kind of like, they're kind of aware of this. They're putting in little clauses, like if you're displaced by technology, then you're entitled to this severance payment. Or there's a pool of displaced workers and the employers must hire from that pool first before, if they fill any new job in the, they must hire first from that pool. So these are all standard things that a union would negotiate with the corporate employer. But the moment that like, they become more aware that this, this, this threat is imminent, then they're going to negotiate even harder. Right?

• 20:28Speaker 00

  Terms, so, so all of this is very sensitive.

• 20:31Azeem Azhar?

  What, what do you think is a workable zone of agreement to bridge this chasm that's going to emerge?

• 20:37Speaker 00

  I think the workable thing will be you install a super focus agent, you pay us one tenth or one fifth of what you're paying the human and everybody's happy. I'm not sure what you're asking me is even, no, just kidding.

• 20:47Azeem Azhar?

  No, I mean, I mean, in terms of the relationship with the workers and I think in terms of, you know, 10% of Philippines GDP, right? What, what, where is there a, a sort of balanced approach where, or, or is it just the law of the market and, and, you know, we hope the labor market is flexible enough and there's enough enterprise in these countries to, to spring up new, new jobs in all of the promising areas that will emerge?

• 21:12Speaker 00

  The problem is particularly acute in the Philippines because it's about 10% of their GDP. It's 2 million workers and those workers, that's, that's one of the best paths into the middle class. If that reduces by an order of magnitude in size, I think there's, they're just really strong. Social repercussions in that country. I don't know how to fix it. I I do think it's going to take some years before before all of this is done. I was just giving reasons a moment ago why technology diffusion is a bit slower than you think it will be like even though the tech is ready. It takes some time before it's widely adopted, right? So we do have some time to react. But I I just can't imagine like these jobs that can be done at one tenth the cost using AI, they they are going to be replaced.

• 21:55Azeem Azhar?

  Yeah, no, I mean, it it's the the logic is there, the economic logic, the historical analogies are there and. What we'll also start to see is the firms who get there first will be able to compete better than the firms that that don't. And I'm not sure if this will hold for AI, but it certainly held for automation in factories in the seventies and eighties and nineties. Which was that while there were job losses, it tended to be in the firms that didn't automate because they got outcompeted by the the companies who who did. I want to pivot to the other area that I think you've spent a lot of time thinking about. You're such a polymath. You you cover so many areas.

• 22:36Azeem Azhar?

  But let's talk about about China. I think we've all seen that there's innovation across so many sectors in China. Electric vehicles, semiconductors, solar, uh maybe even photolithography and so much more going on. And one of the things I think that has allowed that has been that from the, you know, the top down direction. There is this tournament style competition between mayors of cities and Chinese cities are essentially nation sized. They have twenty million people, thirty million people with great universities, extreme educational competition, attainment focus to get those uh top schools.

• 23:15Azeem Azhar?

  And the mayors themselves, they they act a little bit like, you know, mini Y Combinator bosses or mini venture capitalists. Pushing key technologies, fostering ecosystems to support research and development and competing with each other. As to who can be, you know, the self-driving car capital, the smart city capital of China and and of the world.

• 23:40Azeem Azhar?

  To what extent does that dynamic drive out the rate of adoption and development of these types of technologies at the moment?

• 23:50Speaker 00

  Yeah, I think the way you described it is perfect. So I think for people who are not familiar with China, on the one hand, you might think, based on mostly Western propaganda, that China is closer to North Korea as a polity or an economic system. On the other hand, you could say, no, it's actually closer to Singapore. Because Singapore also is not the freest political system, right? It's a little bit more authoritarian than maybe we're used to in the West. So is it more like North Korea or is it more like Singapore? And I would say it's much more like Singapore. So you have this economy in which the people who run a particular province or run a particular

• 24:31Speaker 00

  city, as you said, which could be 20 million people or even 100 million people, their incentive to rise in the party is economic and technological development. And in a way, their job is more like that of a CEO than it is like a Western politician who has to get re-elected in two years. And if you think about what pressures is a CEO under, someone's going to measure their performance, like the market or investors, or maybe the Communist Party Central Committee is going to look at the numbers from Guangzhou from 2025. And they're going to ask you like, well, what happened to your GDP? What happened to your average level of education? What happened to the number of patents that were produced? You know, all these things, they're going to get graded on.

• 25:16Speaker 00

  And of course, they still have a little bit of sensitivity, like if the local labor union protests or people are very unhappy about some policy, that also gets registered, right? And they're graded on that. So they're kind of balancing the way a CEO of a company has to balance, public opinion, performance of the company, internal R and D of the companies. That's kind of more like-

• 25:37Azeem Azhar?

  It sounds a bit like working at Bridgewater, frankly, with all this grading going on, it was popping up in my head. So that seems to be why we often misunderstand, if you're sitting in the West, that dynamism within China.

• 25:54Azeem Azhar?

  Do you think that that sets up the conditions for more rapid AI deployment than we might see, particularly in the US? I mean, the US has always led on these types of technology deployments as they expand out the economy because it is a leader in developing them, because there's a lot of capital, the market is big, it's a single market, so it's a great place for the entrepreneurs to sell. And of course, American bosses have an incredibly positive attitude towards technology by and large, so they're willing to move forward. Just many more of them are early adopters than you might find in the UK or Belgium or Germany. But is there

• 26:35Azeem Azhar?

  anything that you've seen that suggests that maybe that calculus changes over the coming few years with AI?

• 26:42Speaker 00

  I think that all of those factors that you mentioned, and I'll add one more, which is that as a fraction of the total student population in the country, in China, they're about two times more likely to do really hardcore STEM majors. And so people there tend to be able to do math and do really hard STEM stuff. And so there's actually no shortage of engineers. There's just so many good, talented engineers there that they can make progress in many, many different things at the same time. And so combined with what you mentioned, that, you know, the people there are just more pro-technology. Like, a lot of what, how you feel about technology, we don't really know how this whole thing is going to play out, right? So it's all vibes, right? So it's like, oh,

• 27:27Speaker 00

  you could have a society that like is very nervous about technology, is worried about ecological disasters or AI existential risks, or you could have another society that says like, well, we got much, much richer just in my lifetime. And a lot of that was due to just climbing the technological food chain. So we should just keep climbing it, right? And that's really where China is, China's much, much more optimistic. Like, if you talk to some random Chinese person, you say like, oh, we're going to have AI at the hospital, but the AI is first going to talk to you before you talk to the doctor and it's going to like do some triage. The average Chinese person is not thinking like, oh, what about my privacy? Or what if the AI makes a mistake? Or what if Microsoft gets all this data?

• 28:10Speaker 00

  Those are things which a typical American might think right away, but the Chinese people are more like, this is awesome, and this AI is gonna like, you know, so just that vibe difference is is a huge factor.

• 28:22Azeem Azhar?

  Yeah, that that vibe difference actually shows up in data as well. So Edelman, which is a PR company, has a long-running trust survey, the Trust Barometer. And for the last few years, they've been asking about people's attitudes towards AI. And this is a consistent, you know, longitudinal dataset. And whether people are roughly optimistic or pessimistic about it, there are many, many more layers to it. I encourage listeners to go off and get the Edelman Trust Barometer. What I found fascinating was that if you look at the US and the UK, roughly seventy to seventy-five percent of people are a little bit pessimistic about the the impact of AI on their lives.

• 29:02Azeem Azhar?

  And if you go to Asia, not just China, to Korea, South Korea, or to Singapore, it's it's the reverse. Seventy-plus percent are optimistic about it. And like you, I think it does come back to people's actual, just, you know, to use a word that's a bit unfashionable right now, but their lived experience, right? They they remember living in the village. They remember, even if you're in your, as young as your early forties, you will have these these memories and you'll have the memory of your life changing and transforming from living in twenty square meters, two hundred and forty square feet, with a shared bathroom on the corridor in a block of apartments shared with other families,

• 29:44Azeem Azhar?

  to living in some of the most futuristic cities in the world with, you know, all of the luxuries and and more in some cases. So I I think that that does contribute to a sense of of optimism. And it's, I suppose it sits with your observation of the depth of technical and engineering talent. But does that?

• 30:09Azeem Azhar?

  I mean, does that, does that really matter in the current moment? And I'll give you the two, the two thoughts about that. The first is that, you know, we are scaling human capability with AI systems. We are delivering graduate level server tools for fifteen dollars a month. And, you know, for fifteen dollars a month with, with Gemini or twenty dollars a month for Claude, I, I can suddenly spin up a lot of third year university students, hundreds of them. You used to be an academic, you know how good they can be and what their limitations are. But that transforms the reach of individual. So does that labor advantage in terms of high, deep STEM really hold beyond the next two or three years as AI gets better?

• 30:55Speaker 00

  Well, a lot of this depends on what you think about fast takeoff for AI. And not just, as I was saying earlier in our conversation, not just fast takeoff in the capabilities of the AI, but also the diffusion of it. Because you, you can have a situation where the AI can really do good stuff, but no one's letting it do good stuff. That, that's actually the situation in customer service or agentic stuff is like, there is this lag, right? So it's also true that I think there will be a long period of time where a skilled engineer who's trying to figure out how to do injection molding more efficiently or something to make like a car body, you know, that kind of physical stuff is still going to be done by human engineers. It's going to be a while before AI takes over that kind of very spatially oriented

• 31:40Speaker 00

  physical stuff. So, you know, I think some of the people in Silicon Valley who believe in fast takeoff and are very hawkish about the China AI competition, what they are thinking is, oh, all of these advantages that Steve mentions that the Chinese have, those are all going to be irrelevant once we have AGI, which is sort of the scenario that you mentioned. I think that time scale is going to be somewhat longer than what they think. So I don't think you're going to wake up and all of a sudden like, oh, the, you know, the nation of Denmark, because it has so many smart AIs, it's suddenly competing against China or the US, you know, using the AIs and not the people. But I, I think that's not going to happen overnight.

• 32:24Azeem Azhar?

  Well, I have an essay on fast takeoff coming out tomorrow morning. So for all of those who are listening, you'll all receive it exponentially anyway. So you'll see it in your inbox. I also want to touch on your perspective on this word competition or or race. Is it competition in the sense of being a zero sum game? I mean, sometimes it feels that this isn't like the scramble for Africa or the race for oil reserves in Baku and Indonesia and Pennsylvania. Is it really a zero sum game?

• 32:57Speaker 00

  No, I don't. I don't think it's zero sum at all. I mean, I think it's very healthy. All these labs competing to build better and, you know, assuming you're not, if you're worried about existential risk, it's a slightly different question, right? Because when people are racing, maybe they're less careful about safety. But in terms of just like incremental technological progress or progress in capability of AI, I think the competition is very positive. It's great. And, you know, like, I'm super happy that there are open source models coming mostly from the Chinese side that are almost as capable or just as capable as the top proprietary models. And that's incredible. The amount of like, building that's being done by tech entrepreneurs using those open source models is incalculable, actually.

• 33:40Azeem Azhar?

  Yeah, it's wild. I spoke to somebody in the medical space a few weeks ago and they said they, you know, swapped over to DeepSeek. They dropped their inference costs by 97% and it's made a real difference. Actually, not just on the cost. It's about all of the other things that you can now do because it's financially and economically viable to go off and use the AI to do it. And my favorite use of DeepSeek is when I'm on a plane. So I have DeepSeek R1, the, I think it's a seven billion parameter distilled version on my MacBook Pro. And, you know, when you're on a plane, you suddenly don't have your cognitive brain assistant with you. And now I do. Of course, it's not as

• 34:21Azeem Azhar?

  good as Claude Son at 3.7, but it's really better than, better than nothing and actually quite helpful to sort of maintain some degree of productivity and co-working. So I work very collaboratively. With with the models throughout my day. But this open source shift is really is so interesting. And I'm curious about your read on DeepSeek. I've heard a few things. I've heard this is China using open source to undercut huge investments in US industry as if China is this one thing. And and I've heard other things like, you know, that Lang Weng Feng believes that open source, open weights creates a more competitive space where one can get to AGI faster.

• 35:05Azeem Azhar?

  And there are other theories running around. What is your read on it?

• 35:10Speaker 00

  Yeah, before I answer that, let me just mention one other thing about the benefit from open source models, which if you follow the academic research, so professors of computer science or theoretical physics or math or whatever, working on AI related stuff. We went through a period a couple of years ago where like the academics were kind of locked out. But because of these open source models and the lower cost of compute, all kinds of really interesting research and papers are being written by academic researchers now. It's kind of a renaissance of, you know, new ideas, new insights into AI that are coming because these academics, instead of having to pay millions of dollars to do some kind of training run, they can use an open source model, they can tweak it, or they can do less expensive training runs.

• 35:53Speaker 00

  So so all of this is like multiplicative for innovation. Now, in terms of DeepSeek, DeepSeek was really not in any way part of the Chinese government. So if you follow AI in China, they had a list of national champions, which were the usual suspects, you know, like ByteDance, Alibaba, Tencent, Huawei, you know.

• 36:14Azeem Azhar?

  The kind of things a huge bureaucracy would be able to identify.

• 36:17Speaker 00

  Right. Companies that were already leading internet companies that had an AI lab and they were doing their thing. And by the way, one thing that's escaped attention in the West is a lot of the models produced by those companies that I listed are also competitive with OpenAI. So so it's not just one DeepSeek model coming out of China. It's a it's a handful of models that are at the frontier. But prior to the DeepSeek moment, DeepSeek, if you recall, was A quant hedge fund, which turned its attention to AI research because the founder Liang is a true believer. He really wants to build AGI. I have an interview on my podcast, which I'm going to release probably in the next month,

• 37:00Speaker 00

  with a researcher who used to be at DeepSeek. And, you know, his view, knowing all the people there, is that especially Liang is deeply committed to open source. And he just views it as the right way to advance the ecosystem. And Sam Altman, whom I also know, has also said the same thing. So remember, he says that the closed source is on the wrong side of history. We really want to get to the point where we can release open source stuff. They've now stated their intention to do so. Easy to do so after you've raised forty billion dollars at a three hundred billion valuation.

• 37:34Azeem Azhar?

  Why not?

• 37:35Speaker 00

  Pressure's on a little bit. But I actually don't think DeepSeek has much to do with the Chinese government. You know, maybe now things will change because they've become such an icon, they're so iconic for the Chinese AI effort. But everything they did up till now really has almost nothing to do with the Chinese government.

• 37:53Azeem Azhar?

  Yeah, I mean, I think that's really my main read as well, which is it's come out of left field, out of this high-frequency hedge fund. He's a very motivated and driven character who understands what matters to him. And who wouldn't be having so much fun doing this kind of research work with great people? And now a lot of momentum behind you. You know, people are waiting for the next DeepSeek model to launch. And they want to put it to its test against all of the benchmarks. Let's think, though, about how that changes the nature of deployment of AI. The way I look at this is that with these open source, open weight models,

• 38:36Azeem Azhar?

  which means that you can get access to the sort of internal mechanisms of the model, it enables a real flourishing of different types of models.

• 38:50Azeem Azhar?

  We tend to think when we're using ChatGPT of these AI models being highly generalized. But in business, what you really want is you want a model that is really, really excellent and cheap to run at the process you're putting it at. That it knows the winners of every Olympic marathon since the first one was run is of no use if its job is to do customer onboarding. And so there is this new approach, right? Where you you do this reinforcement learning based fine tuning on smaller and smaller models and you can push model performance in that narrow domain, perhaps a generation forward, right? Notionally, you can take a GPT three class model, distill it, make it small, fine tune

• 39:31Azeem Azhar?

  it, and then it becomes a GPT four class model in the domain that you you care about. And my hypothesis is that that's going to be incredibly commonplace over the next few years. This will be the way that software companies and businesses actually deploy these models because they'll want them to be small, they'll want them to be lightweight, they'll want them to run efficiently because if they run efficiently, it's fewer flops, it's less power, it's ultimately cheaper. And what you end up seeing is rather than these single monolithic powerful models, you see this society, this mesh of models and powering agents interacting with each other.

• 40:13Azeem Azhar?

  How does that align with your view of how things might develop now that we've seen open source can do what it can do?

• 40:19Speaker 00

  I I think that that is how things are going to evolve. And our our company is in a sense a bet on that kind of thing. Now, the the previous logic that someone like Sam would have given you, say, a year ago or a year and a half ago, was we're just going to crush you because the next version of our big model is going to be so awesome. All the tweaking and stuff that you did will just be eclipsed by the increase in capability of of our big model. And I think that hasn't really turned out to be the case. And and I think, you know, I think there will be a broad diversity of models of different sizes applied to different tasks in the economy.

• 40:59Azeem Azhar?

  That broad diversity and these models will be embedded in this series of agents. You know, where we started this discussion with Super Focus, where the agents that you were building for BPO and Call Center, that

• 41:14Azeem Azhar?

  I'm curious about, you know, whether there's not another flourishing of of agents now that we have cheaper open source systems. But back in twenty twenty three, I think it was, AutoGPT was this little framework that made its way onto onto GitHub, and it was the first sort of attempt to let people build somewhat autonomous agents, right, using what was then, I think, GPT three point five under ChatGPT, and it had a hundred thousand GitHub styles in a month. And back then, that mattered a great, a great deal. But I also see a population explosion of the use of specific tools with greater degrees of

• 41:54Azeem Azhar?

  autonomy in our daily lives. You know, I have workflows, many, many different workflows that I can address directly that trigger off tasks for my research work or just my admin work. And increasingly, you start to string the workflows together. And there's an L L M that's making a decision about where it should go next. And that's starting to look a little bit more, you know, agentic, because a longish process can can take place. What are you seeing? And how do you think that model evolves?

• 42:23Speaker 00

  I would say we're at an early stage for that. I don't know of too many people that are, you know, maybe you're one of the outliers in terms of how much productivity gain you're getting from, you know, using L L Ms in the way you described. As a company, we're still in the situation where we have to basically exquisitely architect and test the system for the customer, because you don't, you never want the AI saying the wrong thing to someone who's called in, who's a loyal customer of the company, right? So, so we're very worried about tail risk and mistakes that the model makes. And so, one of the things that I think the general public doesn't understand is like, how much rigorous statistical testing is required to know what are the tail risks associated with this deployment

• 43:07Speaker 00

  of autonomy? That could be a driverless vehicle, that could be an automated customer service device. You you need to understand what the tail risks are. You don't just want to turn this thing on and then discover like, oh, overnight, it got into some bad loop and pissed off a hundred thousand customers. Right? You just can't have that, right? Now, you're more fault tolerant because if you're just like, oh, this is just my productivity. So I'll let the model make a few decisions about this and then I come back and I'm like, yeah, it didn't do a good job. Just scrap that in enterprise production. You can't, unless the purpose is research. If the purpose is like something that really matters to a customer, cannot take those risks. So there's a lot of what we do, which is almost like statistical rigor, like designing a test system,

• 43:51Speaker 00

  running agents through that test system, characterizing what happens, showing it to our customer. There's a lot of stuff like that, that the general public doesn't think of at all when they think about AI.

• 44:00Azeem Azhar?

  Yeah, I, I think that comes back to one of your earlier points in discussion, which is about what deployment speed ends up looking like, because you're absolutely right. I, I have a particular agent that works, I would say 95% of, of the time. A lot of useful information, including your podcast, Steve, are on YouTube. They run for an hour. I rarely have time to listen to them. I do try to, but I'll fire these, you know, podcasts into my agents and it comes back with a presidential daily brief style, structured summary. It's incredibly useful and I would say about once every couple of weeks it fails for whatever reason. I can live with that, but if you've got an agent in customer service and, and it's failing one in a hundred times

• 44:44Azeem Azhar?

  and there's no fallback, that's expensive, it's problematic. And, you know, that, that's actually just a soft problem, let alone a hard problem in a medical situation. I think every person who's developing software knows that getting to the first 70% is trivially easy. It's the climb up the last 30%.

• 45:00Speaker 00

  And with autonomous vehicles, of course, you know, like the, the failure mode is like, there's a crash, there's a death, there's a multimillion dollar lawsuit and, and so, you know, that's like an extreme case. But even in customer service, you don't want cases where the AI, you know, pisses off someone.

• 45:15Azeem Azhar?

  Well, it, it all adds to that point about the, the, the barriers to persuading people who've been used to working with, you know, a particular degree of, of fault tolerance. I mean, I remember just, we're, we're circling back, but with BPO, when BPO launched as a sort of style of industry, it wasn't really even called that in the late seventies, early eighties. The completion success rate for these human outsourcers was about 30 or 40%. And that was good enough to get started back then. It's not good enough now, right? Because they are so much better. So if you're competing with with AI, you can't go in and say, well, this is 25% success rate. Right? You have you have to go higher. We're we're gonna run, go on, jump in and then we're gonna run short of time. I've got one more question for you. Yeah.

• 45:56Speaker 00

  A byproduct of trying to be rigorous about how well the AI performs means you're also being rigorous about how the humans are performing in the same task. And and you often learn shocking things that the managers are very shocked by of like how often their their agents give the wrong information, human agents give the wrong information to the customer.

• 46:14Azeem Azhar?

  Yeah, OK, so we've just opened that and I have to absolutely agree with agree with that. The expectation that we have on the machines is a much, much higher level of tolerance than the expectation we have of of people. And I think that that is a a vibe that will start to change as we get more and more experience with this. I I've used Waymos many, many times, but I've actually had now two problems with with Waymos. One problem was it was too scared to turn left because it was rush hour traffic. So my 20 minute journey turned into an hour and a half and several calls to support because the Waymo got scared and, you know, couldn't get through the aggressive humans.

• 46:56Azeem Azhar?

  And and the other time was when it locked me in the car and wouldn't let me out, which in of itself was was quite amusing. Have you used the Waymos, Steve?

• 47:04Speaker 00

  No, I use the Waymos whenever I'm in the Bay Area and in San Francisco. And I've had one, not really bad failure, but I've had failures. And so the failure rate's not that low. It's just, of course, we didn't, we didn't drive, you know, head on into a concrete wall or anything. But, you know, I've had failures like the ones that you just described.

• 47:21Azeem Azhar?

  Yeah, it's still a wonderful, wonderful experience. OK, I I we're gonna run out of time and I want to be respectful of your time. Super insightful, going down from the kind of granularity of being at the forefront of these call centers in the Philippines through to building the technology, trying to understand what's happening with China and what open source means. I knew with you that we would cover many, many dimensions in our conversation. So thank you very much. I I have just a question for you, like in a future that is full of AI agents that are a little bit reliable and working well. What's something that you might end up doing more of and what might you do less of?

• 48:01Speaker 00

  Again, I'm going to tell you something incredibly crazy. You can break the news here. We have been testing. So what do we have? We have a platform that controls the behaviors and the knowledge base of an AI, and it can converse naturally with people. We have been researching putting that brain into a small plush toy. So imagine a small dinosaur bunny rabbit that your kid is carrying under her arm. She's your kid is two or three years old, and this thing is teaching your child perfect French, or how to count, or telling it stories, singing little songs. And we find that the systems we've built, kids really enjoy interacting with.

• 48:41Speaker 00

  And just imagine the learning possibilities, like there's a window of neuroplasticity where you can acquire a foreign language at native level fluency when you're quite young. If you just hear it, maybe you talk to your nanny. Well, now every kid can have that.

• 48:54Azeem Azhar?

  Well, Steve, there is a Simpsons episode on exactly that, which I will look up and send to you. Everyone else, don't watch that Simpsons episode. You should listen to Steve's very rich podcast, which is called Manifold. It is really genuinely one of my favorite shows that I try to listen to, sometimes read summaries of. I am hosting these Friday Live discussions every Friday at 12:00 PM Eastern Time, 9:00 AM Pacific Time, and 5 o'clock in the UK. Thank you, everyone, for tuning in. Thank you, Steve, and have a great weekend.