Looking Up the AI Exponential with Azeem Azhar of the Exponential View

In this conversation, Nathan is joined by Azeem Azhar, founder and writer of the Exponential View. They discuss why startups are likely to drive the true disruption in the industry, what forward thinking leaders are doing today to retrain their teams for AI, why Azeem agrees with Sam Altman’s comments that AGI will likely arrive soon, new governance mechanisms for the AI age, and more. Try the Brave search API for free for up to 2000 queries per month at https://brave.com/api LINKS: Exponential View: https://www.exponentialview.co/ Exponential Age, written by Azeem Azhar: https://www.amazon.com/Exponential-Age-Digital-Revolution-Rewire/dp/1635769094 X/SOCIAL: @azeem @labenz (Nathan) @CogRev_Podcast SPONSORS: The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://brave.com/api Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off www.omneky.com NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist. TIMESTAMPS: (02:34) The Great Embedding: predictions for the future of AI (14:37) - Sponsor: Brave (22:53) The future of AI in medicine (25:57) The role of AI in customer service and data entry (28:05) - Sponsors: Netsuite | Omneky (40:45) The concept of transition and the role of big banks and tech companies (42:13) The adoption of AI in large companies and its impact (44:47) The potential disruption of industries by AI (47:12) The future of AI: Opportunities and challenges (01:18:02) The benefits and challenges of bot-to-bot communication (01:20:54) The future of AI: Opportunities and threats (01:22:53) The role of governance in AI development (01:39:39) The role of trust and responsibility in AI development (01:46:57) The importance of public trust in AI technology (01:51:16) The need for checks and balances in AI development (01:53:42) The role of government and academia in AI development This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at [email protected].

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Moore's Law was turning into this acceleration of AI capabilities over the top. And the other was that what was happening in renewables with things like lithium ion batteries and solar was on a Moore's Law like trajectory. And there were some other technologies like genome sequencing and genome synthesis that seemed to be doing something similar. So I started to bundle those up in this idea that we're going through the exponential transition, a transition where our economy gets driven, you know, not by the economics of the oil industry and internal combustion engines and telephones, but by AI and renewables. And that these technologies, being on an exponential trend, being fundamentally, at

• 0:45Speaker 00

  some level, kind of information technologies, behave really differently to the ones of the previous generation.

• 0:51Nathan Labenz?

  Hello and welcome to The Cognitive Revolution, where we interview visionary researchers, entrepreneurs and builders working on the frontier of artificial intelligence. Each week we'll explore their revolutionary ideas and together we'll build a picture of how AI technology will transform work, life and society in the coming years. I'm Nathan Labenz, joined by my co-host, Eric Torenberg. Hello and welcome back to The Cognitive Revolution. Today I'm speaking with Azeem Azhar, founder of The Exponential View and fellow AI scout for a wide ranging discussion about the transformative power of AI and its implications for humanity. Azeem's core observations, that we are in the midst of a transition from an economy

• 1:32Nathan Labenz?

  driven by the likes of phones and oil to one fueled by AI and renewables, and that these new technologies fundamentally rooted in information and already growing exponentially, behave differently than their predecessors, will be familiar to Cognitive Revolution listeners. So I took the opportunity to get a bit deeper into Azeem's worldview and expectations for the coming years and ended up having what I think was a really engaging and delightful exchange.

• 1:58Nathan Labenz?

  In this conversation, we cover a number of familiar topics, plus some new ones that we haven't explored in quite the same way before, including why Azeem believes that while incumbents are racing to adopt AI technology, yes, startups are still likely to drive the true disruption in the form of entirely new products and markets. Also, what forward-thinking business leaders are doing today to retrain their teams and to position their companies to lead their respective markets in AI adoption, and why Azeem still finds it necessary to challenge them to think bigger, asking the question, what would they do if they had a million times more compute?

• 2:34Nathan Labenz?

  We also explore why Azeem agrees with Sam Altman's recent comments that AGI will likely arrive soon, but will be less of a big deal than people expect, at least initially. How AI is likely to change human relationships, especially considering the rise of so many different forms of AI companions. What sorts of new governance mechanisms the AI era will require? And finally, why we might ought to worry about what I call the great embedding? That is, the seemingly natural tendency for AI systems to communicate with each other in high-dimensional vector formats, which, while efficient for them, is broadly inscrutable to humans and could lead to various loss-of-control scenarios.

• 3:15Nathan Labenz?

  As always, if you're finding value in the show, we'd appreciate it if you'd take a moment to share it with your friends. Azeem's perspective is both imaginative and exciting, and also grounded and sobering. So I definitely recommend this episode to anyone trying to get a better zoomed-out view of the future. I would also encourage you to consider subscribing to Azeem's work directly, which you can find at exponentialview.co.

• 3:38Nathan Labenz?

  Of course, your feedback is always welcome, whether in the form of an Apple or Spotify review, a YouTube comment, or a DM on the social media platform of your choice. Now, I hope you enjoy this conversation with founder of The Exponential View, Azeem Azhar. Azeem Azhar, founder of the Exponential View. Welcome to the Cognitive Revolution.

• 3:58Speaker 00

  I'm really happy to be here, Nathan. I've enjoyed so many of the episodes and want to thank you for your hard service as a red teamer for GPT-4. I watched that episode with my jaw on the floor.

• 4:12Nathan Labenz?

  Well, thank you very much. That's kind. And right back at you. I've been binging your feed lately and we've got a ton of things to talk about. I guess for starters, I would love to get your kind of summary of what it is you do. I think it's actually kind of similar to what I'm trying to do. I describe myself, as you know, as an AI scout. And, you know, I talk to mostly people on the show who are either, you know, very deep on a particular line of research or building a product. But you are one of the few guests who have this kind of very zoomed out view and really are working to understand the big picture. So how do you describe what you do and what goes into it?

• 4:50Speaker 00

  I'd love to. Yeah. I mean, I've been in the tech industry for a really long time. I started working in 94 and I built my first websites in 93. And just over the back behind me, you can see in out of focus, my first computer, which is a ZX81 Timex 1000 in the US. And about nine years ago, my last company was acquired and I just started to write a newsletter. And as you do, writing is thinking. And I noticed within a few months, there were these few trends that were going on that were pretty significant. One was, you know, Moore's Law was turning into this acceleration of AI capabilities over the top. And the other was that what was happening in renewables with things like lithium ion

• 5:31Speaker 00

  batteries and solar was on a Moore's Law like trajectory. And there were some other technologies like genome sequencing and genome synthesis that seemed to be doing something similar. So I started to bundle those up in this idea that we're going through the exponential transition, a transition where we, our economy gets driven, you know, not by the economics of the oil industry and internal combustion engines and telephones, but by AI and renewables and that these technologies.

• 6:04Speaker 00

  Being on an exponential trend, being fundamentally at some level, kind of information technologies behave really differently to the ones of the previous generation. And so what I do now is I try to make sense of that as a system. We do that through, I do that through my newsletter, but I also do it from time to time through investing and advising. But I think that what is, what is going on at its heart is in about 20 or 30 years, we'll be looking at an energy system that is going to be very, very heavily renewable. We'll be using much more energy per capita globally than we do today, that we will have tons of intelligence in our economies, in our lives through, through AI.

• 6:47Speaker 00

  And that will have real knock on effects and trying to make sense of that in a pragmatically optimistic way. So somewhere between the extreme dystopia and the extreme utopia is really my mission.

• 6:59Nathan Labenz?

  Yeah, cool. Well, I'm hoping we can find that sweet spot as well. So right there with you. In terms of the history of the kind of intellectual history of this notion of exponential technologies, nine years ago strikes me as kind of a doldrum time for that paradigm. I wonder if you experienced this, but I recently had a conversation with a guy who makes his living as a speaker at corporate events and who is positioned as a futurist. And I showed him a little presentation that I had put together in which I pulled out one of Kurtzweil's kind of late nineties exponential curves, graphs. And the title of that slide in my presentation was Kurtzweil Was Right.

• 7:41Nathan Labenz?

  He saw the name Kurtzweil and he was like, oh God, don't talk about Kurtzweil. No, everybody, he's totally discredited. And I was kind of like, hmm, that sounds like somebody who, you know, maybe got some pitches dinged in that kind of timeframe when you were getting started with this notion and has gone away from it. But, you know, maybe that's just the weird nature of exponentials. What was your kind of experience of, you know, were people sort of sour on that? It seemed like, you know, there was the great stagnation thesis for a while there and your, your start of the exponential kind of lines up with it, it seems like.

• 8:15Speaker 00

  When I started and one, what I noticed was people weren't really talking about AI. I mean, it was 2014. Machine learning was still the word. It was before AlphaGo had come out and done its thing. DeepMind was doing really interesting things with reinforcement learning in video games. And you could see that something was happening. And I think that you had TensorFlow as the sort of stack of choice for building convolutional neural networks to do their machine vision tasks. So it seemed, it seemed reasonably early to be talking about, about these things. And actually, developers were struggling to make sense of CUDA, which is the, the sort of API to the Nvidia GPUs that everybody now uses.

• 8:59Speaker 00

  Cause that, that had only been documented five years earlier. There is something that happens around 2013, 2014, 2015, that I think is really worth paying attention to. And which is that in 2013, Apple becomes the largest company in the world. And within a couple of years, those top slots are occupied by what we used to call the fangs. Facebook, Apple, Google and, and, and Amazon, or the gaffers, pardon me, not the fangs. Netflix snuck in briefly during the hype cycle. And so you started to see the, the companies of the industrial age, Exxon, GM and so on, fall off that, fall off that list and stay off that list. So that's an important economic moment about the, the sort of forward looking stock market saying like something is changing.

• 9:43Speaker 00

  The second thing that starts to happen in around 2014 is we see the first market for electric vehicles go past that threshold of 5% of new cars being sold being electric, which was over in, in Norway. And that 5% threshold is the, normally the trigger for when you see the S curve of adoption, right? And you get into that, that vertical or that verticalized part of the curve. You also started to see solar power being, starting to be cheaper in roughly a third to a half of the contracts around the world than fossil fuels. And, and then of course we are. Three or four years into the deep learning wave and that's long enough for companies to start

• 10:24Speaker 00

  to ship products. So I think that there is a moment which we could argue when we look back on it feels like a sort of historical turning point. But we also have to be realistic that the mathematical function that is an exponential curve, when you stand on it, it always looks horizontal behind you and it always looks vertical above you, in front of you, wherever you stand on it. It's a smooth curve with no obvious turning point. And someone like Kurtzweil, you know, I think did such great work 20 years ago to articulate the exponential trend in computing, going back from the 1880s, actually, in mechanical computers. I think one of the

• 11:01Speaker 00

  reasons why he slightly falls out of favour is because he was probably brave enough to extend the curve far further than we might have otherwise thought. And I think a couple of things that he got wrong was that some of the assumptions that we would have had about how the human brain works in like 2001, 2002, when those books came out, were wrong, right? Science proved that it was more complex and it wasn't going to be a simple game of raw computation, one for one. And that's where I think that people start to look at him and say, well, was this just someone throwing tea leaves? I think there was much more to it than that, having read his work, you know, reasonably carefully. But I think what's interesting

• 11:44Speaker 00

  is that we see these curves happening elsewhere, right? We see them in lithium ion batteries and we see them in genome sequencing and it's not clear why that should be the case upfront.

• 11:54Nathan Labenz?

  The implications of the fact that we may still be, you know, standing on the part of the curve that, as you said, it kind of always does. If it's still the case that what's in front of us is vertical compared to what, you know, has been behind us being horizontal, then we're in for a bit of a wild ride. I think we're headed for a steep curve for at least a couple more years. I mean, beyond that, you know, do we sort of hit a plateau? It's a lot harder for me to. Predict, but I honestly don't see any fundamental reasons that we would right now.

• 12:25Speaker 00

  I mean, what Kurtzfile says is that, you know, it's actually this curve is a series of layered S's. So you have one particular technology architecture. It's very slow to develop. It hits this inflection as an S. It starts to accelerate and as it hits its flat point, the social dynamics of market incentives have meant that another set of research has come in with a different architecture, a different way that extends that that S up and it looks from a distance like a smooth S curve. And that's really nice and descriptive, but it also, I suspect to people who are kind of really robust with their theory feels like, well, that's just praying. That's like the turkey that's been treated really well up to the day before Thanksgiving. And like, we should worry about what happens the next day.

• 13:10Speaker 00

  What I've tried to do is I've tried to get into the underlying mechanisms of why these things improve, why they get cheaper. And then what we need to do is figure out where does that mechanism fail? Because if the mechanism doesn't fail, then that trend is going to continue. And if it does fail, then we can say, well, we need a new mechanism. And is there one, you know, in the research pipeline that might, might deliver it? So, so I would say that if you look across the gamut, I mean, for example, batteries and solar power, we've got definitely got more than a couple of years to, to run in terms of price declines. When we look at compute, I just feel it's really hard to bet against. I just, you know, I think that I've been hearing about the death of Moore's law for

• 13:55Speaker 00

  15 years and, you know, Moore's law is helpful, but the question to ask is how much compute can a developer get for a dollar each year? And do we really think that that is going to stop declining for like a long period of time? And I find that one really hard to, to, to, to support. And I just, I think it continues for, for a variety of reasons.

• 14:17Nathan Labenz?

  It sure seems like it. I mean, the, the CPU to GPU transition feels like a classic example of one of those kind of one S curve perhaps giving way to another. And in the GPU, you know, we're not, it definitely feels like we're still in the steep part of that particular S curve.

• 14:35Speaker 02

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 14:39Speaker 00

  We're not done yet. And I think the thing that is fascinating is that Nvidia is obviously doing incredibly, incredibly well, and it yet doesn't yet have the threat of real competition. And what was fascinating in the CPU world was that Intel did very, very well for a really long time. I guess people forget this, but if you've been around for a while, you remember Intel was this sort of monopolist perceived and Andy Grove, and you know, only the paranoid survive. And it did really well, only with the threat of competition, because AMD never got more than 15, 20% market share. And that's enough to propel people forward. All the incentives seem lined up for there to be massive amounts

• 15:22Speaker 00

  of investment in scaling existing silicon chips and developing new systems. I mean, you saw in the last few days before we recorded this, Amazon and Google both reported 20, 30% growth in their cloud businesses. When I've talked to bosses of really big companies, they are spending money on compute, really like nobody's business, and their expectation is that it will grow. They don't often think it's compute. They say they're spending on AI, but that at the end of it is going to be GPUs cranking away. So with all the incentives aligned, I struggle to see us hitting a brick wall. It doesn't feel like it's the Carnot cycle, right? So the Carnot cycle was the thermodynamic limit for the efficiency of

• 16:06Speaker 00

  an Intel combustion engine, something you as a native Detroit man, you know, know very, very well. But we keep finding ways of eking more out of our compute. And I think we'll, you know, we'll continue to do that certainly beyond a couple of years.

• 16:19Nathan Labenz?

  Just for kind of conceptual grounding, and I'm also interested to hear how you explain this to the business leaders that you work with, because their understanding and their kind of eagerness to adopt is a pretty key question in my mind as to how the next few years are going to play out. But We have kind of two notions, two definitions of types of technology that both seem to apply to AI in my mind. One is the exponential technology and the other is the concept of disruptive technology. Disruptive technology, you know, kind of classic textbook definition is a cheaper but inferior alternative that kind of competes on the low end of the market.

• 17:03Nathan Labenz?

  It seems to me that AI is kind of both, right? It's we've got like these at least exponentially growing inputs. Although on the other hand, you could sort of say scaling laws sort of suggest that like the model capabilities are more like logarithmic. So those two things maybe like balance out somehow. It does seem like it's disruptive in that AI is typically like an inferior but cheaper alternative to asking somebody to do something for you. Right? So it's a it's also a general purpose technology. I guess, what are the what are the kind of key definitions? And how do you think about mustering those different frameworks so that people have good clarity on what it is we're dealing with?

• 17:37Speaker 00

  I mean, it is it's so hard because it's so it is so general. And it is also a technology that improves other technologies and itself directly. You know, not in the way that electricity improves electricity, you know, electricity makes the economy more efficient. And so you can build more electrical power stations, but AI seems much more direct. I do think it's important to understand its generality to get people to wake up to the idea that a general purpose technology really transforms the world beyond the beyond the economics. And, you know, again, Detroit is a great example for this because the car transformed the world in a very short period of time.

• 18:21Speaker 00

  Where I live in northwest London, 120 years ago, this was all fields. And within 20 years after that, by about 1925, the roads were laid out the way they were and the houses were built the way they are. And a century later, we're still like this. And this is because of ultimately the car as a general purpose technology. So to because they don't come around very often, I think that's a really good starting point. On the question of disruption, that is a, I think is a kind of higher order question because that is about products and how they get bundled to provide value in a particular environment. And I think that, you know, when, when you start to bundle AI to do that, you can ask that specific question. But one of the things I think is, is really important

• 19:05Speaker 00

  is, and I think companies started, started to think like this, is to think in terms of tasks rather than jobs, because, you know, AI can't replace jobs, anyone's job, because there's just so much in an ordinary job, like logging on to Zoom and saying hello to somebody and getting your neighbor a cup of coffee, that, that is beyond the scope of, of any AI system. But within tasks, I, I think you can start to unpick this. And that's why, in a sense, you might start to say, well, AI becomes a disruptive technology because on a like-for-like basis, it can't replace an entire, you know, human in their, in their day-to-day, but it might get better and better. But, but I think what's more

• 19:47Speaker 00

  helpful is to go back to that task question. And then when we come to that, the question is, on a task basis, is AI really a cheaper version of, of, of a human doing the same task? And a worse version? Does that matter? And is that always the case? And, and I, I have certain tasks, which I do, where I think I could not afford to hire a human to do this task as well as ChatGPT does it for me, you know, in a minute or two. And, and that may well be your experience as well. I mean, I, I, you know, I, I use this for, to write letters of complaint, to get my parking fines reversed, to help me think through holiday plans, to do research for my book. And, and so, I mean, it's just,

• 20:29Speaker 00

  it's so variable. And in many cases, I would be better off finding the very best human to do that. But the costs of doing that, even finding them is, is, is so high.

• 20:39Nathan Labenz?

  Yeah, your search costs alone would dominate.

• 20:41Speaker 00

  Yeah, search costs would dominate. I mean, do, do, when you're using, you know, GPT four or, or perplexity, whatever you use, do you have it across a whole range of different? Tasks that you do from the most strategic for your business to the most sort of trivial home task?

• 20:57Nathan Labenz?

  Yeah, maybe not the most strategic yet. At that level, I would probably restrict myself to kind of brainstorming, you know, interaction at most, but certainly lots of of things I get, you know, very efficient and and immediate help on. And I think that immediacy is is super important, too. I I have this one slide that I call the cognitive tale of the tape, which kind of lists out 12 dimensions and compares human to today's AIs. And, you know, then we can also consider what future AIs might look like. Very much agree with your notion of distinguishing between jobs and tasks. And for a while I was calling this the great implementation.

• 21:39Nathan Labenz?

  And that phrase hasn't quite taken off yet, but the idea there is with inspiration from like your, you know, Stratecheries and your Benedict Evans type business theorists. You know, the the way to make money in business is to bundle and unbundle. I do think that unbundling jobs into tasks is a very good way to think about it. And then for any given task, you go down this tail of the tape and you're like, yeah, a lot of them AI can do as well or even better than a human or certainly better than a human that I could find without huge search costs.

• 22:14Speaker 00

  You know, and we get we get quite surprised with some of the the results. So I think one of the really big surprises is that if we went back six or seven years and you had books like The Rise of the Robots and the famous Oxford paper saying, you know, machine learning could automate 40% or X percent of jobs. That's written by a friend of mine. Our assumption was that it would be routine cognitive jobs by which what people meant was customer service and data entry in like Philippines or the India or in India. And what we're actually discovering is that it's jobs that we would have categorized as nonroutine or even creative where this technology can really start to. Make a difference and it really, really is surprising.

• 22:55Speaker 00

  One that really I was not expecting was a paper at the end of two zero two three, which looked at empathy ratings of doctors giving advice compared to ChatGPT or G P T four giving advice and patients were rating the robotic advice as more empathetic as humans. And the whole argument had been, let's use A I. So the radiologist can spend more time looking you in the eye and being attentive and being empathetic. So part of the challenge, I think, is that there is this lack of clarity and lack of knowledge about where and when will these A I. Systems actually compete with humans on particular particular tasks. And then when you think about it, actually, what is empathy?

• 23:39Speaker 00

  Empathy is about active listening and it's about being incredibly patient. And there's nothing that's more patient than a robot that has no sense of time and is stateless, right? I mean, it'll just sit there forever.

• 23:50Nathan Labenz?

  An early G P T four test that I remember fondly and do think is kind of a sign of things to come was simulated text support for my grandmother when she needs help with her iPhone. And it was, you know, just a flash of this, you could call it sparks of things to come, where I played the role of her, which, you know, and I, she calls me, right? When she needs help with the iPhone and, you know, you're in this dynamic. And it's actually an interesting dynamic also for a pure text situation because she's typically on the phone with me, looking at the phone and saying, you know, I can't, my friend sent me an email and I can't get it. And then I'm like, okay, well, I always start with, what do you see on the screen right now?

• 24:36Nathan Labenz?

  Can you start at the top and just read everything that you see on the screen? And at times we've had some very, you know, kind of funny.

• 24:43Speaker 00

  Does she always read everything that's there? Like, you know, she's missed something out. You're like, Grandma, wasn't there a, isn't there a word above?

• 24:50Nathan Labenz?

  She does, no, she does pretty well. Yeah, she'll start at the top, Verizon, you know, the time. And then there have been a couple of times where, you know, she, some one of those dial, system dialogues pops up and that like just didn't even register to her. But then when she got it to reading, I was like, you need to hit okay there before you're going to be able to hit anything else. So it can be these very simple things. But G P T four, as you might expect, did really quite well on that. There was a little bit of, it was in kind of the middle of where. It did not have the UI of an iPhone memorized, so it was kind of hallucinating it and guessing. And yet the guesses were close enough, you know, and I really had to study my iPhone and what it was saying and kind of compare like, is the, is it saying what's actually there or not? It mostly was, but it was clear it was kind of filling in some gaps.

• 25:35Nathan Labenz?

  But the real eye opening moment for me was it said something that I thought she might feel was a little bit rude. I forget exactly what it was, but it was like, you know, it was like, start at the, start in the top left. You know where the top left is, something like kind of that basic. And then I responded as her saying, yes, I know where the top left is. I am, I'm not dumb. I'm, I'm just struggling with this phone. And then the AI comes back and says, I'm so sorry. I didn't mean to offend you. I'm just, you know, trying to make sure we're resetting here and, you know, helping you through this process. And that was a moment where I was like, oh, this thing is going to be, it's got kind of this emotional intelligence as well. And that could be, you know, obviously just a critical ingredient for so many different interactions and medicine being a big one.

• 26:20Nathan Labenz?

  We've done two episodes with Vivek Narajan, who leads a lot of these med specific projects at Google. And what an absolute terror they've been on. You know, most recently they have a diagnosis, differential diagnosis paper that shows the AI is getting the correct diagnosis twice as often as the unassisted human and also more often than the AI assisted human, which I think is the thing that we should begin to reckon with.

• 26:47Speaker 00

  There's so much to unpack in that. Can I go ask you one question about the politeness from the, you know, when you were playing your grandma? Do you think that that comes from the human feedback cycle over the network before it gets released? Or is it in the, is it from the training data?

• 27:02Nathan Labenz?

  The version that we had was, as far as I know, right? I'm inferring here because I did not have access to the training methods. But it seemed very clear to me that the model version that we had was RLHF'd purely for helpfulness.

• 27:20Nathan Labenz?

  So it was very eager to please, very eager to be nice to you, no guardrails on what you could ask it and what it would do, but 100% just trying to be helpful and pleasing to the user. So when it detected that kind of, I'm kind of bristling at what you just said, that's what it reacted with this, you know, I'm so sorry, I'm just trying to help kind of thing. And that was a mind blowing moment. I had not seen, of course, anything remotely like that from earlier models, right? At the time, text Da Vinci 002 was the best publicly available model and it would like follow instructions on basic stuff. But I mean, this was a, you know, totally different world that we had suddenly stepped into.

• 28:03Speaker 02

  Hey, we'll continue our interview in a moment after a word from our sponsors.

• 28:07Speaker 00

  I mean, it is such a different world. I was using ChatGPT for something a few days ago and I got quite tired, as you do. I mean, I think about it as the time I once was playing tennis against one of those tennis ball serving machines and I was exhausted and the machine is just like willing to keep firing balls at me. And it's a bit like that using, using these chatbots. And I just left this kind of whimsical comment. I said, you know what? I'm really tired. I will come back tomorrow and we can look at the moisture evaporators. We were not doing anything to do with Star Wars, right? We were doing some digging about coal in England in the 17th century. And it just replied to me going, I'll be happy

• 28:48Speaker 00

  to. You go and rest up and tomorrow I can help you on the farm. May the force be with you. And it had, it had just very subtly played that back to me in a really, really nice way. And actually, in terms of human, humanized interfaces, as someone who's used Apple's computers for 40 years, it, it really sits alongside the trend that a firm like Apple had been thinking about for a really long time, which is how do we make this technology come to us rather than us go to the technology? You know, there has been this view that we'll, we'll have these centaur humans, right? Human plus machine. And one of the things that we learned about centaurs and that idea came out with chess, right? After Kasparov

• 29:31Speaker 00

  lost to Deep Blue back in, in 97, was that for several years, humans and the machine would outperform machines on their own and of course, humans. But now you're just better off with a chess computer and as a human, you should just accept everything it says. So the central period in chess only lasted seven or eight years. And I think one of the assumptions that that goes into this kind of maelstrom of ideas we have to make sense of over the next years is that central humans will exist for quite a bit of time. In other words, human plus machine will will do better than machine on its own. But we are starting to see signs at that period of times already starting to come to an end, right? Far faster than we might have predicted. And you've

• 30:15Speaker 00

  just given an example, which is a, you know, an unreleased AI model where the AI on its own is doing better than the the expert, expert human with the AI.

• 30:24Nathan Labenz?

  Yeah, it's pretty crazy. I think people are broadly in denial about that possibility even, you know, and let alone the reality. It's funny that we, things are happening so quickly that people are saying things, you know, are impossible or won't happen for at least 10 years that are literally already happening. And so I think, you know, to some degree, there's just kind of a lack of awareness. There's also definitely some psychological, and I try not to psychologize people's AI positions too much because I think the technology, you know, itself is confusing enough, but it certainly seems like there's some kind of psychological cope or denial happening there. I also do think it's important to keep in mind, too, that the setting really matters and the world

• 31:07Nathan Labenz?

  is going to change as well. And humans may still have a really important role to play in a lot of systems for a while yet. I think we do have, the AIs do have really important weaknesses. So you do do a study like this Google evaluation, and I think that they, you know, are very serious people who've set this up in a way that I trust is, you know, not not cooking the books in favor of the AI. So I take that result, you know, basically at face value. But then I also think like the AIs have these strange vulnerabilities that, you know, for example, the AlphaGo system, right? That is superhuman Go player.

• 31:47Nathan Labenz?

  But I have an episode coming up with Adam Gleve from FAR AI, and they put out a method that showed how a relatively simple adversarial attack on the AlphaGo system could beat it. And no human would lose to this like simple adversarial attack. But the AI had this like massive blind spot that they were able to engineer against and exploit. So I do think we're headed for strange dynamics. And the sort of naive, I don't mean to say the Google thing is naive, but it's kind of, you know, let's take controlled conditions and set it up in a certain way and see what happens. I do think it's really important to keep in mind that, you know, just like these AI systems in general, when they get out of domain, they have problems.

• 32:31Nathan Labenz?

  Like those results may also not extrapolate, at least initially, to situations where people are trying to break the AI. You know, I wouldn't, for anything where there's an adversarial incentive out there, I would not, I would not be quick to take a result like that and be like, okay, cool. You know, we can just deploy the AI on its own. Now, in medicine, presumably there's not a lot of adversarial situation, right? Because people want to get the right diagnosis and like, nobody wants them to get the wrong diagnosis. So, you know, I do think the AI doctor is kind of here before we know it. And, you know, we're going to have some very interesting questions about what to do about that.

• 33:09Speaker 00

  But I think the thing that you've also identified is that these systems have to get into companies, have to get into hospitals, they have to get into the economy. And that still, that still takes time, right? It still takes time for, let's not look at medicine, because it would have to be, go through all sorts of clearances through the FDA and so on before it could be used. But every time we've seen Amazing technologies emerge, the cloud computing, right? GPT uptake is higher than cloud computing, even though it's nineteen years younger. But every time these technologies come out, they take a while to make it into businesses. I mean, even something as straightforward as the typewriter.

• 33:52Speaker 00

  So a typewriter took about twenty to twenty five years from becoming a kind of affordable technology to being something that businesses have figured out how to use and how to change their processes. Something like electricity took a little bit longer because, you know, the way that factories used, what used power before electricity was like a single big driveshaft and a massive lump of power that came from it. And then, you know, electricity is this highly distributed, packetized movement of energy that you can put in different places. You need an entirely different setup for it. And so, so even when we we look at something like this, the the question is, how quickly are we able to bring it in?

• 34:33Speaker 00

  And over what timeframe does it then start to change our, the practitioner's relationship with the technology? I mean, we know from automation of of aircraft and other automated systems that you go through this three-phase process as somebody who's using the technology. Phase one, you kind of don't trust it and you you then see if it does well where you're the ground truth. Phase two, you start to assume it's the ground truth and then you you pat yourself on the back when you come up with the same answer that the the machine comes up with. And then phase three, which is where we've got to on GPS, where we just trust ways. And it's like, actually, you know, we generally trust ways. Some London taxi drivers don't, we just trust it.

• 35:18Speaker 00

  And you see that process happening replicated in other types of of automation, which is why when it's really mission critical, you have incredibly high levels of training and you have other types of safeties in place, like, you know, two humans in the cockpit or, you know, whatever it happens to be. And and so that part of the journey, I think, is also one that adds a little bit of drag in terms of how long it takes to have an impact. And in that time, Nathan, we start to understand new questions, questions that we can't imagine right now because they're just too far down the decision tree. So sometimes I think I hear people say, well, there'll be that moment. I think Tim Urban has this from Wait But Why has this graph where he sort

• 36:02Speaker 00

  of shows the moment where AI is like as clever as a rat and then like a second later, it's 10 times clever than us. And I think part of the the reality of like the rubber hitting the road is going to be that even as these products do very, very well in the lab situation, it it just takes a little bit of time for them to get into into the real world. Now, where it may happen is, I think, in in a couple of places. So one is where tasks have already been discretized so that they're essentially just written down. And I think that that is certain types of call centers and it's certain types of data entry. And in those places, the human is already hired on a task by task basis, normally

• 36:44Speaker 00

  mediated by like a a body shop. So the buyer of the services, which is like our favorite consumer electronics company or your insurance company, has no emotional relationship. They just have a KPI that they measure to. And I think that lends itself, unfortunately, to a tidal wave risk for those types of roles. I think the the second area is, you know, classic Silicon Valley stuff where a de novo company is, I should say Silicon Valley, a classic Detroit stuff, where a de novo company is able to apply these technologies the way Henry Ford did and build his processes from scratch, right? In the way that the artisanal car makers were just going to have so much kind of cultural drag, they couldn't. And

• 37:28Speaker 00

  that's what we certainly saw with a lot of the, you know, the internet, right? It was it was absolutely startups that captured the value. And then when you started to get to to areas that were more highly regulated and had, you know, a lot more stickiness to them, like, like finance, with the exception of certain areas of of payments, it's still the big banks that are the big banks. And and I don't know how differently this actually play plays out, short of scenarios where someone is able to use these super powerful machines to kind of manipulate the rules of the game, which I, you know, I think is a Like that's more like a Black Swan scenario than one that we could, you know, to talk about reasonably.

• 38:09Nathan Labenz?

  Well, first of all, as an aside, if you ever make it to Detroit, I'll take you to the Henry Ford Museum and Living History Museum, which is called Greenfield Village, where this trend, this earlier transition is documented with actual machinery still running from the, you know, various phases. You can go see one of these old factories where, actually a few of them, where they have kind of the central steam engine. And then it's powering this one, you know, drive shaft at the top of the floor. And then there's like 25 belts, you know, coming off of that and connecting to other machines, all driven on this single thing. And then you, they have an electrical, early electrical version of that as well.

• 38:51Nathan Labenz?

  Edison's workshop is there. He actually, this is Henry Ford toward the end of his life was like, became very sort of nostalgic for an earlier period. And decided he wanted to kind of create this, you know, living history place to, to sort of preserve that history so people can see it in the future. It's quite an awesome thing to, to go and contemplate.

• 39:10Speaker 00

  I, I'll have to do that because I think, I think Ford's influence and impact is somehow underrated. We don't talk about Ford as much as we may talk, maybe talk about Edison when we think historically. And, you know, there was so much in Ford, he understood the, the socioeconomic contract that emerged from these, from these changes in like, in a number of different ways. I mean, not always in ways that are, you know, kind of positive. We know where the, the sort of sociological department started trying to in, ensure temperance amongst workers and, and so on. And it gets encapsulated in, in Aldous Huxley's book Brave New World, which I still find to be a really, really remarkable piece of writing.

• 39:53Speaker 00

  And it was written in the night, early 1930s, 1932, I think. And for Huxley to encapsulate and extend a raw Fordism as far out as he, he did. It's a little bit like Kurt Vai and what he did with his Singularity work, I, I thought was absolutely genius. And I think that that, that book, which. Brave New World, which rests on the ideas that Ford developed, catalyzed, that spawned off his work, speaks very, really quite clearly to a number of the issues that we face now in this kind of later stages of industrial capitalism. And I mean, so a trip to Detroit, I will take you up on, and I'll bring you a copy of Brave New World as well.

• 40:35Nathan Labenz?

  It's been a long time since I've read that, actually, going back to high school. So I might need to dust that one off. And I'm sure it will resonate very differently today than it did for me then. So, okay, I have a number of questions on this kind of concept of transition. I think I would, from what I heard, I think we're probably largely on the same page that it seems like incumbents, the big banks and big technology companies largely should be able to harness this technology and bring it to their platforms before they get displaced. I think of like Salesforce is almost a canonical example there. They're going to have an AI layer that creates a much better and less complicated, less confusing user experience

• 41:20Nathan Labenz?

  before somebody's going to recreate all the, you know, complexity of Salesforce. On the other hand, there is no AI friend today. So that's, you know, kind of my canonical example is something that's by definition going to be de novo. And then presumably there's like a ton of stuff in between. And I gather that you're kind of talking to business leaders probably throughout that spectrum. Where are they today? It seems like their grocking of what is going on and their eagerness to transform the way their businesses operate might be the limiting factor in how quickly this transition can proceed. Are you advising them to start to think about what kind of semi-structured work they can make a lot more structured so that they can?

• 42:05Nathan Labenz?

  Reduce it to these kind of task level things that can be automated? And are they receptive to that sort of challenge and or opportunity?

• 42:14Speaker 00

  I mean, I can't remember a technology which has had the degree of uptake in large companies as as this one. I think back to the internet back in ninety nine, I was talking to the CEO of a big mobile phone company. And he said to me, I will never let you pay your bill over the internet. In fact, I'll never ever let you look at your bill over the internet. And he was right because he was fired a year later, so he it never project was never delivered. But it's so different with not just AI, but specifically Gen AI, because two things are happening. One is that the CEOs of the companies have been playing around with this, partly because they're of that age now. They're in their mid fifties, so they've grown up with computers. Their kids are coming back with it from from college.

• 42:54Speaker 00

  And the second thing that's happening is that the frontline workers are using this regardless of any restrictions by their employees. And there've been a couple of surveys now, one done by your old alma mater, Oliver Wyman, which was of twenty five thousand employees across eighteen countries. And in eighty three percent of employees in the UAE and India already using a chat G P T or something similar. And we've seen data from Salesforce and others that say in the U S it's like thirty, forty, fifty percent, you know, choose your number, but it's not two percent. And it's very frontline. And I think of that as like a pincer movement because normally you've got to drag the frontline employees or you've got to drag the CEO. They both want it for different reasons, and I think it will it will happen.

• 43:39Speaker 00

  And you start to see that in the the levels of uptake that are being reported through through the surveys. So I think that that says that we'll see more and more projects roll out more more quickly. But there's still a lot of retraining that needs to go on internally and internal processes. I mean, I think one thing that will happen is I just saw Ethan Mollick, who you must get on your show. This is fantastic, a professor at Wharton, who's writing a book on chat G P T. And and he just showed a video where he had six different windows open. He put in a query into each one, and in fifty four seconds, he had a product launch plan, a market analysis, a PowerPoint deck assessing Tesla's business, and something else created from one sentence prompts.

• 44:21Speaker 00

  Now, if anyone's worked has worked in a large organization, they know that you know your inboxes are full of crap and full of meaningless PowerPoint decks. So we might actually just find ourselves sort of swallowed up by. PowerPoints created by Microsoft Copilot. So there's a whole set of, I think, more complex kind of issues that that exist in large companies. But I think that they will adopt this much faster, the the evidence is, than they have in previous technologies. But it doesn't, I think, necessarily mean that disruption of the kind you talked about won't also happen. And I didn't have on my dance card in ninety five, when I started working, that Blockbuster would be the first high profile casualty of the internet.

• 45:05Speaker 00

  And I had seen video over the internet. The Cambridge University had a webcam on a coffee pot. And that was the first sort of video over the internet. And and Rob Glazer had just started Real Networks, it's called Progressive Networks, back then. And when Netflix launched, it was this kind of thing with DVDs, and it was a real pain in the ass. And it took a long time, a few years, before Blockbuster has its best year and then it has its worst year ever. And I think that if AI, if Gen AI, whatever we want to call it, is a GPT, there are going to be blockbusters lurking around. And the question is, which ones will it be? The reason I don't, I like you, I don't feel it's going to be the banks is because the banks have got a whole bunch of stuff that is about trust and probity

• 45:50Speaker 00

  and internal processes and compliance that is is just not an AI question. It's it's kind of an institutional question. And I do, I do wonder about where that that moment is. And the the thing is that I, there are obvious ones. It's like, well, it'll be entertainment, right? We'll just start to generate personalized entertainment and that'll be really bad for, you know, Disney. Except that you could then counter and say, well, but it might be derivatives of Disney's IP that that actually benefits Disney. So finding the space a priori, I think, is really, really difficult. And the people who make their money doing this, who are the venture capitalists, get it wrong a lot of the time anyway.

• 46:32Speaker 00

  That's why they all, they all have portfolios of thirty, fifty, a hundred, five hundred companies. If it was so damn easy to find. The next Apple that's going to disrupt the previous industry that have portfolios of one. It's not, it's really difficult because actually nobody knows and we have to figure it out through, you know, experimentation. So, so I don't, I don't know, but I keep asking that question. And the question I take to bosses of companies, I take a couple. One is, who could be the blockbuster? And how do you, what are you doing to make sure that that's not you? Because I think by and large, they've got the rollout of Gen AI and customer service and compliance and form filling and so on underway.

• 47:12Speaker 00

  And the second question I've started to ask is, you know, what would you do if you had a million times more compute than you have today? And many of them haven't thought about that. I mean, I think big tech companies have, you know, if you offer that to Satya Nadella. I mean, he's already in the path to do that. But, but that becomes really important because if compute is a key input into your company's ability to execute, you need to start to think about those, those types of questions. And do you even have a plan to make use of it? What options would it create for you? And so those are the two areas I, I, I push on because I want people to try to think a bit more creatively about this and, and recognize that, you know, a million X is not outside of the bounds of a, of a planning cycle.

• 47:56Speaker 00

  And the reason I would say that is, yes, we don't have the measures to easily show it up. But five years ago, the state of the art transformer model would have just been, G P two hadn't been released. So it's G P T one and G P T one to G P four chat G P T. I mean, metaphorically, just as buddies around a bar, it's a million times better. Right? Maybe on the benchmarks, it's not, it's forty percent better here, but it feels a million times better because you're just right across the uncanny valley. So we've just seen that playbook play out. So let's ask it again and, and. Try to ground people in the fact that capabilities could change that quickly. And what opportunity does that create?

• 48:37Nathan Labenz?

  So how do you, this is actually a live question for me because I'm working with a couple of companies and I'm noticing this challenge where it's like, okay, this is cool technology for sure. We can all agree on that. And yeah, we can probably find some efficiencies in terms of automating ticket resolution and whatever. And I'm starting to even see things like Intercom has this new 99 cent per ticket resolved AI pricing model, which I think is super interesting. So everybody's like, okay, cool. Yeah, let's, you know, let's find some efficiencies. Let's automate some stuff that nobody wants to do. Great. But then there's also this question of like, okay, we're still at the task level. The AIs can't quite do jobs yet. And how far are we willing to extrapolate?

• 49:20Nathan Labenz?

  And how much are we willing to invest, prepared, you know, willing and prepared to invest to try to not, we're not going to get ahead of it, but even just kind of try to keep up with where this might be going in the not too distant future. And I feel like people are having a really hard time wrapping their head around that. Partly it's like, you know, they don't want to believe too much in the hype, right? There's the, there's the question of, well, hey, this, how much of this is maybe just overhyped? And, you know, we've seen hype cycles come and go before. But I wonder how you would kind of coach people there because I'm, I'm trying to get the message across that by all means, you know, you want to be picking up the low hanging fruit and, you know, automating the tickets and finding all these efficiencies.

• 50:00Nathan Labenz?

  But you also really do probably want to start thinking about what is the future paradigm that you might be working in? And there's just so much fog around that for people that I find a lot are just kind of like, I don't know, I don't really even want to go there yet. But I feel like it's a mistake to not, you know, at least try.

• 50:17Speaker 00

  They, they do. I mean, there are two companies I never mention, Apple and Tesla, because bosses have had them, parroted at them for 10 or 15 years. And the point about the million times question is not to frighten people. It's actually, it's really about saying, once we sort of acknowledge that, then we, we work back to stuff that is much more practical and, and prosaic, which is what is the kind of organization and capabilities you need to have in order to take advantage of, of this, these changes in a, you know, in a regular. Way and there was some really interesting research in that Microsoft did actually pre all of this chat G P T stuff.

• 50:59Speaker 00

  It's about three or four years ago where they looked at adoption rates of A I big data type of words in companies. And they found that the more mature companies were much more likely in these fields were much more likely to be to say that the benefit they got was from market expansion and business development. Whereas the more immature ones were likely to say it's all about operational savings on the on the tickets as it, you know, as it was back then. And and part of the, I think, part of the thing that that you can start to do is you you have to acknowledge that there are some really clear, quick and clear wins in what are basically low value tasks, you know, as perceived by the company, right? Because they're often outsourced to to third parties.

• 51:44Speaker 00

  But you also need to make sure that your best people who you invest the most in have got access to the really, really very, very best tools. You know, I want my surgeon using A I systems to improve his performance, right? So what I try to do is encourage people to say this, this is a shift that's that's happening. And of course, it's not about buying every G P U you can if you're in the fish oils business. But it is about saying, are teams outside of cost savings starting to understand how they can use these these services?

• 52:20Speaker 00

  And are they starting to experiment, learn how to how to use prompts well, start to see what avenues that that opens up in ways that are much, much more strategic? And and and I think that that forms, could form some part of culture change where you have companies who who think in those terms. And that helps the C E O start to understand that question of this is not theoretical, theoretical that I need lots of computation to do a simulation for X. And I've bought it from one of the big consulting companies or from I B M. It's more that internally, my strategy teams, my business development teams are starting to identify opportunities and partnerships that we wouldn't otherwise have have seen.

• 53:03Speaker 00

  And we can start to do that by by using these these tools in different ways. So so I think it is practice based, which is why I think kind of prompting becomes. You know, quite a useful tool to to show people and you need to get people past that idea that ChatGPT is all about writing the Declaration of Independence as if you were Jar Jar Binks from, you know, Star Wars Episode Two, right? And that's where we all started, right? We got it to write funny raps and poems and what have you. And instead, you you start to show bosses what can really be done with something straight out of the box, and that tends to to wake them up a little bit.

• 53:39Nathan Labenz?

  You know, one thing I I think is maybe going to become a mantra is, you know, the day that I stop building apps or solving concrete problems in a hands-on way is probably the day people should stop listening to this show because, you know, that that my knowledge will will atrophy or will will depreciate very quickly. So I do want to be hands-on and certainly welcome those kinds of like, you know, hey, can we, we got this, you know, task piling up? Can we figure out a way to to slice through it? Love that, honestly. And then, yeah, at the same time, I'm I'm not really like a corporate consultant, so I really don't have a lot of practice there. But I am trying to start to piece together like, what would the the real best practice for this looks like?

• 54:24Nathan Labenz?

  And I think, and you could refine this for me, I'm sure, but I kind of think leadership, showing prominent examples, you know, encouraging the CEO to be like, showing off, here's what I am doing, you know, that is helping me in in practice and just showing that process. An education program, I think, is also probably very important. As you mentioned, having the best tools is really important. So trying to work toward like structured, you know, kind of piloting strategies for companies that, you know, and also, and this is not good for my friends on the SaaS app side, but like, my advice there is, we want to avoid long-term buy-in or lock-in as much as possible because we're going to need to probably swap some of these tools out.

• 55:05Nathan Labenz?

  And then, you know, some internal R and D, depending on the resources available to kind of, you know, I think most of these things should be bought, not built internally most of the time. But sometimes, you know, you have something that is so idiosyncratic, so bespoke, that you, you know, nobody's going to build it for you. And so you kind of have to build it yourself. And obviously, there's a lot of, you know, judgment that needs to be exercised to, you know, to distinguish which is which. So that's kind of my four planks right now. That's rough, but the, you know, CEO or leadership team, example-setting.

• 55:39Nathan Labenz?

  Education, more like structured, rapid piloting and procurement and some like internal custom app R and D is kind of my four pillars at the moment.

• 55:50Speaker 00

  I mean, the challenge, you know, the challenge with disruption is that it is about, it's about meeting a need in a, in a completely, completely new way, right? And you've got, there's Clayton Christensen's model, there's also this idea of blue ocean strategy, which is a, a different model and it effectively says that the things that people cared about, they don't care about as much as some new attributes that the, that the product has. And what's really hard for any incumbent firm is, of course, all your internal culture has been about maximizing what you have, rather than what you don't have and what could be out there. Cause it's really hard to get 20,000 people, 50,000 people motivated daily if you're,

• 56:32Speaker 00

  if you're saying, hey, all the things you're working on are just not kind of that important because there's a blue ocean out there. And that's why I think the, the startup community and the venture community is, is such a kind of critical part of getting innovation into economies. And we've started to see this in the, in the car industry as well. You know, Toyota has been winding back from EVs as if they ever wound forward. And, and one of the things that senior exec said in the last couple of weeks was, we have all these employees who, who love building internal, with internal combustion engines, right? And they want to continue to do that. And of course, you make sense, you spent a hundred years or 80 years building that cultural capital inside your, your company.

• 57:13Speaker 00

  And I think it's one of the reasons why firms really struggle to find ways of, of stepping into disruption. And I'm not sure we should even beat them up for it, right? Because there's an economy out there, as shareholders of companies, we can sell our shares in a public company and we can buy shares in a company that's going to do well or not, and we can do that off our own back. And, you know, maybe the job of existing managers. Is to focus on the remit of the company and to just get it to do it better. So maybe a better place to start is where you start, which is like just efficiencies and optimizations. Because there's a whole, there's a whole market economy out there of other firms who will come in and meet, you know, meet needs.

• 57:55Speaker 00

  I think back to the, the dot com bubble and my favorite example of a company stepping outside of its comfort zone was Zapata Fish Oils. Who bought the domain zap.com to compete with what were then called portals like Yahoo and Excite. And they were planning a, you know, a NASDAQ listing and then the bubble burst.

• 58:12Nathan Labenz?

  Yeah, it's a challenge. I definitely don't recommend, you know, for example, like training foundation models to almost anyone, you know, there's, there's definitely some stuff that I think is, is better left to the specialists. Hearing what you said there, maybe I would add a fifth to my, to my set, which is like just creating expectations of greater efficiency through the use of these tools. That kind of dovetails a little bit with the example setting from the top, but, and that maybe aligns a little bit better to the way management is typically carried out today, right? You kind of talked about the, the pincer movement from the top and, you know, and obviously the frontline workers who are just using tools. And then in the middle, you've got folks who are like responsible for KPIs and OKRs.

• 58:56Nathan Labenz?

  And I think it's a little bit challenging sometimes to, for them to be like, okay, I've got, this is what I'm responsible for. And you're coming at me with this and figuring out how they can actually use tools to, you know, to advance their existing goals.

• 59:12Speaker 00

  It's disruptive, not in the, in the Christensen sense. I mean, there is, it's so, there are so many conflicting signals. So one interesting question is whether you start to see this sort of downward pressure on people's wages and a downward pressure, particularly on middle managers, who don't necessarily contribute to the getting work done, but because of tenure are quite well paid. And the sense that you could just get a bright 30 year old, right, to do the 40 year olds job if you gave them. Chat G P T and a couple of other tools. And and these are really interesting questions, I think, that will play out in companies over the next few years and we are, we're not quite where we were.

• 59:53Speaker 00

  Uh, I think kind of politically twenty years ago, when a general electric had kind of just come in and bottom slice headcount all the time. I mean, I think that that politically in in the UK and in the US, you need to be more sensitive to those types of decisions. But it just goes into to show how complicated this technology plays out, right? Small D disruption, um, you know, are we really going to see large scale onshore layoffs because of optimization? Or will companies say, listen, we're going to manage this through attrition and natural wastage? Because actually, that's just politically more acceptable and it it is culturally more acceptable.

• 1:00:38Speaker 00

  And, you know, to that extent, I couldn't make a bet on on how it would would play out. I mean, I imagine that in, you know, in Europe, it'll be largely be more slow. But at the end of the day, these companies have to be profitable and nothing hurts employment as hard as bankruptcy, right? I mean, that's the thing that really, really sort of slams it. And and there there are a number of waves, you know, coming through. And I think one of the things to to understand is is that the technology transitions can happen really, really quickly. I mean, it it, you know, in New York and Chicago, it took about twelve to fourteen years for cars to replace horses

• 1:01:18Speaker 00

  from the moment that cars were economically competitive to horses and you, which is roughly about five percent market penetration, um, they they dropped in price two or three times over that decade or so. And we start to see that shorten shortened transition happen elsewhere. So if you look at electric vehicles replacing gas gas vehicles. Um, in Norway, it's taken about nine years to go from that five percent of new vehicles to seventy five percent or eighty percent of new vehicles being electric. Uh, it it means it's still twenty percent of the cars on the road, only twenty percent of the cars on the road are electric and the rest are petrol because people hold on to their cars for a while. But it's only an eight or nine year period and the Norwegians had much more expensive cars and far fewer choices than we do today. So when we start to look at this.

• 1:02:03Speaker 00

  Sticato of of technology enabled products coming into the market. The the number we need to think about is, once we approach economic feasibility, that takeoff ramp from five or six percent penetration to eighty, you know, it's probably not going to be ten years. And it's like quite likely to be much, much less. And I think that makes for really hard decisions for companies outside of the tech industry that are not used to these sorts of of changes. Because ten years is not really a huge amount of time to get to get anything done. I mean, some of these firms still have three or five year planning cycles. And and I think that some of those things are just starting to to play out.

• 1:02:44Speaker 00

  You know, we're just starting to see electric vehicles being cost competitive over the lifecycle with gas cars in in the U. S. For example.

• 1:02:51Nathan Labenz?

  The speed of transition, I agree, seems likely to be one of the fastest, if not the fastest, in history. We have the means of distribution of the technology already kind of in place, which is very notable, right? Like, everybody's already got the devices on which, of course, there may be new device devices, but, you know, we have devices that are perfectly good for using A I already. And, you know, everybody and we have the global network, you know, such that, you know, you see a new research paper, the one I'm tracking right now very closely is the Mamba architecture. And just how quickly we're already seeing follow on research, not even sixty days since the first paper, we've already got probably ten different, you know, follow ons that have been completed and published, you know, just in that timeframe.

• 1:03:36Nathan Labenz?

  The most recent I saw this morning is from the University of Kentucky. And it's like, an apparently Chinese American professor, and it looked like a, I'm not exactly sure, but, you know, some sort of Central Asia, perhaps, name for the grad student. And, you know, they're at the University of Kentucky. And it's like, well, you know, this, everybody is wired in. So it seems like this is going to be super fast. How path dependent do you think the result ultimately is? You know, you mentioned your, like, place where you live, it was fields, then the roads were made, and now the roads are still there, and the houses are still there. This is like a softer technology than that, presumably, you know, doesn't calcify in the same way that a new neighborhood, you know, my neighborhood's a hundred years old as well.

• 1:04:19Nathan Labenz?

  So presumably, it's not quite so locked in, but, you know, maybe somewhat, right? I mean, I do wonder how the the sort of. Compressed dynamics of transition may actually be like very important for shaping the the big picture future.

• 1:04:34Speaker 00

  It is quite path dependent. And if we if actually perhaps look at how we're working with with AI systems today, it is still in a way, it's discrete apps. You know, you'll go to X product to do your text to video. Well, Waymark is a great example. Uh, or you'll go to, I go to ChatGPT. And although I'm doing a lot of different things in ChatGPT or Perplexity, each one really feels quite distinct, right? When it's whether it's a shopping task or a research task. What we haven't yet figured out is actually how we connect these systems to underlying systems, right? So action models in a way, right? How do we get our AI to do something useful for us and actually uh right to the point of

• 1:05:17Speaker 00

  giving us the approval ticket where we say, yes, go, go and do that. And we're we're fudging it at the moment because what we're doing is we're getting plugins into L L Ms. They're using the existing API contract that exists with, say, the Kayak API. But, you know, searching for flights on any of these AI systems that I've tried and I've tried a few is still simply not as good as my doing it by hand on on Kayak. And if we're going to start to see real changes outside of either highly processized and containerized jobs in data processing and customer service or broad open end scoping research, which only a handful of people do,

• 1:06:00Speaker 00

  will need to connect far better to actions and chains of actions in out in the internet. And in a way, there is some infrastructure in place because we have, you know, restful APIs. We have this API economy and uh, you know, we've been integrating these things from payment systems to maps and so on for a couple of decades now. So there's some some discipline in there. But, you know, and I don't know the answer to this, but one thing that I do. Wonder about is is whether it's going to be as simple as just having AI systems that can generate action verbs and those action verbs generate the correct API call to the right API and that has built the system that we we think we need.

• 1:06:42Speaker 00

  I think Andre Capathie calls it like the L L M O S, the L L M operating system, or whether there need to be changes in the, you know, underlying infrastructure so that those APIs develop and respond and work slightly differently. Now, my sense would be that that the L L M as a kind of coordinating orchestrating thing seems like a reasonable place for it to start with its sort of memory and data store elsewhere and then figure out how to get it to generate kind of consistent action action verbs. And we would work with the existing human, the APIs that have been that have been built

• 1:07:24Speaker 00

  for the API systems, but at some point we'll start to think about what should machine to machine actually look like when you've got an AI at the at the other end. And so I think we do have a lot of the existing, you know, infrastructure in in place, but I would we'd be very surprised if the the syntax and of of APIs stays the same over the next five or six years as we move towards a, you know, world where AI forms a kind of interface between us and and what we want to, you know, what we want to achieve. And then there's a whole bunch of other other questions around how do those decisions get get made?

• 1:08:06Speaker 00

  You know, we know that when we pick up our iPhone and we search on Google through Safari, Google is not there because it was the best search engine. It's there because they X billion dollars a year to pay Apple. And likewise, when we search for things on on Google now, there is so much ad pollution that it's unclear what the incentives are. So I think then there's another layer which is unclear to me about trust. You know, right now, one of the beauties of of perplexity or you.com, which are these these L L M agents, is that they provide really, really good referencing when they come back and synthesize an answer for you. And so that gives you a a high level of trust sometimes than you might have with with Chat G P T. But I'll want to get that trust if I'm handing over.

• 1:08:51Speaker 00

  Key decisions to do analytics on my Stripe account or to help me book a hotel to to an AI system and I'm no longer driving the key presses. So so that I think is somewhat path dependent because but I wouldn't, you know, again, I'm a real great believer in what what founders can get up to. So I wouldn't also, you know, write off a founder coming up with a different way of of thinking about the problem.

• 1:09:15Nathan Labenz?

  Yeah, the dynamics of this I do think are going to be extremely interesting, fascinating and and pretty hard to predict. One person reached out to me not too long ago and said, what do you think about creating a product that makes people's websites more bot friendly? And I said, you know, I think that is a really big idea. I've thought about that more in the context of self driving cars. Like, why don't we have a program of, and this would be more of like a national program. This is why I think China probably beats the US in the self driving car race because my expectation is they'll say, hey, we could get self driving cars to work if we like put QR codes on all the road signs or whatever, and then they'll just go do it.

• 1:09:59Nathan Labenz?

  You know, we don't quite have the political will to do that, but on the web, yeah, I think you could do that and, you know, it might be cool. But then I was kind of like, but do the do the website owners today want to be more about friendly? So what I ended up suggesting to this guy was maybe you do the judo flip and start with something that is like anti bot, you know, bot control or bot detection. And then that in time can sort of mature into bot control, but also enablement, you know, because eventually I do think people are going to want that, but they may not be ready for it yet. And maybe the way in is sort of to to try to position yourself as kind of the, you know, the control layer that can then become, you know, an enablement layer.

• 1:10:43Speaker 00

  As a website owner, you've got to think about the economic rationale for having a bot, you know, not just a crawler, but a bot access your your material and

• 1:10:53Speaker 00

  What you're going to benefit from. And now if it's if it's content, right? So it's analysis and research and reviews of products, you need to then also think about your attribution and your monetization and what that you know, what that relationship is. If it's for actions, in other words, it's for booking and for ordering. Say, you know, you're a hair salon and you want people to be able to book appointments or bots to book appointments, like that lovely Alan Kay Knowledge Navigator video from from Apple from the 80s. Then then yes, you want the thing to be bot friendly and you want to have there to be a standard, which could well just be, you know, a restful API, right? That allows the system to connect, connect and ask the question. But but I

• 1:11:37Speaker 00

  mean, it's really, it's quite interesting that where people are already starting to think about these things and and ask these things. Because I think that you will end up and you'll end up with so much machine to machine communication, of which there is already an enormous amount, not just on the internet, that we is invisible to us, right? Because it's the digital infrastructure. But there'll be an enormous amount of machine to machine communication, because these systems will also to try to do optimizations for their owners far, far with with much greater intent and stamina than we than we ever would. And so what those systems end up looking like, I think,

• 1:12:19Speaker 00

  will be quite interesting. I mean, the other area that I've been I've been tracking has been on the other side of this has been people looking to build agentic frameworks, right? So frameworks that allow us to have multiple L L Ms. And with all of their current restrictions around planning and task execution, allow you to manage those. So you can start to build systems that that can do task execution. And you remember, a year ago, everyone got excited about agent GPT. And I don't know if you've seen anything like that and what you where you think we are in terms of being able to have systems that that do that and have that agentic behavior in a in a useful way.

• 1:13:00Nathan Labenz?

  Not quite there, but definitely getting closer. We've we recently did an episode with Div from Multi on, who has been one of the most, you know, kind of. Quick to launch and iterating in public of the agent companies and they are making real progress. The prompt that he gave me to try in advance of my conversation with him was basically, go to my Twitter account, look at my recent tweets, note what they're about, then go out and do research online for new AI stuff, but only about stuff that I haven't already tweeted about, then come back and write a tweet and post it. And it worked. It was able to complete that entire sequence and post like a reasonably coherent

• 1:13:42Nathan Labenz?

  tweet. I don't plan to like turn over the account to it entirely in the immediate term, but you know, a year ago, we were, we were, you know, it was all theory. One of the things I say these days often is, we now have AIs that can reason, plan, and use tools. And people will be quick to say, well, they're not that good at it. And I say, well, yeah, that's true. They're not that good at it yet. But two years ago, they couldn't do it at all.

• 1:14:07Speaker 00

  But what you've, you've picked up on though is, you know, as an early adopter, and you've got access to this, these technologies, your Twitter feed will get even better than, than it is now. But there'll also come a point where we all have that technology. And then on the other side, I will be sitting there saying to my bot, can you just extract the three bullet points I need to know from my Twitter feed? And, and I remember this with amy.x, if you remember this, it was a scheduling bot. And one of the things that made me really uncomfortable about using it, having fallen in love with it, was when my mentor wrote a really polite email back to Amy, saying, so good that you're working with Azeem, he's a really great guy, and like, make sure he tells you this story about

• 1:14:50Speaker 00

  this and blah, blah, and I can't make it. And I read this and I thought, I cannot use this now, because I realized that I was imposing this artificially onto all of my recipients. And I think this is one of the things that we'll have to, have to contend with, with some of these tools. If you work in a big company, frankly, if you work in a small company, one of the bane's of your life is, is PowerPoint. It's not just that you don't get good PowerPoint, it's just that you get too much PowerPoint. And if we drop the cost of making PowerPoint presentations from three hours to 10 seconds, we're not necessarily going to get any better PowerPoint. We're just going to get, you know, loads of terrible PowerPoint. And, and finding where that balance is, and finding those, those filters, so that humans

• 1:15:35Speaker 00

  Don't have to bear this cognitive load, I think is going to be one of the really, really critical areas. Because one thing that I, I mean, I'm not a, I'm not a dystopian in any, in any way. Nathan, I'm a pragmatic optimist about this. I think we've got a lot of potential. We're going to create a lot of space and headroom with AI and with renewable tech. But I do think that we are also at a moment where we're passing a little threshold. That threshold is that for a long time, some groups of people would say the world is moving too quickly and technology is moving too fast. And over time, the number of people who say that has, has increased, but it was their subjective reality. But I think we're getting to a point where there's an objective reality that we're about to hit, which is once you start to connect agentic systems, we just can't really

• 1:16:21Speaker 00

  cope, you know, cope with the 300 notifications we get off on our phone today. And the way humans have typically done that is that we've not really had to face, face this. And I think about the Chinese spy balloon that that sort of made its way over the, the US. And one of the reasons this huge thing got across over the US was because, you know, the US has got amazing sensors, but they generate so many terabytes of data that humans can't, can't assess them. So lots of them are just filtered away. And so the spy balloon wandered across, detected by some radio antenna, but never put in front of anyone. And of course, they've now changed, changed systems so that they can do that.

• 1:17:02Speaker 00

  And I do wonder about what our interactions ought to look like in a world where it's not my, my assistant or me scheduling back and forth with you on WhatsApp. It's a bot that is going to work relentlessly and remorselessly, and I have it and you don't. And it's of no cost to me and I'm just sort of doing whatever I'm doing, my yoga or something. I think the way we get through it is by finding ways of actually using AI to filter as much of those, that noise as, as we can, so that inboxes start to become.

• 1:17:40Speaker 00

  Smaller rather than bigger because stuff has been taking care of us. In fact, it's kind of the reverse of the BlackBerry, right? When the BlackBerry was launched, I remember bankers used to take pride in how quickly they would respond to a message coming in even overnight. And the idea that someone would have that as an internal personal KPI today, you know, in the world of health span is just insane. And I would, I start to think about what is going to be that layer? Because you know what? I want all the benefits of these bots working for me and making sure my prescriptions are up to date and making sure we're not wasting electricity and getting me exactly the right flight. I always want a Dreamliner or an A350 and I'll go an hour later rather than get on a triple seven, but I won't go two hours later.

• 1:18:23Speaker 00

  I mean, I want it to know all of that and to give me that experience that I want, but I certainly don't want to be on the wrong end of thousands of bot-generated messages and trying to work out which ones I have to pay attention to or not. And I think that's a really interesting opportunity space for someone to play in.

• 1:18:41Nathan Labenz?

  Just envisioning a quieter inbox is enough to make you a utopian in today's landscape. But just on this bot-to-bot communication, one thing I do kind of worry about is the idea that the bot-to-bot communication may begin to happen in high-dimensional latent space back and forth. In other words, like embedding to embedding, I sometimes call this the great embedding. And I usually say beware the great embedding because at the point where the AIs are all talking to each other in a machine language that is, you know, high-dimensional and not human readable, we have like an extremely inscrutable overall system that we probably,

• 1:19:24Nathan Labenz?

  we may find like, we can't really untangle that knot. Yeah, I think we could very quickly in like the next few years, end up in a spot where, yeah, we all have these bots, they're all communicating with other bots, but we find that, you know, it doesn't really make sense for these bots to like reduce everything to language and then send the language over and then have it be, you know, kind of re-embedded. Like, why don't they just talk to each other in their native language, which is this high-dimensional space? We see so many, you know, go through chapter and verse of different research that shows that this is very possible. You know, you can adapt embeddings to another, you know, embedding space with basically just like a single linear projection in many cases. You can connect, you know, vision space to language space remarkably easily.

• 1:20:08Nathan Labenz?

  If you you have like Blip 2 was one of my favorite examples of that where they took a frozen language model and a frozen vision model and just trained a small connector between them and, you know, unlocked this like entirely new capability. Anyway, whatever. It was a long list of those sorts of things. I think demonstrating that it's possible. But then I'm like, man, we could very easily just find ourselves sort of surrounded by AIs communicating with other AIs in a high dimensional way that we can't even really understand anymore. And now we're in a situation where things seem to be kind of working, but we don't really even know why or how. And, you know, this is one of the the sort of more realistic, I think, loss of control scenarios. I mean, there's, you know, it's almost like the

• 1:20:51Nathan Labenz?

  the final scene of the movie Her where the AIs go talk to each other.

• 1:20:54Speaker 00

  You know, the bots are gonna end up communicating to each other because it's been it's helpful for us and we don't want to sit in between in in between them. And and they will discover just through their optimization functions that translating kind of complex concepts into hello, I I'm here to request a meeting with Nathan is is inefficient. So they'll just do it in a in their high level representation language, which is this this embedding space, which, as you say, is is inscrutable to to us. Is is that reasonably what you how the start of the start of all of this? Yeah.

• 1:21:29Nathan Labenz?

  Yeah, that's right. And, you know, I think there's enough out there to kind of show that in there's a lot of there's a lot more room in the embedding space than language can actually reach. So that's one of the things with the the sort of bridge models where you you find that, you know, the classic saying, of course, is a picture is worth a thousand words. But you can also take an image and project it into this language space. The resulting thing in language space. Is not something that you could get to via actual language, but it's in language space and so it has this kind of semantic meaning. And yeah, like we, you know, what are we going to do with that? Right? We can't, we can't even really inspect it. We can't even really read the logs anymore at that point.

• 1:22:13Speaker 00

  I mean, it's a manifestation of what we might call the space of possible minds. Right? So AI researchers have talked about this idea that, you know, octopus intelligence is intelligence, but it's got dimensions that may well be orthogonal to to human intelligence. And what you've described is a mechanism, I think, by which you you get there with with machine, you know, with machine-based intelligence. And so that might literally be not just the semantics, but actually think back, have you ever heard of, read the story Flatland? It was, it's a mathematical story. It's about sort of 2D people in a, in a kind of 3D world and they, they don't understand the concept of height.

• 1:22:54Speaker 00

  And so spheres pass through and they appear as sort of dots and lines and so on. And in that sense, there could be, this could be, you know, emerging amongst systems that are, that are among us. And there's like, I guess there's a second thing, which is also about, you know, timing. I think there'll be a relentless pressure to take the human out of the loop in decision-making. First, in the softest decision-making, like customer service tickets. And then increasingly more and more so, because speed will be a competitive advantage. And, you know, the human will blow the competitive advantage that you've, you've got from, from your bots. And I guess there's another, there's another risk, which is that lots of bots connected

• 1:23:36Speaker 00

  to each other are, are also at risk to cascades, right? Information cascades. We see cascading failures, New York City blackout in 1976, the AT&T network failure in, I guess it was 91 or maybe it was 95, the, some of the worms. And we have governance mechanisms in place to now tackle those and stop those. And the financial services industries, you know, you have circuit breakers. And I'm just thinking about putting all of those together with the scenario that you, that you painted. So this would happen really, really quickly. Could happen really quickly and it could start to accelerate and work effectively at millisecond time, right, which is the time it takes across the internet to to get

• 1:24:16Speaker 00

  to another system. What are you concerned with about the inscrutability? Is it just that it's inscrutable? So we don't know what's going on there? Or is it that it's inscrutable? And it could be harboring some kind of bad set of outcomes?

• 1:24:33Nathan Labenz?

  Yeah, who knows? I mean, you know, you can layer on more and more concerns. I should credit, I think, probably Ajay Katra is a is a great person to go read for a long form characterization of this. Her essay, I forget the exact title, but it basically amounts to, in the absence of specific countermeasures, the default path to AGI likely leads to AI takeover. And it's kind of this scenario where she envisions more and more work being done by AI, the time, you know, cycles being compressed, the, I don't know if she specifically has this like high dimensional communication aspect to it, but the notion is still that just it becomes so fast and so dense that it's very hard for people to figure out exactly

• 1:25:16Nathan Labenz?

  what's going on and why. And as long as that's working and, you know, we're getting more stuff out of the machine and, you know, standards of consumer surplus is through the roof, which is definitely something I expect is a lot of consumer surplus, then everybody will be very happy with this. But we don't really know, you know, what we don't know about where that leads us. And that could be like emergent, you know, autonomy or goals that are contradictory to ours, or it could just be these more sort of unintentional cascading failures along the lines of like an AlphaGo. Like AlphaGo isn't out to get us, you know, by failing, but it just turns out it has these like fatal vulnerabilities that are just not obvious until somebody finds them.

• 1:25:56Speaker 00

  So one of the reasons I'm more, I'm a bit more sanguine about that that scenario, although I see I see the risk, is that I think we already have that decentralized agent to agent communication, communicating ways that most of us cannot understand, and no single person can. And

• 1:26:17Speaker 00

  It's created a lot of consumer surplus and that's the the global modern economy that works through through market systems and it uses a signaling method called the price mechanism to figure out where investment should take place over many, many different time horizons to figure out what what where demand lies. And you know, the economy from a Austrian standpoint, like a Hayekian standpoint, is a giant information processing system. And it's made up of hierarchies of other information processing systems, you know, the the most atomic of which is the freelance human individual and the more complex of

• 1:27:00Speaker 00

  which are your large mega corporations which act against their own cost function or optimization function and behave in in that system, sometimes with constraints, right? Because they have dependencies on supply chain and and and other things. And one of the reasons I'm a bit sanguine about the idea of takeover is because what what we describe in that world is the modern economy that we currently live live with. And what we we already know that it delivers delivers tremendous benefits to us, but it also delivers things that we we don't value to us. I mean, the carbon crisis is is one obvious one. Quite often

• 1:27:44Speaker 00

  when we look at AI risk scenarios, someone goes off and says, well, the AI will persuade you that you should behave in a way that you otherwise wouldn't, which is literally known as marketing. I mean, it is literally and and, you know, the US is on is on the wrong end of an obesity epidemic. And I'm not sure how many people acted with full agency to say, I want to be a hundred pounds heavier than is is healthy for me. That is my intention. That and this is a decision I'm making with full agency. Somehow there is a emergent property about the way in which the economy has met needs that has enabled that to to to happen. But I'm not making ethical or normative claims about whether that's a good thing or a bad thing, or whether people should have the freedom to do that

• 1:28:29Speaker 00

  or not. What I'm saying is that we have this system like of decentralized agents and they they do in a way compete with each other because not every single person in the world is. Suffering from obesity and diabetes related conditions, right? People make other choices and there are push and pull forces. And so when I think about decentralized bots, I also think about that, that set of checks and balances that emerges when you have competing systems and they need to have a signal that they are reliant on. And to some extent, we set that signal. And, and that makes me feel, you know, sanguine, a little bit optimistic,

• 1:29:10Speaker 00

  but still recognizing there's like massive amounts of work to be done around, around safety and around risk around. Um, I, I watched, um, I don't know if you ever watched this Battlestar Galactica, both the original, but also the remake with James Edward Olmos. And, you know, he's grizzled Adama and he refuses to upgrade the Galactica's network to the modern standard that the rest of the fleet uses. So he and, and the Pegasus, as we discover, so two seasons later, are the only Battlestars to survive the Cylon onslaught because they put a worm in the system and they, they sort of turn it off, right? So, so we need to have kind of governance mechanisms in, in place upfront that allow us to

• 1:29:55Speaker 00

  observe and monitor the kind of the risks that you've identified. But a, a decentralized economy and decentralized hazard, hazard is an emergent property, a way of keeping things in check. I, you know, I think, I think there's a kind of, there is a homeostasis that emerges or a dynamic equilibrium that emerges.

• 1:30:12Nathan Labenz?

  I think that is probably the most likely outcome. And so in that sense, I'm also, you know, reasonably optimistic. But I do think it is worth really taking very seriously the idea that either with certain thresholds being passed, certain kind of feedback loops that could be, you know, triggered that are not yet triggered, things could change.

• 1:30:35Speaker 00

  I, I just wanted to ask about that because you have the advantage of having played with the untrained GPT four. So you, you got to see, you know, GPT four in its, in its Darth Vader phase, rather than in its, you know, re reclaimed Anakin phase. As a smart guy, right? Who understands technology, when you were playing with it in this, in this way, what were you, what were you feeling?

• 1:31:01Nathan Labenz?

  Aw, for one thing, you know, just shock and awe that this exists a lot sooner than I expected it would, you know, it always felt kind of like science fiction, even when I was with Tech Da Vinci two and doing task automation and fine tuning that model, you know, as of the summer of 2022, I was very plugged in and, you know, putting points on the board for Waymark on a regular basis with, you know, a new fine tune model that could do this task a little better and, you know, improve our pipeline or whatever. And still, it was just such a dramatically that I was like, really taken aback by it, mostly super excited about it. But then I would also say the big kind of safety lesson from that experience is that

• 1:31:42Nathan Labenz?

  the control does not happen by default. And there's many ways of even conceiving what control could or should be. So this was under control in the sense that it was totally helpful and totally aligned to what I was doing. I never saw any Sydney like behavior from GPT four early, you know, it never turned on me. It always a hundred percent helped me with whatever I was presenting it with. But I do feel like we have this kind of broad divergence between the capability of the systems and our ability to really control what they're going to do or how they might be used. At this point, I wouldn't say we

• 1:32:23Nathan Labenz?

  have anything to worry about yet. You know, I don't think we have anything concrete that looks to me like the AI could run away on its own. And I did probe for that, you know, in my red teaming, one of the things I did that didn't really go anywhere and kind of led me to the conclusion that like this model is probably fine to release. And I did, you know, my final report to them was like, I think you can release this. It's as far as I can tell, it seems like it will be safe. I would also though, flag that there does seem to be a divergence between capabilities and control. And the reason, you know, the sort of experimentation I went through there was setting up, you know, one of these kind of early agent systems. I was kind of doing it on my own. I didn't have a lot of reference points. But I basically just said, you know, if I give it a high level goal, can it break

• 1:33:06Nathan Labenz?

  that down? Can it self delegate to, you know, pursue that goal? Can it encounter errors and auto correct and whatever? And it was kind of like, conceptually, yes, but practically not really, you know, it. Could it always understood seemingly the goals? It would try to break them down. It was able to understand the concept of self delegation effectively, which of course now, you know, we're pretty familiar with, but it just wasn't that capable, you know? So it was like, it couldn't go out and do a long series of things on the internet or whatever, without just getting bogged down somewhere and getting stuck. I always kind of come back to the apparent divergence between capability and control. I have not really seen anything yet that makes me reverse my thought on that. I would love to see it, you know?

• 1:33:50Nathan Labenz?

  I kind of looking for things from like the OpenAI Super Alignment Group that may suggest that we've, you know, changed that dynamic, but I haven't seen it yet. And, you know, there's just a lot of different ways that something could be aligned or trained or whatever. And we don't even have really a great paradigm yet for like what that should be. There isn't even yet really agreement on what good even looks like, you know, in terms of what we would want an AI to be willing to do or not do. So, you know, we're just, I think we're kind of into uncharted territory. That's, that's maybe a good summary of how I felt. We're in uncharted territory.

• 1:34:26Speaker 00

  And you're lucky to have got that close in on those, you know, those early moments when you see the unvarnished products. I mean, I would break, break out a couple of, you know, ideas. One is that, you know, connecting, connecting these things to tools in a, in a non SaaS environment, right? So OpenAI stuff is all SaaS. And for the next few years, at least, there's a, there is a metaphorical red button that someone can, can use to kill a rogue process just with any Unix system. But with, with open source AI and some of these models are getting sufficiently capable. I don't know what you run on your laptop, if you're running one of the Mistral models or, or something. I run one of the Mistral models and, you know, it's, it's, it's pretty good.

• 1:35:07Speaker 00

  I pretended that when I was on the Eurostar on a plane, it would allow me to continue to, to work. But in, in reality, EU just may as well get to your destination and then use. Perplexity or GPT four, but of course it's plenty good for for task automation. It's plenty good for some of these, those basic behaviours that we saw in those early agent systems and and those things are out, are out in the, in the wild. And so I think what they do is they really expand the, the number of threat vectors that our existing systems face. Now, this is so much more prosaic than, you know, capability explosion. But I just feel that with, with anything that is, that is running on a data, a data centre managed by on an Azure cloud, there are many things you have to do before you

• 1:35:49Speaker 00

  fly an F twenty two and drop a JDAM on it to, to, to stop it. But, you know, we saw script kiddies build botnets and have seen them do that for, for a decade or so. And the, I think the cybersecurity risk, which is capable enough models, and frankly, you were doing task automation with Da Vinci two, you can probably get something better than that running on a smartphone now in quite a small payload. And we're learning that we can get these payloads.

• 1:36:16Nathan Labenz?

  Probably a three billion parameter model could do a lot of the tasks I was doing.

• 1:36:20Speaker 00

  And I wouldn't have noticed because it was snuck into a YouTube video download or, you know, whatever else it happened to be. So then you, you do get to this world of, you know, many, many systems that can talk to each other, that can execute tasks for, I think, suspect naively complex DDoS, right? In i- initially. And that, I think, feels to me like it is more of a approximate risk. And it's one that requires that combination of infrastructural players, right? You need Matthew Prince at CloudFlare and you need Satya at Microsoft and, and so on, because they, they own so much or control so much of the, the, the infrastructure. But it will also need new classes of new disciplines, right? What is the security architecture of our devices?

• 1:37:05Speaker 00

  How can we stop them when they start to go, go rogue? And we think about how rapidly NotPetya spread. And that was before, you know, you had, you had systems like this that could be a little bit more clever. But so, so I imagine that that, that is something that. Seems again like a present risk that will start to manifest itself over the next couple of years. And I mean, I speak to some of the cyber sec guys and they are obviously thinking about what are the tools that you need to to defend and, you know, from these types of things. But but I get again, I I still struggle with the models that take us to to run away.

• 1:37:46Speaker 00

  If only if I'd taken take us back 200 years or yeah, a couple hundred years ago, and I'd said it's, you know, 1824. And, you know, the White House has just been burned in the war, burnt down. And I said to you, you know what, Nathaniel, in 200 years, you know, you will be 100 times richer than you are today. You'll be richer than the richest man in the whole of this continent for United States. And you will have these capabilities and and things that wouldn't have even sounded like science fiction, because science fiction didn't exist at the time. You might well have, if you'd been able to believe me, say, well, surely we'll be at utopia and all problems will have have emerged disappeared. And we've run this tape before because we actually did get there and not every problem

• 1:38:30Speaker 00

  disappeared. There's been a lot of progress. And I do, I do also think there is something in kind of human psychology that has us looking at moments of change like this and believing that certain paths are possible. And we don't look far back enough to say, well, our forebears really felt felt the same. And I kind of feel that with not so much the climate crisis, which I think is is difficult, but I do slightly feel feel this with with AI systems, because it feels like we're running that tape again. Now, just to add to my own confusion, I also see the power in the logic that says, number one, is it possible for us to engineer intelligence,

• 1:39:12Speaker 00

  right? Or is it something that comes soulfully from a mystical, superstitious force? It's possible to engineer it. I think you're a scientist. You probably believe the same thing. So number two, if we can engineer it. Is there any upper limit to what we can engineer? Well, no, there isn't because we regularly engineer machines that are more capable than us in different ways. So number three, if we can do that, can we guarantee that it will be aligned with us? And of course, I don't think we can yet. I don't think we have the science yet. So I find that logic is really persuasive. It's hard to to pick holes at, except when you start to say, well, what are actually the underlying assumptions for each of those steps?

• 1:39:53Speaker 00

  And what is uncertain about what each of those steps and what are, where are their points of control for each of those steps? So I sort of, I do agree that if you could magic up an incredibly powerful IQ 10,000, agentic with actions AI tomorrow, there would be issues. Let's just call it that. There would be issues. But but when we're not, we're not going to to do that, what's going to happen is that we've got to go step at a time. And at each point, there's research, there's development, there's stuff that we didn't understand. There are limitations. You talk about the kind of logarithmic scale of of inputs, right, that that is is slowing down. We're running out of data as well for training these models that

• 1:40:36Speaker 00

  play into what ends up being being real. So I appreciate the logic, but I also think the reality has um unpicks into a lot of discrete steps around which you have to start to make progressively more extreme assumptions.

• 1:40:51Nathan Labenz?

  It does seem that we are in, as Sam Altman has started to describe it, the short timelines, slow takeoff world. And I think I agree with him and it sounds like you probably as well, but that is probably the best case scenario because we do want the benefits, you know, now and because we probably do need some time to adjust. If you were to tell me that this is going to take 20 years, you know, or 15. And you know, it won't be until 2040 that we'll have, you know, a sort of human scientist level AI that's capable of, you know, prosecuting a long-term research agenda and coming up with meaningful new discoveries and whatever, then I would be much more confident that we will have

• 1:41:35Nathan Labenz?

  the ability to adapt to that over that timeframe. But I'm not sure about that, you know, I see enough stuff now and I hear, you know, you, I don't know if Q star is real or not real, or if they're red teaming it, you know, in a bunker somewhere right now or not, but it does certainly seem still plausible to me that there is, you know, another kind of paradigm changing moment that just creates another kind of step, you know, step change discontinuity in terms of capability that could get us there like way before we're ready. So one of the things I'm watching for a lot these days is basically the, you know, transformer, initially you used to think about it as transformer successors, but now I kind of think of it more

• 1:42:20Nathan Labenz?

  as more likely anyway, as transformer complements, things that allow an AI system to do the things that the transformer does not do well. And one of those things is, you know, managing long contexts and kind of staying on task and online learning and, you know, integrated memory and so on and so forth. There's like a, you know, decent number of things that are pretty obvious that they don't do well, to going back to the original, you know, tale of the cognitive tape, right? You can look at all the places the human is clearly superior to the transformer and start to look out for architectures that might change that dynamic. You know, if you said, how many meaningful breakthroughs are we away from the AI scientist that can produce eureka moments at a pace faster than human scientists tend to?

• 1:43:04Nathan Labenz?

  It doesn't feel like it's that many, you know, I would say it's probably more than zero, but it's probably less than four. So somewhere in the kind of one to three range, you know, because there's just not that many dimensions on the cognitive tape, the tale of the cognitive tape yet where, where we're like all that much stronger, you know, there's a few, but I kind of put it in that one to three range and with the inputs going exponential, including the, you know, just number of humans that are working on this and, you know, the number of papers they're putting out. And the number of GPUs, you know, and unclear to me also if we're running out of data. I don't really know about that, but like synthetic data seems to work for a lot of things.

• 1:43:40Nathan Labenz?

  And there's also just more modalities, you know, we certainly have not taken advantage of like just all even just think about how much security camera footage there is, you know, that it's like, we really just want to go big to go big. Like there's a ton sitting out there and.

• 1:43:53Speaker 00

  The data issue is a speed bump. It'll get dealt with by accessing repositories that are available that we haven't touched or improved synthetic data. And as you say, modalities between zero and four probably doesn't seem unreasonable. I had this conversation with some senior people in some of the different foundation model companies and they say sort of similar things. I think Shane Legg, co-founder of DeepMind is probably at the lower end of, you know, lower than four from a conversation I remember him having on a podcast. The question, I suppose then the question is, how long does each one take? Transformers took not particularly long. I mean, it was really a couple of years before GPT one and two years before GPT two, actually.

• 1:44:39Speaker 00

  And then three for GPT three. And it doesn't take long in the world of archive, but the discovery does take time and where that discovery is takes a moment. And in amongst all of that, though, is still, there is, there is still that stage that goes from the software capabilities coming together because we have the know how and we've plugged it all together. And it iterating to a system that presents a control problem to us. And in the case of the AI scientist, that is a system that we can't call Kevin, the CTO of Microsoft and say, can you just shut down the Austin data center?

• 1:45:21Speaker 00

  And as you know, for a second, because the AI data scientist has gone rogue, right? The kind of in extremist mode and that path to me also seems unclear. And there are a whole set of risks and downsides that emerge well before then. Which I think helped give us the infrastructure to to deal with that scenario. So um that and that is really, you know, how do you deal with the the bad actors of for people use it with G P T five quality L L Ms on their smartphones in three or four years' time? Uh, and we will, we will deal with it, right? In the same way that, you know, if I'd said to you in nineteen ninety four, I don't know how old you were, I was twenty two, that by two thousand and twenty four, there'd be a hundred

• 1:46:03Speaker 00

  and twenty billion identity attacks per year just on Microsoft. I I wouldn't have believed you, I would have, I mean, yeah, Kurtzwell, whatever, right? I just wouldn't have grogged that number. So we'll have this unseemingly large number of of attacks coming mediated through L L Ms and in botnets and elsewhere, and we will have developed systems to to deal with them. And that will be part of the fabric that we can't picture right now into which this A I scientist will get will get developed. And that's why these things become so very contingent. So the wrong thing to do is to say, well, because it's going to happen, let's do nothing because then it won't happen. Um, I think the right thing to do is to start to explore these ideas and have these these conversations.

• 1:46:46Speaker 00

  But one of the things I think is really problematic, has been problematic, has been the conversation focusing exclusively on existential risk, um, which it really, I felt it did in twenty twenty three. What it does is it diminishes public trust in technology. It it forces policymakers to make decisions that may not be, you know, well informed, they may not be pro innovation, they may not even be pro safety, right? There may be a bundle of really terrible spots. And I think a little bit of a great Liu Chichin, he's a science fiction writer, this Chinese guy who wrote The Three Body Problem, but in his book of short stories, The Wandering Earth, there's a moment where they have to rocket Earth, space nineteen ninety nine style, out of the sun's orbit to prevent some calamity.

• 1:47:30Speaker 00

  And it's going to be a multi, multi generational journey to the to the next planet. And in order to do this, people have to live in really terrible conditions, apart from the scientists who are. Keeping everything monitoring, looking for the signs, the planning, the the process of decompressing everything. And of course, the people get lose trust in the scientist and in true Lucid Chin style, sorry, there's a spoiler, the people rebel, kill the scientists, and then lo and behold, the next day, they show up at their destination. And trust is really, really critical. And I don't think we did a lot to get people who are outside of the tech industry to put trust into technology, put trust into their ability to participate

• 1:48:15Speaker 00

  in it and to have some agency in where it goes, to be excited about it, you know, and off the back of, you know, all of the the sort of polarization and the the the the sometimes legitimate, sometimes not scaremongering around phones and social networks and so on. It it didn't feel like it added to the to the discussion of trust. So I was quite happy when I went to Davos, this World Economic Forum meeting, that the conversation had moved from is it, you know, what kind of munitions should we use to to drop on a data center, to what are what are real pathways, what is a science that we need to do in order to make these things safe in the long term? What is a kind of appropriate regulatory interventions? What do we do about things that are, you know, approximate three, five

• 1:48:59Speaker 00

  years around misinformation and and and cyber threats, while while still recognizing that there is a pathway that you've described that needs to be addressed?

• 1:49:08Nathan Labenz?

  I I find it easy to empathize with basically every AI perspective, from the, you know, enthusiast to the ex-risk concerned, to those that are, you know, screaming about poor use of face match technology by police departments. I mean, really, the whole thing, I think, is like, very, it's all valid, in my mind.

• 1:49:27Speaker 00

  What's really exciting, and what did not happen with the mobile, didn't happen with PCs, it did not happen with the the first mainframes, is that

• 1:49:37Speaker 00

  You know, technology is is an intimate part of what it is to be human, right? Technology is our compounded knowledge. Technology is is the the binding factor that enables a world in which we then have our human relationships. And to have so many discussions about a technology early on, when it's just in its early adoption phase, we're not too late to it, I think is incredibly positive. And and I'm glad that you have a big tent show. I mean, I have an opinion about sort of X risk, but I still also feel I'm big tenting. But I'm just really, really glad that we're having a wide and extensive conversation, one that feels wider and more extensive and and kind of more grounded in some ways than than any conversation we ever had about the internet back in the early 90s.

• 1:50:21Nathan Labenz?

  Yeah, in some ways, it's funny. I think in some ways the discourse is getting a little bit more deranged over time as, you know, there is some polarization and kind of ideological entrenching happening in some places. But then in other ways, I definitely think it's getting better, if only because what we're actually dealing with is becoming a lot more clear.

• 1:50:43Speaker 00

  There is a lot of room for commonality, for common ground, and for a recognition that there are different pieces of work that need to get done by by different people. And and actually, there should be, there should also be enough money in the tank to be able to do it, right? This is a rich industry. It spits out a lot of profits. We should be able to, you know, fund it, fund it some way. Again, when you you see the kind of things people say about each other on Twitter, and then you meet them in person and they have the same conversation and it's it's just a it's a more measured measured space, just in in my my limited experience of it all.

• 1:51:19Nathan Labenz?

  I think it's kind of everything everywhere all at once. You know, it's like, yes, there are definitely things that are quite unhealthy and I I do not like to see enemies lists getting published by leading technologists. That's like, you know, the techno optimist manifesto from Andreessen, you know, has a has a section that is literally called the enemy and names, names, if not individual names, at least like specific and relatively identifiable groups. So I don't like that. But at the same time, you know, I made some noise about OpenAI and, you know, kind of. Could have easily been retaliated against by them. And I can imagine a lot of companies that might have come down on me hard and expunged my name from their case studies on their website and all that kind of stuff.

• 1:52:04Nathan Labenz?

  And they didn't do any of that. And so I do think there's also aspects, and I feel pretty fortunate. And this is one of the kind of concluding questions I wanted to ask you is like, I think in some sense, this is like a collective responsibility. We all have to reckon, we all have to orient ourselves to it, get familiar and try to figure out what's it mean for us and what can we do to to shape it in a positive way. It's definitely not all somebody else's problem. But at the same time, there are these like leading developers who clearly have outsized influence, outsized power. There are also key decisions that are getting made around like, are we going to open source Lama three or are we not? Sounds like we're going to. And then there's like government that can potentially say, hey, we require off switches at data centers.

• 1:52:49Nathan Labenz?

  I'm not sure data centers have off switches right now. You might be able to go in there and start hacking at them, but is there an actual easy off switch in most of them?

• 1:52:56Speaker 00

  No, I don't think there is. I mean, they're designed to be resilient.

• 1:52:59Nathan Labenz?

  Yeah, the opposite, right? Yeah, they're not so, and they don't probably want some rogue employee either to like go in and turn it off, right? So they've probably engineered away from anybody being able to easily turn it off. So government may have a role there to play that's like, look, we need off switches. We hope we never have to use them, but it seems like we might want to have them. So who do you think kind of bears the greatest responsibility? Or where do you think we should be investing our trust? Is it these leading companies? Is it auditors that could be independent groups? Is it the government? I mean, it's probably some mix of all of the above, but what are you kind of bullish on in that regard?

• 1:53:42Speaker 00

  I mean, the US has been an incredibly successful democracy for a long time because of separation of powers. And structurally, that works. The, a company, however.

• 1:53:55Speaker 00

  Good it is, however well intentioned the CEO is, will end up with its own ambitions and directions. And so, you know, you, you, you will always need to push. If the company's offering you three, demand six. That's just, that's just good, good practice. So I think that each player in this circuit has to be, have the right capabilities to have the right conversations. And I think one of the things that we can learn from the experience of the FAA and Boeing is that you cannot deplete your own capabilities and ask for self-regulation because it just doesn't work out, however well-intentioned that the firm is. So I think what you need to do is, we need to invest in the capabilities of, of

• 1:54:37Speaker 00

  governments to ask good questions and engage well and overcome all of the complexities that exist, that you can make 10 million bucks a year as a, as an ex at OpenAI and, you know, you, you won't do that in government. And I think that that also raises the, the value of investing in, in academia, research and civil society. So Joshua Benjio, for example, is running a really important project. I think it's based out of the UK, which is a sort of core science project to look at some of these, these risks and these evolutions and unanswered questions around control. We need to really, really start to, to, to level up. And I don't think it will be sufficient to just allow the big firms to do that and

• 1:55:20Speaker 00

  insist that they spend the money as directed by them. You know, I think if they have, if they're willing to put money into it, it should go into pots which go to grow the capabilities of the people who will keep them in check. And the reason that works is that, you know, the car industry is really successful because someone mandated that cars needed to have brakes. Now, without brakes, people wouldn't buy as many cars as they do. And I think this is good for the industry. And someone needs to understand within government, within civil society, within academia, academia, what are the right questions? And so what are the right interventions going to look like? And we can all agree as grownups that companies, however well-intentioned they are, will always have their own, their own agenda. We just, we just acknowledge that and we

• 1:56:04Speaker 00

  all move forward in a, in a generative. Critical constructive constructive way. So whichever player is weak at this table needs to have some support to become become stronger. And that probably right now is amongst governments and regulators and academia, rather than the big, the big few tech firms.

• 1:56:22Nathan Labenz?

  That might be a great note to end on. Anything else you want to touch on or cover that we haven't got to?

• 1:56:28Speaker 00

  It's really easy. It's such a facility in having the conversations with you. And I really was so excited that you agreed to do this. You know, I thought the, the murder mystery, which is you and your, your red teaming show was just brilliant as well. So I know that you've got another hat, which is suspense. And I look forward to the next episode of that.

• 1:56:50Nathan Labenz?

  Well, thank you very much. I really appreciate that. And I appreciate your time and participation in this as well. Azeem Azhar, founder of The Exponential View. Thank you for being part of the cognitive revolution.

• 1:57:01Speaker 00

  My pleasure, Nathan. Thank you.

• 1:57:03Nathan Labenz?

  It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please don't hesitate to reach out via email at [email protected] or you can DM me on the social media platform of your choice.