AI's Biggest Problem May Be Mathematically Impossible to Solve | Stephen Wolfram

Explore API-driven IT advisory with CIO Analytics, a single platform to run your IT department that brings together your data, AI-powered analysis, and expert guidance to help you improve IT performance. Check it out here: https://www.infotech.com/cio-analytics?utm_source=youtube&utm_medium=social&utm_campaign=cio-analytics-youtube-organic As AI reshapes work, help your team build the human judgment and practical skills to adapt with McLean & Company’s Human-Centric AI Program here: https://hr.mcleanco.com/training/programs/human-centric-ai?utm_source=youtube&utm_medium=social&utm_campaign=digitaldisruption What if we can never fully predict or understand AI and need to rethink how we use it? Stephen Wolfram, creator of Mathematica and Wolfram|Alpha, explains why artificial intelligence remains fundamentally unpredictable and how we can make it useful despite those limits. Stephen joins us to discuss large language models, AI-generated code, and the difference between generating convincing answers and performing precise computation. He explains how combining generative AI with computational tools can help bridge that gap and why understanding what you’re building matters more as automation advances. The conversation also explores how AI could transform cybersecurity, powering both attacks and defenses. Stephen considers how these increasingly complex interactions could outpace human understanding and what that means for oversight and AI transparency. He explains why safeguards need to operate outside the AI itself to limit the consequences of unpredictable behavior. Follow for more conversations about AI, technology, innovation, and digital transformation. In this episode: 00:00 AI beyond human understanding 01:29 AI hype vs. long-term potential 05:35 ChatGPT and natural language interfaces 06:47 Combining generative AI with Wolfram Language 08:41 CIO Analytics: API-Driven Advisory 10:00 Understanding the world through computation 13:17 How Simple rules create complex behavior 16:15 Human-like AI vs. precise computation 21:08 Why AI struggles with 100% accuracy 22:39 McLean & Company’s Human-Centric AI program 23:37 Where AI works and where it falls short 27:17 The risks of AI-generated code 29:49 Why understanding matters more than implementation 33:51 Can we understand AI’s black box? 36:37 Computational irreducibility explained 39:49 Why AI needs external guardrails 42:08 The limits of AI control 43:25 AI-powered cyberattacks and defenses 45:03 Automation and the future of work 49:48 How Wolfram research uses automation 54:00 AI coding and the rise of small software tools 55:57 Choosing what to build when more is possible 59:58 Finding value as an AI startup 01:01:33 Leading innovation at Wolfram research 01:07:47 Bringing Wolfram|Alpha to life 01:10:48 Why managing scientific research is different 01:13:01 Building teams that think broadly 01:15:07 Matching people to projects and preserving their work 01:19:30 Advice for founders: Build something you want 01:21:03 Why CEOs need to understand their business 01:24:03 Hiring beyond the polished interview 01:26:15 Final thoughts 01:27:02 About Infotech Our links: Visit our website: https://www.infotech.com/?utm_source=youtube&utm_medium=social&utm_campaign=podcast Follow us on YouTube: https://www.youtube.com/@InfoTechRG

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:01Speaker 00

  Look, lots of crazy things will happen. There'll be more cyber attacks. The defense will be AI and the attack will be AI. There'll be a whole ratcheting up of those kinds of things up to a level that is sort of, you know, far beyond what humans can individually understand. In many areas, there'll be things where it's battle between the AIs. That will be, in some ways, disquieting to people, that there's a lot of stuff that's happening sort of at the AI layer that humans don't really understand what's going on. What was discovered in AI was essentially a wild force. The era of where you can expect to understand how the machine works inside, it's no longer here.

• 0:40Geoff Nielson?

  AI is only becoming more powerful, but are we actually capable of understanding, controlling, or even predicting what it will do? Stephen Wolfram doesn't think so. He thinks it's a fundamentally wild and untamable technology, and we're going to have to figure out how to live with it safely. Stephen Wolfram is a legend in AI and computation. He's founder and CEO of Wolfram Research, creator of Mathematica and Wolfram Alpha, and has been writing books and papers about the computational universe for over 40 years. We explore what AI can and cannot do, what we can control and what we'll have to learn to live with, and what leaders need to understand before handing over more decisions to machines. I'm Jeff Nielsen, and this is Digital Disruption.

• 1:29Geoff Nielson?

  Well, Stephen, thanks so much for joining today. It's an absolute pleasure to have you. And I wanted to ask you a little bit about, I guess, your outlook when it comes to technology and, I guess, the society around it. But before we talk about outlook, maybe it makes a little bit more sense to back up and talk about the now and where you see technology as being right now. And the reason I bring that up is, you know, you've been in computation, in AI, in these spaces for so long. And, you know, we're at a point now with transformer architecture, with LLMs, where they seem like they're truly transformational, like we're on this sort of exponential curve. But at the same time, there's so much hype around them that there's this counter narrative where people have grown disillusioned, there's a backlash.

• 2:12Geoff Nielson?

  And so I guess the first question I wanted to ask you is, do you believe in the long-term promise of this technology? Or do you think it's mostly hype and smoke and mirrors? How much more trajectory do we have here to go?

• 2:24Speaker 00

  Right. I mean, what has been discovered and invented with AI in recent years is important, useful, long-term, all those kinds of things. Is it everything that everybody's extrapolation thinks it is? No. Is it something which will be of long-term importance? Absolutely yes. I mean, it's, you know, there are, there are a whole series of kind of steps that have happened in the development of technology in general, computation in particular, even AI in particular. I mean, in, you know, if you go back 2011, 2012, it was kind of like, deep learning is actually going to work? I mean, neural nets were a thing from the 1940s.

• 3:05Speaker 00

  And, you know, in the, in the 1950s, people are like, neural nets are going to do everything. And then 1960s, neural nets are useless. 1980s, they're kind of interesting. Then, you know, 2011, 2012, the breakout moment of, yes, we can do image recognition of cats versus dogs and so on, but neural nets, wow, that's useful. You know, things in, like, for example, image recognition haven't advanced that much since that time. There was a step where suddenly that was possible and hadn't been possible before. Same thing happened with things like speech to text. And then the ChatGPT moment in 2022 surprised everybody, including the people working on it, including those of us who've been following this tech for a long time.

• 3:49Speaker 00

  Nobody knew at what point sort of fluent generation of text would be possible, or even whether, whether it would ever be possible. Turns out it's possible. Now that we know it's possible, and go back and say, how did this come to happen? We realize there's a whole bunch of science we don't know about what made it possible. We can talk about that a bit. But then, you know, then in more recent times, kind of the, yes, you can have loops where, you know, code is being generated and so on. I think the, the thing to understand is most of what's happened is these kind of steps up, together with a bunch of good engineering around these sort of core tools that have allowed them to be actually applied in different places. I think, you know, a lot of the, you know, if you look at sort of the, the core part

• 4:34Speaker 00

  of what's going on, it hasn't changed that much after its initial step. But if you look at how useful is it, it kind of changed a lot just by the fact that it's plugged into all these other systems. It's actually something that you can include in your workflows and so on. I mean, I think that the, uh, this sort of idea that we've got, you know, we've got nothing worked, then ChatGPT, then coding worked and so on, let's extrapolate that to the moon, that's kind of silly. Um, I think that there will be other kind of step moments, you know, I'm guessing there'll be one for humanoid robotics at some point, um, which will have important effects in the world, just as the current, you know, text generation, code generation thing has

• 5:19Speaker 00

  had important effects in the world. I think that, uh, you know, a lot of things that have been the traditional technology that's been built out in all kinds of different domains will continue to be valuable. This is just another tool that makes more things possible. I mean, it's like, back in the day, we had, you know, programs that had command line interfaces, and that was it. That was all you got. And then there were graphical user interfaces, and that opened up all kinds of different possibilities, different levels of usage, different kinds of users, same with mobile. The fundamental thing we got with ChatGPT was linguistic user interfaces. I mean, in fact, technology we'd built back in 2009 in Wolfram Alpha was kind of the, the

• 6:02Speaker 00

  first version of that, of that kind of idea. Um, that was, has been very successful. Uh, it's kind of aims at a, a higher level of fidelity, a, a less breadth of coverage than, uh, than what LLMs do and so on. But it's, it's something where, uh, you know, what we got, a big part of what we got is linguistic user interfaces. Very important, doesn't mean that the things behind that, uh, are all kind of, you know, what was there before doesn't make any sense anymore. It still makes perfect sense. It's just there is a, an additional kind of piece to the whole story. So that, that would be my, um, I mean, look, I think there's a lot of detail about what, where things are going.

• 6:47Speaker 00

  I mean, people say, you know, we're going to have the zero hallucination LLM, which is kind of a meaningless idea, because, you know, you can ask it to write fiction too. And the, the, the point of fiction is it's about something that isn't sort of the reality of the world. I think the thing I'd been much involved in, I mean, the technology we'd built over the course of years is our computational language, Wolfram language, whose objective is to have kind of a precise computational representation of things in the world. Whether those things are abstract things, whether those things are cities or chemicals or images or whatever else, it's a way of having sort of a computational representation

• 7:29Speaker 00

  of the world from which you can do precise computation. And the thing that's kind of emerging as a terrific workflow is you kind of use the linguistic user interface provided by modern LLM, AI, and you say, oh, this is vaguely what I want to do. It generates a piece of computational language code. And then the critical thing is, that computational language code is set up so that you as a human can read it. And so you can say, is this actually what I meant? If yes, great, that becomes a sort of solid brick that you can start building your whole system on top of. If no, well, iterate again.

• 8:09Speaker 00

  But the, the, uh, kind of, that's been, so it's kind of combining what has been, at least for us, a kind of, uh, four or five decade journey of kind of creating this sort of computational representation of the world, which is something that's all about sort of precise representations of things, together with this, you know, just do kind of what I mean, much more like kind of human, uh, sort of basic human thinking kind of approach that's provided by sort of the modern AI opportunity.

• 8:41Geoff Nielson?

  I talk to a lot of CIOs and everybody knows that running a successful IT shop is extremely hard and AI has only made it worse. You know, the drill, immature processes, a fragmented data ecosystem and an unlimited demand for IT. It all makes for an impossible job. And yet, nobody has invented a unified operating system for IT. We're literally the last function without one, until now. That's why Infotech created CIO Analytics, a single platform to run the IT department. We call it API-driven advisory. It's powered by your data, our analyst insight, and it works with your existing tools right out of the box. If you're interested, click on the link in the description to request access and check

• 9:26Geoff Nielson?

  out CIO Analytics in action for yourself. An Infotech team lead can get you started same day. Better context, better insight, better IT performance, only with API-driven advisory, only with Infotech. The notion of computational representation, you've mentioned it a few times and you know, I've, I'm familiar enough with, I guess, some of your, your research and your writings to know that this is tied to some, you know, much deeper ideas about the, the way that the world is understandable and modelable. Can you share a little bit about, when you talk about computational representation and, you know, you, you gave examples as, you know, kind of different as, you know, chemicals and cities.

• 10:10Geoff Nielson?

  What are the implications here? How do you understand our underlying world and how does computational representation get built on top of that and why is it useful?

• 10:19Speaker 00

  Okay, so there's a couple of different levels. I mean, this rabbit hole goes extremely deep. Actually, it goes as deep as anything can go, so to speak. But the thing to understand is, there's, on the one side is the depth of the rabbit hole about how the physical universe is built and so on. On the other side is what we humans understand and think about and sort of have narratives about. And kind of my goal in, in kind of, in, in sort of our computational language is to make a bridge between the kinds of things humans think about and are capable of thinking about and the kinds of things that exist in the world. Now, when it comes to, so in other words, one wants, the, the sort of, the big idea

• 11:04Speaker 00

  is to have a computational language, kind of a notation for representing things in the world in a precise way that is where, where you're, you're able to, to kind of take, well, you're, you're, think of it, you know, like, for example, you know, mathematics has a notation that God invented 500 years ago or so now with plus signs and equal signs and so on. We're trying to represent things which you can think about computationally in sort of a notation that you can use to communicate with a computer, but you can also use as a framework to think about as a human, so to speak.

• 11:45Speaker 00

  So that, that's kind of, that's, that's the objective on the side of kind of computational language. It's a bridge between humans and what is computationally possible. On the other side, it's like, okay, what ultimately is computationally possible? And that has to do with sort of how the universe is constructed. And that's a, that's a much deeper rabbit hole, which in my, in my life, these things are somewhat connected, but it's, it's a fairly long, long kind of chain. I can explain it a bit. I mean, you know, to explain sort of the, my, my life is sort of consisted of alternation between building technology and doing basic science, about five iterations so far.

• 12:28Speaker 00

  And that's been something where, you know, the technology has provided tools that have let me do a bunch of basic science. The basic science has given me ideas that let me build technology. It's been actually really amazing and powerful and it's something that, you know, I didn't, I didn't plan it that way, but it's the way things have worked out. So when it comes to kind of the, the science side of it, the, the sort of, the big question is, we've got the natural world. How do we formalize what happens in the natural world? How do we talk about what happens in the natural world? And sort of the, if you go back, I don't know, to antiquity, people sort of thought

• 13:12Speaker 00

  about it in sort of philosophical terms of how, how nature ought to work and so on. Then you get to the 1600s and the big idea was, let's use math, which had been invented for other purposes, let's use math as a way to represent the natural world. That worked pretty well. That gave us kind of the whole chain from, you know, calculus to a lot of modern physics and things like this. The, the sort of, the big thing that I realized back in the, oh gosh, how long ago is it now? Nearly, nearly 50 years ago now, is that sort of math is one particular way of formalizing things, but it's not the only way to formalize things. Computation provides a more general way to formalize things.

• 13:57Speaker 00

  And by computation, what I really mean is, you're just specifying definite rules and then you're saying, what are the consequences of those rules? You know, when we, we typically write programs where we kind of set them up for particular purposes, but you can also just say, imagine that you have some set of rules and then you just apply those rules over and over again. What happens? So this leads to a sort of very basic kind of science that I call rulology, the study of rules and what they do. And what the, the most significant thing you discover is that even when the rule is very simple, the behavior that it can produce can be extremely complicated. That's something very unintuitive.

• 14:39Speaker 00

  People, people are always pushing against that idea until they actually try some computer experiments. And then they're like, gosh, that's not what I expected. You know, it turns out even a very simple rule can produce very complicated behavior. And that turns out, I think, to be sort of the secret that lets nature do all the complicated things we see in nature. That even though the rules underneath can be very simple, the behavior is complicated. That's what we see both in physics and in biology. And the, the big excitement for me in the last few years has been seeing that really come together and really being able to derive a lot of the core theory, laws of physics and now a bunch of the core results of biology, which is something that one's never really

• 15:22Speaker 00

  had a theory of before, from, from these kind of computational ideas. That's, I would say, that's the kind of metaphysical underpinning of thinking about the world computationally. When it comes to kind of the actual operational rules that lead you to features of physics, those rules are very incomprehensible to humans. The level at which we humans can interact is a much higher level. And so there's kind of this, this thing of what is, how far can we get from the kinds of things that we can understand? And that's where our computational language is really the crucial idea that lets you get from the things humans understand to what we can get done computationally. Then there's a much deeper level, which is, given the idea of computation, how do you

• 16:07Speaker 00

  understand sort of metaphysically how the whole world is put together?

• 16:15Geoff Nielson?

  I'm glad, thank you for that. I'm glad we went down a bit of that rabbit hole because I think it's useful background as we get to now this layer of computational representation. So, you know, if we understand that everything starts with rules, can be computed, can be represented and used through AI, what, what do we do with that? Like, how do you, where would you want to actually leverage those models? How does it interact with, as we were saying, this, this view of AI that it's a language tool? Now we're talking about something much deeper and not even necessarily talking about reasoning, but actually a deeper understanding of, of the world.

• 17:00Geoff Nielson?

  Where, where do you point that and what does that, what does that help you do?

• 17:03Speaker 00

  Right, so, you know, if you go back 400 years, there wasn't a lot of sort of formalization of science and kind of the description of the world. It was like, the world works the way it does and we can argue about how it works and it's all philosophical and it's all kind of something that is done by pure human thought without kind of the solid structure of formalization. What we got in the 1600s was the beginning of sort of a solid structure of formalization and that's what's given us most of modern technology and science and so on. It's very powerful. It's different from what modern AI does. Modern AI does something that we didn't know how difficult it would be, but we sort of

• 17:45Speaker 00

  know it's possible, which is to do something like what human brains do. In other words, the, the things that are possible with formalization and with, you know, running explicit computations billions of times and so on, those are not things human brains do. Those are things that we are able to do and we learnt that we were able to do as, as kind of a, a, a formalized branch of how to, how to do things. It's different from what brains do. Brains, you know, nobody runs code in their mind. It's just not something, computers run code. Humans, you know, if you say, here's a program, what does it do? Run it in, in your brain.

• 18:27Speaker 00

  You know, that's not something anybody can do beyond a very few steps. It's, it's a different kind of activity. Now, the kinds of things that brains do well are things like, is this a picture of a cat or a dog? That had been completely out of scope for sort of the, the kind of the solid formalized computation. So one has to understand that what, what sort of modern AI is really doing is something a bit like what human brains are doing. And probably the reason, you know, when you say, is it a picture of a cat or a dog? Is there a mathematical definition of what's a, what's a cat, what's a dog picture? Not really. The, the thing that we, you know, a human can say, yeah, that's a cat, that's a dog.

• 19:09Speaker 00

  And the thing that makes us impressed about AI and makes AI useful to us is that its decision about what's a cat, what's a dog is probably pretty similar to ours. So it's a great proxy for us, so to speak. If you want to run faster, more, you know, et cetera, et cetera, et cetera, you can use the AI instead of using a human, but it's doing something which has the same kind of fuzzy, well, I don't know if it's a cat in a dog suit or, you know, is it, is it really a picture of a dog or whatever else it is? It's, it's something that, it's, it does the same kind of, sort of fuzzy stuff that brains do, which is very useful for many purposes.

• 19:49Speaker 00

  But for example, you know, the, the skyscraper in your background was not built with fuzziness, so to speak. It was built with a bunch of detailed engineering that made use of the kinds of things that have been discovered over the last few hundred years in kind of mathematical science and so on. And I think that's a

• 20:10Speaker 00

  You know, that's the distinction. There's kind of modern AI is broad, but not computationally deep. Then we have kind of formalization where things are deep, but not as broad. There's, you know, what you can formalize is what's convenient to formalize has been. That's starting to change, but that's a, that's a different story. It has been things where, which are more on the kind of objective scientific side, not the, you know, let me express human feelings, so to speak. That's, that's more on the kind of pure human side and more on the side of the kinds of things that modern AI can do.

• 20:53Speaker 00

  So I think, you know, what one's seeing is there's sort of two complementary buckets. You know, there's the precise computational formalized part, and there's the modern AI kind of human-like part. The two together are super powerful and they can leverage each other in all sorts of ways. I think, you know, the thing you also have to understand on the kind of AI side, it's like if you have a task where getting 90% success is a total win, then, you know, the AI is going to do great. If you have a task where 100% success is the only thing that's acceptable, you're going to be very frustrated. You're going to keep on poking at the AI and it's going to usually do the right thing and occasionally do something crazy.

• 21:37Speaker 00

  And by the way, that's not just a feature of we'll make the technology better. That's an intrinsic aspect of kind of the kind of thing that modern AI is. And I think that that's, you know, if you, that that's sort of a different branch from the, oh, we need to get it precisely right. And it's going to be very well defined. It's going to be less broad than what you can achieve with kind of modern AI. That's, that's the kind of computational branch. And, and that's the place where, you know, in, in things, the modern AIs are getting better. I'd like calling our technology when they need it, need it. You know, it's like hook up the MCP server. That's the beginning. And then there's lots of pre-training that's now been done to get sort of modern AIs

• 22:20Speaker 00

  to be able to use, you know, Wolfram Language and so on. And that means that the AIs can do the same kinds of things that humans do, which is when humans need to compute something, they, you know, turn to our tech and so on. They don't try and run the code in their brains, so to speak.

• 22:40Geoff Nielson?

  Digital Disruption has partnered with McLean and Company to bring you the Human-Centric AI Program. If you're a knowledge worker, you've probably noticed that your colleagues are all using AI, and it's not always pretty, but it's a solvable problem. It's not a tools problem, it's a people problem. And yet we keep training on tools. McLean and Company's Human-Centric AI Program is a judgment-first learning experience built to drive AI impact at scale. Three self-paced learning paths plus live expert-led sessions and content that evolves as AI does. Better judgment, better outputs, better outcomes. Check it out at the link below to learn more. So with all of that in mind, where do you see, I guess, the next frontier

• 23:23Geoff Nielson?

  of the applications of this technology? And part of what I'm asking is, we tend to talk about AI as though it's, you know, the next frontier in automation and it's a tool for automation. But I think it's clear from this discussion, it's not exclusively a tool for automation. There's other ways to use it and it does more than that. It's, you know, it's something that we conceive of as being similar in some ways to the human brain, as you said, it's not exactly one-to-one. So how should we be thinking about it? And what does that mean in terms of where we expect to see it applied in the next handful of years?

• 24:05Speaker 00

  Well, I mean, places where kind of human judgment was useful, there are a whole bunch of those places where you can kind of fill in some AI judgment. It'll be different, but it'll be faster. And, you know, lots of things, that's, that's one kind of place. Another kind of place is just by the fact that modern AI systems have ingested a lot of kind of the textual knowledge of the world and other kinds of knowledge of the world. It's useful for kind of pulling together sort of different threads from different places. I mean, there's all sorts of success. You know, people say, oh, it proved this math theorem. What it really did typically was, you know, there are three or four million papers in the literature of mathematics.

• 24:49Speaker 00

  It's ground up essentially all of them. And if there's something where you pull a result from this place and that place and the other place, no human mathematician will have seen all of those places. But it can, it has seen those places and it can kind of pull those threads together. And it's kind of mashing up kind of things which were sort of already human knowledge, but putting them together in a way that's very useful. And it's kind of like, you know, Web Search did a piece of that, but this is doing something much sort of more, more fine-grained and, and, and more ambitious than that. But it's the same kind of idea. It's taking existing human knowledge and being able to refactor it and use it. And that's something where lots of very useful things will come out of that.

• 25:33Speaker 00

  I mean, there are plenty of places where, and, and by the way, that, that's the kind of thing, whether it's medical diagnosis, whether it's legal precedents. These are places where that same kind of sort of taking the ground up kind of text of the world and being able to put pieces of it together is super useful. When it comes to sort of, sort of reasoning and so on, I mean, to be realistic, humans are not that good at that and nor are the AIs. I mean, if you really want a big tower of kind of this follows from that, follows from that, follows from that, you're doing computations. You're, you're running some rule a million times or something. That's not what human reasoning does.

• 26:18Speaker 00

  And I think, you know, people were very surprised when sort of ChatGPT turned out to be able to, you know, do logic. I think the reason it can do logic is because it sort of puts pieces together, sort of it knows patterns by which things go together. And that's what logic was to Aristotle a couple of thousand years ago. It was like, here are all these arguments that people make. Well, there are these patterns in how these arguments are put together. And that was kind of the original form of logic. And that's the kind of thing that an AI system can readily understand how to do. I mean, it's, it's something where it's kind of following the patterns of if this, then that, then that. If you say to it, okay, now do that kind of same logic

• 27:01Speaker 00

  as it's applied in a computer with a billion, you know, trans gates in its, in its microprocessor or whatever, that's not what it does. It does the same kinds of things that humans do, sort of following patterns of how things fit together and, and applying them. And that's, that's great and that's useful for certain kinds of things. It falls completely flat for other kinds of things. So, I mean, I think, I'm in terms of sort of the, you know, what, what do we actually see happen? We actually saw every company thought, you know, AI is cool. We've got to have an AI plan. And the, you know, initially people were like, let's just put AI into everything. That didn't work very well.

• 27:42Speaker 00

  Then there's sort of a, a realization that there are use cases that work. There are other ones that don't work. You have to try things. Depends on the particulars of what, you know, your business is trying to do. There's sort of patterns, as I say, the, the general heuristic is if, you know, if getting it right 80% of the time is what you want, then it's a good thing to try. And, you know, even if you're writing code, getting it right 80% of the time when you can't read the code is sort of a disaster. And so people have kind of said, we're just going to have all that code written by AI. And then they realize, oh my gosh, we just got drowned in, you know, AI slop code. It's going to take us years to dig ourselves out from underneath this.

• 28:27Speaker 00

  That's a feature of the fact that you just said to the AI, go do this thing. And it probably did 80% of it is right, 20% of it is nonsense, and nobody knows. And that's, you know, for example, our computational language, sort of the big, the big thing there is that the language is sort of the highest level representation that anybody has of computational processes. And so it gives you the best chance to have humans actually understand what it was that got written and say, yes, that's right. That's what I meant versus no, you, you AI missed the point and we should try it again, kind of thing. I think that the, the, the sort of, you know, what's, what's happening

• 29:10Speaker 00

  sort of society, I suppose, is that there's, I mean, there's, there's a lot of kind of overreach of many things. Like I was, you know, I was just at, seeing something where.

• 29:26Speaker 00

  People were kind of programmer type people were saying, we're going to discover math. And they were producing kind of math paper like output. But they were saying, but we don't understand anything that's in here. And it's like, this is not going to end well. You know, this is kind of, you know, and what's, what's clear is that the value of actually understanding things has gone up. The value of being able to mechanically do things has gone down. I mean, the fact is, you know, I've spent much of my career trying to automate a lot of kinds of computational tasks. And for me, the fact that people say, oh, my gosh, we can automate all this programming is like, yeah, that's what we've done for the last 40 years.

• 30:10Speaker 00

  Um, and you know, the, the pattern of, of, uh, with our technology, for example, you know, widely used by CEOs and CTOs of, of tech companies, not so widely used by kind of rank and file programmers. Why? Because the CEOs and CTOs understand what they're trying to do. They're, they're kind of, for them, it's like, and then I just want to implement it. And then I'm going to go on, figure out the next thing I want to do. For the more rank and file programmers, it's been about, give me a spec, then I'll spend the next month implementing that spec. Ask me to make a new spec. And it's like, well, that's not my job. That sounds really hard to me. So that, that particular thing of kind of rank and file programming has been, you know,

• 30:54Speaker 00

  in, in all the various programming languages and so on, has been a thing that, you know, we've, we've automated a lot of that. And the people who, you know, the CEO CTO crowd who use our tech are, you know, writing that one function that for a low level programmer would be, oh, that's a thousand lines of code that I have to custom write, so to speak. What happened with sort of AI coding is now the AI can write those thousand lines of code. And it's like, yeah, we, we'd already automated a lot of that stuff. It's great. It's useful. We use it a bunch. There are a bunch of kinds of situations in which, particularly when you're kind of gluing that sort of this thing together with that thing.

• 31:37Speaker 00

  There's a, you know, there's this thing that is in that format and it's developed in that format and it works that way. And there's this other thing that works a different way. Kind of this making the glue that goes between those things, making the kind of connective tissue of, of, of kind of, uh, technical systems. That's a place where a lot of modern AI and AI coding is, is super useful. Um, and something where in a sense, it's very shallow programming, but it's very messy, detailed programming. And it's also something where it's fairly easy to check whether you got it right. Um, which, which means if, you know, if you were in the 80% right, 20% wrong scenario,

• 32:17Speaker 00

  you can get to iterate and, and get it so it really is a reliable thing. But, you know, I, I think that the, um, the thing that, uh, uh, you know, there's going to be more value to kind of general thinking, general understanding, general strategy, less value to kind of the, let me learn this, let me have learnt this tower of ideas, uh, kind of, um, uh, over the course of many years and then apply it because it's increasingly the case that you can kind of get that kind of, uh, boost of information or that refresher or those kind of, uh, those aspects of, of that kind of thing. You can get that down automatically.

• 33:01Speaker 00

  And so the real value is, do you have a big picture understanding what's going on? It's sort of, who can tell the AIs what to do? The AIs do what they do and automate a lot of things. But the question is, what, what, what's involved in telling them what to do? And again, in, you know, in my, in my world, I've, I've spent a lot of effort kind of trying to understand sort of at a, at a more kind of thematic level, how a lot of things work, um, both in science and elsewhere. Um, and, you know, I, I see kind of computational language as the general formalizing mechanism for taking the ideas that you have that might be kind of vague and making them kind of

• 33:43Speaker 00

  precise and formalized and something that you can kind of, uh, you know, step by step build on.

• 33:51Geoff Nielson?

  That, that piece is very interesting to me and you touched on it a couple of times. The, the notion of how important it is to understand things and being able to, you know, get a little bit deeper and not just have the answer, but understand how you got to the answer. And it's, it's interesting to me because I feel like with a lot of the more modern LLMs and, and AI tools, the auditability is actually quite low, that, that they're set up in a lot of ways like a black box. You get the output, but maybe you don't understand the output and maybe they don't want you to understand the output. To what degree do you consider that a concern? Do, do we need to get better at, you know, custom building tools where we can understand every piece of it?

• 34:34Geoff Nielson?

  Is it just understanding the logic or understanding the prompt? How do we, how do we find, I guess, the right level of understanding there? And is it everything or is it just some subset?

• 34:47Speaker 00

  Well, there's a lot of things that when we succeed in formalizing things, we get to do things like build the whole tower of modern engineering. It's, uh, you know, formalizing things is very powerful. It allows you to go beyond what humans can do, so to speak, fundamentally go beyond what humans can do. In the, this question of, of can you understand what's happening inside? Well, you know, when we work with humans, we don't understand what's happening inside. You know, when you manage humans, you try and say, hey, I want you to do roughly this. You give a prompt, do this. And if you're lucky, the human will do roughly what you hoped they would do. You know, that's how a lot of the world operates and it's pretty useful.

• 35:30Speaker 00

  And insofar as that's how, you know, AI and LLMs work, it's going to be useful there as well. If you say, I want to know exactly how that LLM came to that conclusion, good luck, that's not going to work. It's, it's even at a theoretical level, that's just something you, you, you, you can't make a narrative that explains why it happened. And that's, there's sort of fundamental reasons for that. It's worth remembering that, you know, if you look at the natural world, the natural world just does what it does. We then have science that attempts to build a bridge between the natural world and the things that we understand, we have narratives for. But we don't expect to understand everything that happens in the natural world.

• 36:12Speaker 00

  I mean, when it comes to the arc of, of the history of technology, it is interesting that, you know, the, the primary means of transportation back in the day would be things like horses. People expected to know how to get the horse to do stuff. They did not expect to know how the horse worked inside. Then the industrial revolution came and people started saying, we can see the cogs, we can see the levers. You know, we understand how the machines work inside. That era has largely come to an end at this point. The era of where you can expect to understand how the machine works inside, it's, it's no longer, no longer here. Now, is that, why is that?

• 36:55Speaker 00

  Well, ultimately, I think it has a very deep sort of theoretical, uh, uh, it's a, it's a consequence of very deep sort of theoretical principle as this idea that I call computational irreducibility. So the question is, if you've got some system that's described by certain rules, you want to know what the system's going to do, you just apply those rules over and over again, maybe a million times or something, and just see what the rules lead to. And that tells you what the system is going to do. But the question is, can you expect to jump ahead and say, oh, I've, I see these rules. Now I can jump ahead and say, okay, in a million steps, they will have done this or that. Can you sort of reduce the amount of computational effort to know what will happen?

• 37:38Speaker 00

  A lot of kind of the, the achievements of mathematical science had to do with various kinds of jumping ahead that were found to be possible. The thing that turns out to be the case is it's sort of a fundamental thing that many computational, computational processes are fundamentally computationally irreducible. You can't jump ahead in general to see what will happen. So you could say, well, you know, when you can't jump ahead to see what will happen, there are all kinds of consequences. One of them is that what happens may surprise you. You didn't know what was going to happen because you couldn't jump ahead. And so what actually happens may surprise you. So when people say, you know, I'm going to, I'm going to sort of constrain my AI to

• 38:23Speaker 00

  do only things that I want and not things I don't want. Well, to constrain it to only things I want, you have to essentially be able to jump ahead and say, what's it going to do? I'm going to make sure it doesn't do the things I don't want it to do. Well, the difficulty is that's something that is impossible in general because of computational irreducibility. So that gives you a sort of choice in terms of the technology. You can either say, I'm going to have technology that I guarantee to understand. It's computationally reducible. That means you could in principle, jump ahead and see what happens, which, which sort of begs the question of why do you want the technology in the first place? It's to do things where you couldn't jump ahead to see what would happen, where the technology is actually doing something that's valuable to you.

• 39:06Speaker 00

  So, so branch number one is we want the thing to be computationally reducible so we can understand what happens inside. But then there's a limit on what it can do. And there's a lot, there's, it's, it's not doing the things that are sort of the most computationally valuable. So that's branch number one. Branch number two is we're going to do something that's computationally irreducible, like a lot of what happens in the natural world, like what happens in brains and so on. And we're going to, uh, know that some, some part of what happens will be unpredictable to us, will be a surprise to us. I think we have, you know, it's very clear, we've passed into the zone where computational irreducibility is all over the place.

• 39:49Speaker 00

  And the idea that we're going to be able to have, you know, the kind of the audit trail that tells us exactly what happened and why and so on, it's not going to happen any more than it's going to happen for human brains. And it's, it's, um, uh, you know, that's a, that's again, it's a question, you know, when you think about how do you use this technology? It's a question of knowing it might surprise you. So if, you know, if there's a thing where it really mustn't do that, you've got to put something outside of that, that sort of AI technology to make sure that, uh, you know, I don't know, the, the, the car can't drive itself into, into, you know, uh, off the side of the road, uh, for, you know, you, you've got to have other things that

• 40:32Speaker 00

  make sure of that, that aren't part of the sort of main AI loop.

• 40:37Geoff Nielson?

  I was just, as you were speaking, I wrote down, related to the, you know, the horse metaphor you were using, that it, it sounds like you're describing AI as a, as a tool that can't fully be harnessed, harnessed in the sense of actually physically putting a harness on it, right? That it in some ways is wild because of its irreducibility, right? Like, like a wild animal, that like something that exists in the natural world.

• 41:02Speaker 00

  It's amusing, because I was using this horse analogy a few years ago, and I, and I was talking about, you know, the value of putting harnesses on horses and using them to, you know, pull carriages and plows and things like that. That was before the term harness became common in, in kind of the AI world. And I don't, I don't know what, you know, what the interplay between those two things was, but I was just kind of amused that now everybody's talking about harnesses. And yes, it's, you know, what was discovered in AI was essentially a wild horse. And a lot of the value comes in putting the right kind of harnesses on the wild horse to get it to do things that are useful to us.

• 41:42Speaker 00

  I mean, it can also do things that are just purely, you know, for horse civilization, so to speak. The horses can run wild and do all kinds of things, which the horses, no doubt, find pretty interesting, but where we are like, I don't know why the horse did that, and I don't care about that. But if I can put a harness on the horse and have it pull my, my carriage along, then that's super useful to me.

• 42:05Geoff Nielson?

  There's, I think, maybe two separate ways to be thinking about the harness here. There's, if we think about frontier AI models, there's a lot of talk, as you were referring to, about, you know, harnesses in general, and making sure that we're trying to minimize risk, or, you know, if you listen to some narratives, existential risk to humanity from not knowing what these tools are going to be able to do, and, and not really having a sense of their capabilities, or, you know, where their capabilities could go. There's also a harness in the sense of just, you know, if you're using AI as an individual or as an organization, having some sort of harness of this is what I actually expect

• 42:48Geoff Nielson?

  it to do, and having some sort of guardrails. Are you, I thought I heard you say that we shouldn't, it, it sounded like we can't just expect to put a complete harness on this thing, and just say, everything is going to be reducible, everything is going to be auditable. That's not, that, that's not kind of a reasonable trajectory for us? So, so given that, what, what do we do? How big of a risk is this? And I guess, what's your level of optimism to concern about how, how we have some harness on this horse?

• 43:24Speaker 00

  I mean, look, lots of crazy things will happen. There'll be more cyber attacks, they'll be more complicated, the defense will be AI, and the attack will be AI. And there'll be a whole, you know, there'll be a whole ratcheting up of those kinds of things, up to a level that is sort of, you know, far beyond what humans can individually understand. And it will be a, it will be a kind of, in many areas, there'll be things where it's battle between the AIs type thing. And, you know, that will be, in some ways, disquieting to people, that there's a lot of stuff that's happening, sort of, at the AI layer, that humans don't really understand what's going on.

• 44:06Speaker 00

  It's worth remembering that we live in the natural world, which is full of things where humans don't understand what's going on. And yet, you know, even though we don't understand the thunderstorm, we, you know, build a house with a roof, and so it doesn't really matter that there's a thunderstorm going on outside. It's, you know, my expectation is that the same kinds of things will happen. That is, that there is kind of a, you know, a different, it's not the natural world, it's sort of AI civilization and so on, that's out there doing things. And that, yes, there are certain kinds of things where we've had an expectation about how things should work in the world, and that expectation is no longer going to be correct. Like, for example, you know, you're, you're having a conversation, is the thing you're

• 44:51Speaker 00

  having a conversation with a human? Well, that was a good assumption a few years ago. If, if, it's not a good assumption anymore. And, you know, what consequences that have? Well, probably has a bunch of consequences. But, you know, there, there are things that we have to adapt to. And, you know, it's, I think, in terms of, you know, what does the future look like for, I think, you know, I'm, I'm generally, personally, an optimist, and generally, that's, you know, I'm, I'm, you can be, if you look at the arc of human history, you know, there's lots of things about the modern world that are great compared to the way things were

• 45:34Speaker 00

  back, you know, centuries ago, or whatever. And there are also things where you could say about the modern world, we lost a bunch of stuff that was wonderful and rich and human about what was going on, you know, 100 years ago. So, but I tend to think that things will sort of adapt, and that there will be, you know, in terms of, will everybody be miserable all the time because of AI? No, absolutely not. People will be sort of, my guess is that the, you know, people will be as happy as they ever were, which means some people are happy, some people are unhappy, and that's just the human condition. I think the things that will happen, I mean, you know, one thing, as more gets automated,

• 46:17Speaker 00

  the big sort of thing is, well, what do you want to do? In other words, in the past, it was like, I picked this profession, I'm stuck in this profession, I got, you know, positioned here, I learnt a bunch of stuff about this, this is what I'm going to do. And I think one of the things that one realizes in modern times is that it's kind of, one can have much more fine-grained kinds of activities, like the very thing we're doing right now of, of, you know, podcast kinds of things, that, you know, I can kind of, you know, be, be involved with doing, you know, live streams and things like this as a small part of my life. It's not something where I've had to spend a huge amount of effort, I just, you know,

• 47:00Speaker 00

  press the button and switch it on and it goes type thing, because of automation. If it wasn't for automation, I would have, you know, that would be a huge investment to be able to get to the point of doing something like, let's say, live streaming. And I think that has the consequence for people, that more and more, you'll see people who have, you know, five professions type thing, rather than, I put all my eggs in that one basket, and that's the thing I'm learning about, and that's the, the specialized thing that I'm going to do. Now, you know, some people may say, that's wonderful. Some people may say, that's disorienting and terrible. You know, this is, I, I tend to think that, I mean, for me in particular, you know, I'm

• 47:43Speaker 00

  a person who spends their life sort of coming up with ideas and trying to make them real. And, you know, my main way of doing that is building technology, having a, you know, a company with, with 800 people or something in it, to try and turn ideas into real things, so to speak. And so for me personally, having other tools to help turn ideas into real things is, is great, because I've got, I've got plenty of ideas. Not everybody has those particular interests or, or, or directions and so on. So it's, it's a, you know, and it is complicated. Whenever automation comes in, there is, you know, there are things that people learnt

• 48:23Speaker 00

  to do, that now, sort of, the investment they put into it is, is no longer worthwhile. And, you know, what has tended to happen historically in, in an economy, is, you know, at the beginning, everybody's in agriculture, then that gets automated, then you see the pie chart of occupations get more and more fine-grained, as, as sort of the investment to get into any occupation goes down, and you, you know, people are able to do a wider diversity of things. And I think that's, you know, that's, that's kind of the future that one can expect to see, which puts a premium on sort of learning to think in general, learning about as many

• 49:05Speaker 00

  kinds of things as possible, not kind of the very specialized, I'm going to do this particular thing. There are still some fields of human endeavor where the kinds of things that we know how to do are at the top of a very tall tower. That's true in areas of, you know, math, science, and so on, where just a lot is known. And to be able to contribute, you really can't, can't do that without having climbed a lot of that tower. That's not, that's not as, that's not nearly as true in, in most other areas, and I think it's much more possible to, you know, do sort of the, the amount of gig work, so to speak, in some form, is going to expand.

• 49:48Geoff Nielson?

  Well, and it's interesting to hear it described as gig work because some of it could be true gig work and you know, you're in the food delivery business. But I was also thinking about, you know, doing five, quote, gigs at once. And I was thinking about you and the way you describe yourself because it's a, they're gigs in a sense, and it's a very specific type of gigs. But you brought up your company, which has, you know, 800 or so people. And I'm curious how you've seen your approach, I guess, to running a company and structuring a company change over the last handful of years? How you integrate automation and AI? Have you seen, have you seen it change in a big way?

• 50:29Geoff Nielson?

  Or is it just a way for, you know, some of the same roles and, you know, some of the same activities to just be done more quickly than they were before?

• 50:40Speaker 00

  We're both a bad example and a good example. Because, you know, the story of my company has been a story of layers of automation. I mean, that's what we've been doing. That's what we, that's what we've done, both for our internal purposes and for the things that we deliver to customers. It's all about delivering the technology to automate things. So we've kind of already, I've already been living the AI dream for the last 40 years. In other words, I've, you know, in the things I do, for example, in science, I've built an incredibly automated technology stack that allows me to be super efficient in going from an idea to really working it out. So in a sense, the effect of modern AI on our particular organization has been probably

• 51:25Speaker 00

  smaller than almost any other because, you know, we've already automated a lot of stuff. Now there's new things we can automate, and we've been doing that. And it's cool. Some of them have worked, some of them haven't worked so well. I think that the, you know, whether it's, I mean, a lot of things that we sort of didn't imagine doing because there were just too much human effort to do them, we can now automate them. And they don't get done as well as all that human effort will probably let them be done. But you can, we can be broader, if not deeper. And that has allowed us, I mean, both in terms of our business operations and in terms of actually some of our technology strategy, to do some slightly different things.

• 52:09Speaker 00

  There'll be some things that will come out in a little while that are fairly surprising that we've been able to do by using the fact that sort of AI makes breadth cheap. In other words, the, it's, you know, going across a lot of different kinds of activities is now much easier than it was before. I mean, we, we've done a lot of that. Our company is, you know, we build technology that's applied to a lot of different areas. But there's a certain amount of kind of delivering to sort of building out the towers you need for many, many, many specific areas that has never been economically feasible for us and now is.

• 52:50Speaker 00

  So, you know, those kinds of things have affected the way I've done things. I would say that in, I mean, I've, our company is, well, it's a very kind of ideas oriented company in the sense that we're always inventing new stuff and having to figure out things that nobody figured out before. Most, you know, not, you know, the majority of companies never have to figure out things that nobody ever figured out before. That's something, you know, people expect, you know, academia or something like this to do. We've been in a position of many, many times being sort of confronted with and choosing

• 53:34Speaker 00

  to confront ourselves with sort of things that have never been solved before. And it's kind of been a, for us, kind of my own experience in doing basic science is kind of critical because it lets one sort of think foundationally about problems in technology and strategic problems and so on. And that's been tremendously valuable to us. And that's kind of part of the, sort of the culture of our particular company. But I think in, you know, what I see with other people is sort of small utilities, sort of software utilities are now much easier to build. And they can be built by people with much less technical sophistication.

• 54:16Speaker 00

  And sort of big, deep systems, no, you will just, you know, that will be horrifying if you try and do it that way. But the little utility, I mean, I've, there've been a bunch of little utilities that I've wanted for myself for, for decades in some cases, that now just got built. You know, with, with AI programming, they're not, they're not deep, they're not complicated. They're, you know, the, the, sort of the, the thing that, you know, synchronizes this kind of contact with that kind of contact and so on. Nothing very deep. But in the past, it was like the, the barrier to doing it was just a little bit too high and so it never got done. I mean, it's worthwhile in my, in my life in, in science, for example.

• 54:57Speaker 00

  Part of what's happened is that I built technology that so automates things that an idea that is just kind of a whim, I can, you know, it's, things are automated enough that in 10 minutes I can kind of start checking out that idea. If I didn't have that automation, checking out that idea would take a month and I would never do it. But it's because it gets to the kind of do it on a whim level that things get done. And I've seen the same thing happen with, with our company and little utilities and things like this that it's been possible to build, that in some cases are really quite useful. But, you know, they were just a little bit out of reach.

• 55:40Speaker 00

  You couldn't, you know, it would take a real serious effort and you couldn't quite justify that effort. And you didn't know whether that effort would pay off. But it doesn't matter if that effort's going to pay off. If it's only going to take you, you know, a few minutes to, to do the work.

• 55:55Geoff Nielson?

  It feels a little bit like a blessing and a curse to me, that if you can now do a month's worth of development in 10 minutes. And, and I mean, the blessing side is fairly obvious, but the curse is now, I mean, almost like the, a lack of focus or the tyranny of choice, that suddenly your company can be just drowning in all these sort of potential options or directions to take. What do you do with that? How do you avoid that from being, you know, paralyzing?

• 56:26Speaker 00

  Well, look, what you're describing is the frustration that I had for years about why rank and file programmers didn't use our tech. Because, you know, for that, you know, the people who are architects and the people who are really thinking of the next idea, for them, it's a huge win to get everything as automated as possible. Yes, it's true. There are plenty of people who are used to having a kind of, you know, follow this line kind of activity. And, you know, my own view, I'm, I'm an optimist and I'm also very optimistic about human nature and people.

• 57:08Speaker 00

  And I tend to think that, you know, that, that, you know, nobody is, is forced to kind of do that kind of, kind of work. Just as we look back today and say, oh my gosh, all that manual labor that was being done in lots of countries and so on, it's like, it's terrible people had to do that. I mean, we've got bulldozers and things now. You know, nobody has to do that with, you know, in, in, in many countries with, you know, with a, with a small shovel type thing. And I think the same will be seen in the future for a lot of the kind of things that were like, you just grind through and you follow this thing and, and do that. Now the question of, okay, so what do you choose to do? Yeah, well, that takes skill.

• 57:52Speaker 00

  I mean, that's, you know, strategy is important. And I think the, you know, for example, in education system, not a lot is taught about how to figure out what to do. Mostly it's what's taught is given that you are asked to do this thing, can you mechanically do it? And I think, you know, there's, there's, I tend to think, I'm kind of an optimist about people, that sort of people are intrinsically quite good at figuring out sort of what to do. They just mostly have never been given the opportunity to think that way. And I kind of think that, you know, when it comes to sort of the strategy of what you

• 58:31Speaker 00

  should you do, it's, you know, yes, there's a certain kind of thinking about, should I do this? Should I do that? I mean, in my own life, I've made a zillion choices of that kind. And, you know, I've chosen to work on certain kinds of science, technology, business opportunities, whatever else. You know, I've had a rather rich collection of, of potential opportunities and I've chosen particular ones to do. And I think I've been pretty happy with my choices in general.

• 59:06Speaker 00

  And I think that the, the thing that is, you know, well, you know, there are, there are this question of sort of how do you decide what to choose to do?

• 59:28Speaker 00

  You know, it's still the case that executing any, any significant thing is a lot of work. Even if the, you know, things were, AI made a lot of things more automated. That just means the expectations are higher. You know, to actually deliver a real product in the world. Yeah, you know, there are cool examples where some, one, one, one guy or gal just sort of vibe coded something. And then everybody thought it was wonderful and it became a product. Usually that story is exaggerated. Usually the true story is a whole bunch of work that really built all the different corners of something and made it into a real thing.

• 1:00:09Speaker 00

  I mean, I, I think, uh, I've seen, I think more of the challenge is, I don't know. You know, I've had a hobby for, for decades now of, you know, advising startup companies and, and some, well, some companies that are far beyond startups at this point.

• 1:00:27Speaker 00

  And, uh, you know, one thing I've seen now is people saying, you know, I'm doing an AI startup. And like, we're doomed. There's nothing for us to do. The AI is just going to do everything. You know, what are we supposed to do? It's not a paralysis of choice. It's a paralysis of the AI is just going to automate everything and there's nothing for our company to do. This is basically never correct. Because, you know, that's just not, uh, but maybe it takes, you know, if you just say, I'm doing an AI company. And it just is an AI company and that's its only definition, then yeah, that's probably not going to go well. But it's kind of a thing where, you know, once you start saying, well, how is this actually going to be used?

• 1:01:11Speaker 00

  What, you know, what set of people do I understand and can I deliver something for? It, it gets very much more specific and very much more a thing where it's like, oh, it's very nice that I can do this as automatically as I can. And now I can think bigger about the kinds of things that I can deliver.

• 1:01:33Geoff Nielson?

  That, that makes sense to me and, and I like that framing of, you know, thinking more broadly than just the technology, but what it actually impacts and, and maybe who it impacts.

• 1:01:47Geoff Nielson?

  You, you mentioned earlier that your, your company is different in some ways because so much of it is focused on inventing, on innovation, on, on new things versus just, you know, being a widget factory that does the same thing every day for, for 10 years. And I always find organizations like that really interesting and, and probably in many ways, the most challenging to run. And certainly to run well, because it's just, you're not just building the same muscle over and over again. Maybe you are if the muscle is some sort of, you know, cognition or power around, you know, thinking about things.

• 1:02:26Geoff Nielson?

  But, but what are your principles that you've learned over the years that have enabled you to be more successful at, at running a company that's effective at that?

• 1:02:35Speaker 00

  Yeah, I mean, I've been running the company for 40 years now. So, and, um, you know, we've done a lot of new things. It's not trivial to have companies do new things. I mean, at the beginning of our company, our core flagship product, which has been the same product for 38 years now. I mean, dramatically bigger, you know, fancier and so on than it, than it was, but it's the same. It has the same lineage over that period of time. It's, it's something where product was very successful very quickly. And it would have been easy for the company to do nothing different from that.

• 1:03:11Speaker 00

  It took effort from me to make it do different things. And, you know, the number of times I've had to create a special projects group to do the crazy thing that I want to do, that many people think is kind of, at the company, many people think is doomed. That's happened many times, you know, create a special projects group, it does this crazy thing. Turns out it works out. Turns out it becomes a big part of what the company does. Then you do that again. And, and it's, it's usually not with the same people, actually. I've, I've tried to have the idea of having people who sort of just do special projects. But usually when they've been involved in a project that's very successful, they want to stay with that project.

• 1:03:49Speaker 00

  And you kind of have to have a, a new generation that maybe were people who were working on something quite different before, that becomes kind of the new special projects team. The new team that has kind of the, the, uh, the energy and enthusiasm to kind of do something new. And where the particular new thing that's come up resonates with them. I mean, I think that for me, it's, it's, uh, um, you know, injecting new things, not so trivial. Then the question of what you actually build, you know, I have kind of a vision for what I want to build and I push it quite hard. And people always tell me that's impossible. And I say, well, let's figure out how to do it.

• 1:04:32Speaker 00

  And then turns out that we grind it around a bit and turns out it is possible actually. And, you know, it took ideas, often ideas from me. I built up a very talented group of people who work with me and they have great ideas too. And, you know, I don't know how many meetings we've been in where kind of the meeting starts with somebody saying, you know, people have studied this problem for 30 years and haven't been able to solve it. And I say, well, we've got, you know, there's a one hour meeting, we're going to make progress on this. And by golly, we do. And, and why is that? First of all, it's believing that it is even conceivable that you would make progress in that kind of scenario.

• 1:05:09Speaker 00

  Second of all, it's having really good tools that are different from tools that people have had before to go attack things that maybe people gave up on. Because in the past, the tools just didn't exist. And people said, oh, that problem hasn't been solved for decades. We'll just give up on solving it. I think that's what happened, for example, with Wolfman Alpha was a particularly dramatic example of that where that was at a time when AI was really, uh, you know, uh, you know, at its lowest ebb. And nothing worked, so to speak. And it's like, we're going to build a, a system that does kind of question answering across a broad range of domains and so on. It's like, that's just impossible. Well, it turns out it wasn't impossible.

• 1:05:52Speaker 00

  And even believing that it wouldn't be impossible was a critical part of being able to achieve actually doing it, so to speak. So I think that's a, uh, uh, you know, the other thing, realistically, I can make projects happen when I can kind of foresee roughly what the arc of the project is going to be. And if I was just sort of coming up with, hey, here's an idea for a project. And I can't really see how that project works through, it's not going to work. And for me, it's kind of important that I, you know, most of the things we do, uh, I could in principle do them. I have great people who do them, I hope, at least as well as I would do them, maybe better in some cases. But in principle, I can do these things.

• 1:06:36Speaker 00

  It's not like, oh, we're writing mysterious code that you'll never understand type thing. And I found, you know, in the last probably 20 years, I found that some sort of the, the occasional, you know, I'm just going to dive in and solve that problem myself. Even though the problem is being worked on several layers down in kind of the management hierarchy, that that is, is actually not destructive. People might think, oh, that's terribly destructive because it means you don't trust the layers of people that you have. No, actually, the, the other dynamic is the, the, the people, the more junior people say, gosh, it's really nice that the CEO cares about what I do. And even actually isn't as dumb as I thought they were and actually knows a bunch about what I do.

• 1:07:22Speaker 00

  And, uh, you know, maybe I can learn something from them. That's a pretty good dynamic. For me, that dynamic has way overpowered the dynamic of, oh, let people have sort of, uh, agency to do, to make their own mistakes type thing. I think the, um, the thing, I mean, being able to do new things in an organization is partly a consequence of having done new things before. That is, back in the, um, in the 1990s, well, the beginning of the 1990s, company was, was sort of growing very quickly. And I had far more ideas about what the company could do than the company could possibly absorb at the size it was at. And so I decided to kind of throttle back and work on basic science for a few years.

• 1:08:08Speaker 00

  And during which time the company got itself very nicely organized and so on, and was very effective, actually very profitable at that time as well. Um, you know, just sort of steaming forward, doing things that were not, you know, we'd already had a lot of innovation. We were just sort of stabilizing things. And then when I sort of came back to, uh, to say, now we're going to do new things, that was a fairly heavy lift. In fact, the, uh, kind of the, the, the most amusing part of that was sort of, we were already a very distributed company, even in the 1990s and the early 2000s and so on. But, um, uh, so it wasn't as hard to hide, you know, 200 people working on a project as it might have been if everybody had been in the same, you know, office space.

• 1:08:53Speaker 00

  But, you know, I decided it was better to do a projects sort of off on the side, not kind of telling everybody what we, what we were doing. Because I knew that people would say, oh, that's not going to work. That's not going to work. That's a bad idea. Let's not do that. You know, we've got other places to use resources. To the credit of my management team, as soon as we had a product that was demoable, in the case of Wolfman Alpha, um, you know, gave a demo. In half an hour, everybody was like, hey, this is great. You know, we can do this and that with it. Had I kind of told them a year earlier, we're building this thing. It's not really demoable. It's kind of, you know, hard to understand what it is. It would have been a much more difficult proposition.

• 1:09:35Speaker 00

  And, um, you know, I, I, I think that's a, um, sort of, you have to believe, you know, if, if you're sort of providing the vision, you've got to sort of believe in that vision and, and push it. Even, and, and, and not get it so polarized by having sort of people who know the existing vision, so to speak, uh, kind of, um, uh, say, well, wait a minute, that's different from what we've been doing. It can't possibly be right. But, you know, there's no question that the, the driving to get to always be innovating is, and not innovating for the sake of innovating. Innovating because there's actually a good thing to do is, um, uh,

• 1:10:18Speaker 00

  Is not, is not so easy. I think for me, an important driver is I build things that I want to use. And whenever we end up in a situation where, you know, we, we're building something where I'm like, I'm not in the target market for this thing. Um, we typically try and spin those things off. Um, because I know that I can't lead that thing in a, in an effective way because I just, I don't have intuition for what's going to make it great or not. And I also don't have kind of the passion for making it happen. If, if I'm going to be a user of the thing we're building, then I really want to make sure that it's a, it's, it's built in the best way. And you know, it's, um, you talk about management of, of innovation.

• 1:11:02Speaker 00

  I mean, I've been, we have now a, a kind of nonprofit spinoff orphan institute. Um, that is doing basic science. That management problem is actually really hard. That management problem is harder than the technology management problem. And the reason is that in technology, I can more or less foresee what's going to happen in a project there, a month, three months, whatever. And with very capable people, you can kind of, um, uh, you can say, do this project. You check in every so often, but they're going to more or less stay on track. And when they hit bumps, they'll know how to navigate around them. The problem with basic science is that when you're doing research, you can hit a bump.

• 1:11:46Speaker 00

  And to get around that bump, there's no existing kind of approach that gets you around that bump. In technology, there is an existing approach, which if you're a skilled technologist, you're going to be able to, to make use of. But in science, you keep on hitting bumps where it's really, you've got to go back to sort of the basic science, know how to get around that bump. And that's difficult from a management point of view because while I'm, I think, pretty good at this point, at getting around the bumps in basic science, the, you know, junior person X working on that is really not able to do that. And even kind of the middle management there, it, it, it's, it's not as easy. It's not, it's not something where there's sort of known methods for getting around it.

• 1:12:29Speaker 00

  So I've, I've finally been figuring out kind of a management structure for basic science, which I, I think is kind of a, I don't think anybody figured that out before. So it's a, it's another one of these kind of, um, things which you have to sort of figure out from scratch. And we're getting there and, um, and we, we're able to sort of produce basic science in a much more scalable way than had been possible before. Um, and it's, uh, uh, that's been, that's been exciting to me and, and, um, uh, I, I very much enjoy that productivity.

• 1:13:01Geoff Nielson?

  I'm, I'm curious what that looks like. And maybe more broadly, I'm curious when you bring people into your organizations as managers, as frontline staff, what, what are the skills and traits that you look for? That you think are ultimately the, the most important to, for them to be successful and help with the mission?

• 1:13:22Speaker 00

  Right. I mean, different organizations work differently. I mean, most companies, you know, try as they might to not be this way are a reflection of the personalities of their founders. I mean, it's, it's just, it works out that way. You know, I, I, you know, companies I'm advising and things where the, where the CEO says, I'm going to do this, even though I can plainly see they are very different personality wise from what they say they're going to do. And like, that's not going to work. Just, you know, you've got your personality built on that. So I think, you know, in our company, uh, sort of the, uh, keep the thinking apparatus engaged at all times, which often, you know, for example, people who have a lot of technical skill, they'll be like,

• 1:14:07Speaker 00

  I can think about that technical thing that I know about, but something else comes up. It's like the thinking apparatus is disconnected at that point. Um, you know, that's the thing where sort of being able to think about a broad range of things and sort of keeping the thinking apparatus, uh, sort of engaged, that, that's one thing. I think, um, sort of having, yeah. And it's sort of related to that is kind of keeping the big picture in mind. But on the other hand, doing things that are very practical and actually work and not kind of, you know, uh, in, in our particular company, we've been able to have a company with very, very little internal politics.

• 1:14:49Speaker 00

  And people are very straightforward, you know, perhaps some people maybe can't deal with that because people are very straightforward. They say what they think, which, you know, is not always positive about, um, uh, and that's, um, in our organization, that's worked very well. Um, you know, we also from I, I sort of consider the problem, you know, there are certain kinds of people, there are certain kinds of projects we have. One of the key problems of management is to solve the puzzle of how do we connect these people to these projects? So to speak. And, you know, particularly when we're doing new projects all the time, it's like, okay, let's put together a group of people who are going to be good for that project.

• 1:15:31Speaker 00

  And you kind of think, and we have all kinds of internal data on, you know, who knows what within the company and so on. And we try and understand, you know, oh, this is a person who knows about this. I don't know them very well, but let's see if they're going to engage on this particular kind of project. Um, and, you know, fitting, fitting people into projects is, is something I, I consider as sort of a management responsibility. Another thing that I think is a important management responsibility is don't do projects you'll throw away. And so that's a thing which people, uh, you know, at our company, people care about the projects they're doing. And they're not conditioned to not care about them.

• 1:16:13Speaker 00

  And in many companies, some fraction of projects that get done just get thrown away. And that's very dispiriting to people. It makes them not want to really deeply care about their projects. In our company, we've been, you know, we don't throw projects away. Sometimes we hibernate them, sometimes for a number of years. But, you know, it's been a, a notable feature that the projects always end up coming back. And, um, it's sometimes it may take a while because the market may not be ready or we may not have the resources to deploy the project or whatever else. But people fully expect that the things they work on will be things that, um, uh, kind of, uh, remain, you know, remain valid and, and their work doesn't get thrown away.

• 1:16:54Speaker 00

  I mean, I think that the, you know, our company has had very strong R and D forever. I would say that on the kind of internal entrepreneurism side, it's, it's more spotty. And there are plenty of people where they've worked for our company. And I've said, you'll never be happy unless you're working for yourself. You should spin off a separate company or go start your own company type thing. Um, and, uh, you know, I think that it's always challenging when you have a company that runs fairly smoothly. And then you have this kind of spark of entrepreneurism inside it. You know, people are like, oh my God, I can't, I can't stand dealing with so and so. They're just trying to break out of all these systems that we have. Well, you need a certain amount of that.

• 1:17:38Speaker 00

  Um, on the other hand, you can also waste a bunch of time with people who are like, I mean, in our particular company, people who come in and like, I've got this amazing, you know, I'm, I've got all these amazing things I'm going to do. And, and people are like, yeah, let's see what happens. And, you know, sometimes people come in and they say, we're going to do these amazing things. And everybody's a bit skeptical and then they deliver and everybody's excited and very quickly on board. And sometimes they're like, oh, another one of those, you know, let's, let's watch the, um, you know, the, uh, the thing, the thing crash and burn. And sadly it, it quite often does under those circumstances. And I, I, you know, I don't know whether, I mean, I think, uh, a company is pretty good

• 1:18:23Speaker 00

  on the give people enough rope that they can hang themselves, so to speak. Um, and or not, as the case may be. Uh, I, I think, um, uh, our company is, is, um, I mean, the things, some small set of people end up working directly with me. And one of the things that's always, uh, I am a very intense, uh, sort of worker on the projects that I'm doing directly myself. And that's always, uh, I don't know, I'm, I'm much older typically than the people who are working with me. And it's always a little bit funny that, um, you know, that the stamina, I seem to be able to, to muster a lot of stamina to do these projects.

• 1:19:04Speaker 00

  And I think that the, um, uh, you know, that's a thing that, that, um, uh, in our company at least, you know, there are, there are times when there's sort of an intense phase of some project. And it's really like, can you actually muster the stamina to really get through this project? And, and I have to say that, you know, we had, we have, we have great people in terms of doing that kind of thing.

• 1:19:29Geoff Nielson?

  Well, that's great. It sounds like it keeps you young as well. I am watching the time here. I did want to ask Stephen, you know, you've mentioned that you've advised, you know, countless founders, you've spun off, you know, people who have become pioneers in their own right. For people who may be listening to this and have that spark, they want to create something new. They want to be builders. What's sort of your, your best advice that you want to leave them with?

• 1:19:57Speaker 00

  Build something you want. That's point one. I mean, don't have a theory for some industry that you know nothing about. And, uh, you know, it's like, would I want this thing? Okay. Maybe we can build it. I think, you know, if I look at, uh, lots of founders I've interacted with, very different personalities. A certain amount of tenacity is always necessary. You know, it's never easy. It takes a lot of work. You know, people who are like, oh, I'm going to have this great idea and then I'm going to hire a bunch of people and it's all going to be easy. It's never easy. And, you know, if, if that's what you think is going to happen, it's not going to work.

• 1:20:39Speaker 00

  Doesn't mean that people can't be successful entrepreneurs when they've got, you know, young families and this and that and the other. It's just a question of how much, you know, personal tenacity can you put into the thing? You know, and, and really how much, how much energy and stamina you have. I think, you know, it is, it is always surprising to me, the range of personalities that can be successful. I mean, there's a certain degree. I mean, for me, one of the things is, you know, somebody who's very technologically sophisticated. And when some business or, or personnel related thing comes up, they just switch their brains off. Drives me crazy. Um, and the more successful entrepreneurs just don't do that.

• 1:21:22Speaker 00

  It's like, there's a thing that's come up. Well, let me figure it out. And the fact that, you know, I was trained as a, you know, physicist or something. You know, well, I'm going to use what I learned from that about figuring out this personnel problem. Rather than like, oh, that's a personnel problem. I can't deal with that. I think for me, uh, you know, I, I guess I have the confidence to believe that I can understand anything anybody's telling me. So, you know, when it's like, I'm going to hire the expert and they're just going to do it and I'm going to leave them to do it. I don't do that. And I've seen that end badly in many cases. In other words, the CEO has to understand what's going on and you have to, you know, if you've got some team that's, you know, delivering technology

• 1:22:05Speaker 00

  or setting up, you know, some legal thing or doing, or doing some other, you know, doing some other thing. If you don't understand what they're doing, then it's probably not going to go well. Or it has a high chance of not going, going to go well. And that requires a certain amount of personal compliments because people will always be telling you, you know, we're the experts. Just leave us alone. We'll do, can do the right thing. It's just like, just explain it to me, you know, I'm kind of, kind of slow at this, but just, you know, explain what you're talking about. And if you get to the point where it's like, I just don't see how, you know, B follows from A. Push harder because that, that, that's going to be the weak spot where, um, uh, you know, where something goes horribly wrong.

• 1:22:48Speaker 00

  And it's going to blindside you. And by the way, the person who set it up is probably going to quit by that point. And, you know, it's like, you're going to have to understand it. I think, you know, when it comes to, um,

• 1:23:02Speaker 00

  starting companies, you know, I think people, uh, you know, I've never been sort of lucky enough to have co-founders to things I've done. Might've been nice. Um, it's, I don't think, you know, I think that's a challenging thing. I think that sort of the co-founder thing works. Okay. For a few years. As you, as the years go by, there tends to be more and more divergence between people's interests. And you kind of have to plan for that if you can. I mean, I think that, um, um, you know, whenever I see an entrepreneur who says, I know about this thing, but I'm going to hire people who do all these other things. Even though I really, really don't understand them, that's, that's never a good sign.

• 1:23:47Speaker 00

  I mean, it's kind of like, you know, the way you should think about it is on day one, the CEO has to do everything. And, you know, gradually as you understand different functions, you can find great people to do those things. Maybe they'll be better than you at doing those things. Um, and, but, but, you know, it's first you have to understand what they're doing, then you can hire for that role. I mean, I think that when I, you know, when I hire people, it's, um, unfortunately I don't do much junior hiring at this, and I don't do those interviews myself, but I, I am, uh, you know, my basic principle there is, you know, I'm going to talk to somebody. I don't think I can do an interview in less than an hour.

• 1:24:26Speaker 00

  I've given up on trying to, trying to do the 15 minutes, you know, I'm just going to get the key points because I just don't find that works. You know, what I try to do is just let me really understand this person, you know, do I get to know this person? You know, and usually in, you know, I do interviews, they're talking about whatever it is they know about. It's not me asking them a set collection of questions. It's like, just tell me about what you know about. And, you know, talk about their experiences and so on. And really, and, and then, you know, I, I kind of ask myself at the end. Sort of what, you know, given what I learnt about this person with the, you know, with the particular task that I have in mind. You know, how confident am I that they'll be able to do this thing, given what I learnt about them?

• 1:25:11Speaker 00

  And whenever I find that, you know, I realize at the end of the interview, I did three quarters of the talking in this interview. That's usually a super bad sign. It's, it's usually a, a, um, uh, but I mean, that, that's, um, I mean, the, the challenge. When you do more senior interviewing, it's actually, that's very difficult because anybody who's sort of a senior executive type person, has got a lot of polish and drilling through that polish is really hard. And, you know, for more junior people, less experienced people, there isn't a lot of polish. You, you, you know, you see what you have. And then you can understand, is it a fit? Is it not a fit? Whatever else. Um, and, uh, I would say that the, um,

• 1:25:56Speaker 00

  it's, it's more difficult and, um, and, uh, I, I always find it just, oh my gosh, you know? It's like, after enough time, you finally get to the actual person as opposed to the polish, so to speak. And then you can find out, is this a fit? Are they great? Are they terrible for, for what one wants, et cetera. But, but it's more challenging. Anyway, a few, a few thoughts about, um, um, I, I, I would say the other thing. The sort of the keep thinking is an important thing and things go wrong. You have to figure out what to do. And, you know, you have to think, you know, what is the big picture? What are we really trying to achieve? Not kind of get led into every little valley of, uh, of what, what's happening.

• 1:26:40Speaker 00

  And I'm getting reminded that I have to run along a moment here.

• 1:26:44Geoff Nielson?

  I, I, I, I thought you might, and I was watching the clock as well. Well, thank you for imparting that advice on us. Stephen, I wanted to say such a big thank you for joining us. It's been very insightful. We've covered a lot of grounds here. And I really appreciate all the insight and guidance that you've shared with us today.

• 1:27:00Speaker 00

  Thanks.

• 1:27:02Geoff Nielson?

  Most viewers don't know this, but digital disruption is developed by InfoTech Research Group, a leading advisor to technology leaders around the world. If that's not you, you don't need to care. So skip ahead and enjoy our content. But if you are a technology leader, InfoTech helps IT teams get projects done faster, better, and at a lower cost. InfoTech provides unlimited access to practical tools and expert guidance you need to execute at a fraction of the cost of traditional consulting. No matter the project, from AI strategy to cybersecurity to vendor negotiation, InfoTech has you covered. Check it out at the link below and don't forget to like and subscribe.