The Other AI Revolution: Why Predictive AI will Overtake Generative AI in 2027

Predictive AI often gets ignored, yet it may hold the real key to business value in the current tech landscape. Machine learning expert Eric Siegel, author of The AI Playbook and Predictive Analytics, joins the Digital Disruption podcast to explain why organizations should invest at least as much in predictive AI as they do in generative AI. He breaks down how predictive AI can improve real-world operations, why so many AI pilots fail to reach deployment, and where the promise of autonomous AI agents encounters reliability challenges. While generative AI captures the headlines, Eric argues that more dependable returns often come from systems that predict outcomes and improve decision-making. The conversation explores the differences between predictive and generative AI, why meaningful business value comes from deployment rather than impressive demos, and the limitations of fully autonomous AI agents and artificial general intelligence. Tune in to learn how predictive AI could make generative AI more reliable, why business leaders and data scientists need to work together, and whether AI will really cause a job apocalypse. Like and follow for more conversations on AI, technology, innovation, the future of work, and digital transformation. 00:00 Why businesses may be investing in the wrong AI 00:47 Predictive AI vs. generative AI 03:01 How large language models fit into predictive AI 06:59 What kind of AI is solving real-world problems? 08:06 The two “wows” of AI: Impressive demos vs. deployment 09:28 Build your AI strategy with Info-Tech 10:37 Why predictive AI is still delivering business value 12:32 Why most AI pilots fail to reach production 16:25 AI hype and mismanaged expectations 19:31 AI agents, AGI, and the promise of autonomy 20:44 Can predictive AI make generative AI more reliable? 23:57 Why full AI autonomy remains unrealistic 26:59 Large language models vs. human intelligence 29:21 The limits and diminishing returns of AI models 31:35 What businesses are getting wrong about AI 35:35 Bridging the gap between technology and business leaders 37:18 Why leaders need a semi-technical understanding of AI 38:59 Three questions every predictive AI project must answer 42:22 Will AI replace developers and other technical roles? 49:29 Who should own AI strategy and accountability? 53:41 Is AI creating the productivity gains we were promised? 55:49 Could future technology overcome today’s AI limitations? 59:21 How organizations can unlock more value from AI 1:01:43 Eric Siegel’s six-step AI deployment framework 1:04:42 Why AI is a business improvement project, not a tech install 1:06:02 What’s new in The AI Playbook 1:09:30 Eric’s advice for business and technology leaders 1:10:46 About Info-Tech Connect with Eric: LinkedIn: https://www.linkedin.com/in/predictiveanalytics/ X: https://x.com/predictanalytic?lang=en IG: https://www.instagram.com/eric_siegel/ Our links: Visit our website: https://www.infotech.com/?utm_source=youtube&utm_medium=social&utm_campaign=podcast Follow us on YouTube: https://www.youtube.com/@InfoTechRG

• 0:01Speaker 00

  These days, the vast majority of attentions on generative AI, and disproportionately so. This is my extraordinarily subversive statement, and I hope everyone's sitting down. Most organizations should be investing at least as much in predictive AI or predictive analytics as they do in generative AI.

• 0:21Speaker 01

  Generative AI is where the attention and most of the investment is going. So why does Eric Siegel think companies may be overlooking an equally important opportunity? I'm Jeff Nielsen, and this is Digital Disruption. Today, we get past the incredibly vague label of AI and ask what underlying technology is actually doing something valuable? Let's find out.

• 0:47Speaker 01

  Eric, super excited to have you. Thanks for being here. Maybe just to jump into it, you know, I was reflecting. There's this like bad habit that a lot of people in this space are guilty of, and I'm one of them, which is using this like umbrella term AI versus actually clarifying what you mean, right? Because there's, you know, different components under there. And so this is something that, you know, I've heard you talk about before. So under AI, what are kind of the different macro types of AI? And can you unpack those a little bit?

• 1:18Speaker 00

  Sure, it's a great question, Jeff. And thanks for having me on the show, by the way. Yeah, AI is not a monolith. It never has been. It's always sort of been an umbrella term for a collection of things. But these days, you know, without, when it's not qualified and it's general usage, it usually refers to a use of large language models or generative AI. But indeed, there is at least one other main kind of AI, which is very much still in operation, which is predictive AI. Predictive AI learns from data to predict for each individual who's going to click by, lie or die, commit an act of fraud, any outcome or behavior for which there's value for the organization to improve its existing large-scale operations. So that's the technology, unlike generative AI, that you turn to for improving existing

• 2:02Speaker 00

  large-scale operations, targeting, marketing, fraud detection, which satellite should I inspect? It may be running out of a battery. Where should I drill for oil? Any and all the largest-scale existing operations where you want to tip the balance in the numbers game that is business. We don't have a magic crystal ball. We predict better than guessing. And that's generally more than sufficient to improve those. So that turns out to be a very different kind of discipline or project than the kinds of large language model type use cases and applications, those types of generative AI projects. They're really still very distinct. And these days, the vast majority of attentions on generative AI, and disproportionately so.

• 2:46Speaker 00

  So this is my extraordinarily subversive statement, and I hope everyone's sitting down. Most organizations should be investing at least as much in predictive AI or predictive analytics as they do in generative AI.

• 3:01Speaker 01

  So I want to, I want to dive right into that. But I also want to, I want to clarify something as well. Because so predictive AI, to me, as I understand it, is almost sort of like the OG AI. Like, there was some version of this around for a lot longer than LLMs. But one of the things that I think is maybe confusing some people as well, and you know, I want to ask you about it, is it seems like more and more LLMs and generative AI are almost masquerading as predictive AI? Like, they try and plug in somewhere to something that feels like it's predictive, even if it's not necessarily predictive itself? Do you know what I mean by that? Like, it seems, it seems like LLMs are not just positioned in the game of, hey, here's a block of text.

• 3:46Speaker 01

  They're trying to get into the autonomous business. They're trying to get into the, hey, I'm giving you data or an answer business. And they're not necessarily doing that on their own. Do I have that right?

• 3:56Speaker 00

  Yeah, let me answer that and also address, I can hear in the back of my head the screams of protests from a good, a good portion of the listeners. All machine learning predicts, including large language models predicting the next word. That's what machine learning essentially does. There's different ways of using it. You can use a large language model for a predictive AI project. You can use it to predict. It's typically overkill. Maybe not, though. It can help. There's diminishing returns as you increase the complexity of the model, whether it's a large language model or one of the standard classical machine learning methods, like decision trees, log linear regression, even smaller neural networks and ensemble models. But there's diminishing returns because we can't expect to have highly confident prediction

• 4:41Speaker 00

  about who's going to click by, lie or die. Often, we're predicting for humans or enterprise customers or transactions, which is going to turn out to be, which ones are going to turn out to be fraudulent. We don't have a magic crystal ball. So we don't have high confidence, but we, we have the next best thing, which is probabilities, which is putting the odds on those numbers so we can tip the odds as we play these numbers games and treat each case accordingly. So the difference between generative AI and predictive AI used to be known as predictive analytics. It's older, but not old school. Most of its value is still largely untapped. The difference isn't so much about the core technology, but about the, the way it's being used.

• 5:22Speaker 00

  So these are two main categories of use cases of machine learning, and they're destined to remain distinct because they're inherently, there's inherently different thing for those types of projects where you're trying to target large-scale operations more effectively, whether it's marketing or any kind of risk management, insurance decisions, pricing and selection, fraud detection. It's, there's, there's an embarrassingly long, it's embarrassingly horizontal. It's an extremely long tail of use cases. And the type of sort of discipline and endeavor, when you're putting odds and then you're systematically acting on those probabilities, those predictive scores of things output by the model, right?

• 6:07Speaker 00

  So in, in sort of a technical point is when you're using a large language model, it's predicting what should the next word be? Or on that level of detail, right? The next token, which is incredible what it can do. Whereas for the sort of enterprise machine learning projects that I'm referring to as predictive AI or predictive analytics, you're predicting on the enterprise unit. So, you know, for which customer, healthcare patient, company, transaction, automobile, whatever it is. So it's on that level rather than on a per-word level with a language model. And so when we come back to this. So that was a big word salad? Or, you know, do you think that I helped clarify there?

• 6:48Speaker 01

  Well, let's, let's, let's keep, it was helpful. Let's keep down that clarification journey. Because as I mentioned before, people just often say AI, and they, they leave it there. Maybe that's deliberate, maybe it's not. But when, when people say AI is going to cure cancer or these Patriot missiles are using AI, what are they talking about there? Are they talking about generative AI? Are they talking about predictive AI? Or is there something more nuanced than that?

• 7:14Speaker 00

  You know, it's often the case that they're talking about predictive, but it's in the same conversation that started with generative. On the other hand, the two areas you just described could be either, right? There's ways in which they both can pertain. So it just depends on the very particular project, this problem you're trying to solve, the value proposition, right? Are you making probabilistic determinations of an unknown? Whether this is going to turn out to be fraudulent, whether this healthcare patient is going to turn out to have this diagnosis? And, you know, who's going to click by, lie or die, right? Or be, or cancel their subscription, right? Whatever, those are predictive use cases. And those tend to be the, I mean, prediction's the holy grail

• 7:58Speaker 00

  for improving existing large-scale operations. So I've heard you in other episodes say, hey, look, is it turning out that AI is mostly valuable for really unsexy things? And my answer is mostly yes, kind of. It's a very subjective concept, but certainly on the scale. In, in, in comparison to the way generative AI is hyped, this stuff is extremely unsexy. But to me, as someone who has been in love with the concept of machine learning for more than 30 years, and just as much loves its actual viable operationalization, its use, its deployment, its application, right?

• 8:42Speaker 00

  Not just this awesome technology. With all technology, including both kinds of AIs, there are two wow's. And we almost always get to the first wow and, and very rarely, more often not than do, get to the second wow. The first wow is the technology, the core capability, learning from data to predict, or being able to do an amazing demo with, with a large language model, a pilot, right? Sort of a, a broad proof of concept. That's the first wow. The second wow comes with deployment, operationalization, right? With the first wow, the technology, you have created potential value. You haven't necessarily captured or realized that value. And indeed, you definitely have not until you actually deploy it.

• 9:28Speaker 01

  If you're a technology leader, Infotech is a name you need to know for getting ahead with AI. Infotech helps leaders deliver on top technology initiatives. And one of our most popular programs right now is AI Strategy. We've helped hundreds of organizations build out an AI strategy and roadmap using our step-by-step methodology. Infotech's road-tested approach gives you a trusted blueprint to execute faster and avoid costly mistakes. And what you walk away with is a boardroom-ready AI strategy tailored to your needs and ready to execute. What sets Infotech apart is how we deliver. Unlike traditional research or AI tools, we don't stop at guidance. Infotech works with your team to build the strategy.

• 10:13Speaker 01

  You can do this through a focused one-week workshop, ongoing analyst support, or by using our full methodology and toolkit to get it done yourself. The outcome is an AI strategy ready for the boardroom and a team that's ready to deliver. To learn more about our AI strategy approach, check out the link below and don't forget to like and subscribe. I want to come back to something you said earlier because I think we're very close to your thesis here, which is, you know, basically what I heard you say is that most organizations are over-indexed right now on generative AI and under-indexed on predictive AI. And so I wanted to, I wanted to come back to what you were just saying and ask, like, why is that? What is it about predictive AI

• 10:56Speaker 01

  that, that has more opportunity or potential for you?

• 11:01Speaker 00

  You know, around the time this episode is releasing, I'm releasing an article on CDO Magazine, in which I make the case that, yeah, it's over-indexed, at least in terms of venture investments and R and D. But in terms of actual enterprise deployment and projects, maybe on the surface, maybe in terms of the headlines, maybe in terms of the way executives talk to the board and such, it's very under-indexed. But predictive AI is still, it turns out, very much alive. And I give a bunch of evidence to that effect. For example, when I did a broad call for speakers for the conference series I've been running for a long time called Machine Learning Week. And I didn't specify, hey, we prefer predictive versus generative. And it ended up being about 50/50. I mean, there's lots of predictive AI.

• 11:45Speaker 00

  And also just informally, like, this isn't legit data. But the buzz behind the scenes, so often we hear people saying, you know, there's a lot of pressure to use generative AI, but really the thing that's still delivering value is predictive. So even though the bad news might be that, hey, look, it almost looks like we're throwing out the baby with the bathwater, much to the detriment of the organization and sacrificing all this potential value because the overwhelming spotlight and attention is being given on generative AI. The good news is, it turns out that hasn't fully taken hold. And there's still, I mean, there is something sexy about value.

• 12:28Speaker 01

  For sure. And I'm curious, like, to use your words, where does that second wow live? Like, where are we seeing that right now? Versus where are people, you know, all talking about it, but it's kind of hot air?

• 12:41Speaker 00

  Well, to answer that question, I kind of have to answer separately for generative versus predictive. They both have this very parallel story where, you know, pilots don't go to production and most don't. But some do, right? So if it's 5% to 15%, depending, right? Then this is a moving target and there's a, there's a lot sort of conflicting data about it. It's hard to put an exact number on. But I was involved with survey, industry surveys for predictive, and that looked like it was around 15%.

• 13:12Speaker 00

  What happens is with predictive AI, so if it's, you know, if it's 5% to 15% of many, many projects, that's still a lot of successes. And predictive AI has been around for decades, right? I mean, in the 60s, they started targeting marketing and doing credit risk prediction with regression and like sort of primordial machine learning models.

• 13:34Speaker 00

  So there's been a long track record. And even if, despite its age, predictive AI still hasn't fully matured and reached that level of professionalization,

• 13:46Speaker 00

  it's still a lot of success. However, the potential is generally untapped and we could do a lot better. And it's not, on both accounts, generative and predictive. It's not a tech problem so much as an organizational one. In the case of generative, the problem is unrealistic expectations, infeasibility, and that is to say reliability. So it's very, very easy to conceive of a project, a scope, and intended use of what people often use the word agent. I feel that word isn't helping, avoid the over-promising. But whatever you want to call it, right? The system can't be deployed because even though its error rate is only 5%

• 14:29Speaker 00

  and a 95% success rate is incredible, right? Unprecedented, couldn't even have imagined it five years ago. The error rate's too frequent, so it's not viable, at least with its intended, currently intended usage of being fully autonomous. And therefore, it's going to collect dust. Nobody can operationalize it, therefore doesn't provide any value. That's generative. In the case of predictive, it's not that it's got a reliability problem. Nobody, and nobody who's familiar enough with this area or anybody who listens to the following sentence is going to expect too much because it's not about high, well, the word accuracy is already often thrown around. But what that is usually means is that it's really confident

• 15:12Speaker 00

  about both positive and negative predictions. That is to say, really confident about every prediction. It's not a magic crystal ball. It's simply a lot better than guessing. That tends to be valuable. So it's not that it's, it needs more technical reliability, but there's an understandability or complexity issue. It's not a rocket science level type of understanding. It's sort of a semi-technical accessible amount. But stakeholders need to have an understanding and apply just the right particular arithmetic to understand, hey, here are my operations. Here's how often we, you know, we fall victim to fraud in the transactions or whatever problem you're trying to solve. And here's the math that shows, hey, if we take this action in these high-risk cases, we'll be wrong in the other direction.

• 15:57Speaker 00

  We'll have false positives. Let's do the arithmetic. Let's figure out what the bottom line is. That level of complexity often eludes business stakeholders. So there's sort of a cultural catch-up. Yeah, it's already been 40 or 50 years, but, you know, we're so close and yet so far. So just to summarize, generative AI has a reliability issue. Predictive AI has a complexity issue.

• 16:23Speaker 01

  I'm, there's an angle here that I want to talk about around the hype there. And, and tell me if you disagree because I don't even know that I agree with myself right now.

• 16:35Speaker 01

  One of the things that comes to mind hearing about both of those is it feels to me like predictive AI is probably a really effective optimization tool. For any business, it can help you squeeze more juice from the same fruit. The promise that we're starting to hear about from generative AI, you know, more so seems to be like, this is going to transform the way you do business. Forget the fruit you've got right now. Like, this is going to take you from selling oranges to, you know, a smoothie stand. I've, I think I've stretched this metaphor about as far as it can go. But, but it seems to be a very different narrative around it right now. Do you buy my framing? And do you buy that that's actually true?

• 17:17Speaker 00

  You mean that that's, that it's true that that's the way in which it's over-promised and over-hyped?

• 17:22Speaker 01

  Yeah. Are those the conversations that people are having? And is that actually true about the technology?

• 17:31Speaker 00

  Yeah, I mean, look, I, I've been annoyed by hype for a long time. And in the last few years, two things have happened at once. Large language models came out and knocked my socks off, right? In fact, so to be specific, like, during my six years of PhD work at Columbia, before I was on the faculty there, I was in the natural language processing or computational linguistics group. And I never thought, I see what I, we see today in my lifetime. So I'm like, holy cow. And then yet the world is sort of 100 times more excited than I am. And I'm like, hey, whoa, it's not a deity, right? But, you know, calm down a little bit here.

• 18:13Speaker 00

  It's, so it's incredible, but the promising is saying,

• 18:20Speaker 00

  you know, is saying, is promising extreme autonomy, otherwise known as artificial general intelligence. Autonomy, as in doing what a person can do without a human in the loop. So, and that hype was always there and the potential for it and the sort of sci-fi narratives. The way predictive AI was often over-promised was it was, you would use the word highly accurate. And it would be framed in a way that implied that it was really confident about every output, about every prediction. It could tell who was pregnant, that was a target thing.

• 18:54Speaker 00

  Things like that. And we're sort of expressed in terms of, you know, it's just expectation. Hype is always a matter of mismatched expectations. So no, it doesn't predict like a magic crystal ball. Unfortunately, we don't have magic crystal balls. And no matter how advanced and complex the technology gets, we, we still won't. But even then, before generative AI, there would also be the word intelligence thrown around and they sort of these promises of, hey, it's going to become super autonomous. And that always drove me crazy. Can you imagine how I feel now when that's the prevalent narrative?

• 19:31Speaker 01

  Well, and that I completely agree with. It feels like autonomy has become, you know, the, the word of the day. And, and, you know, you mentioned AGI, artificial general intelligence. You mentioned agents. And all of these seem to be thrown around as though the capability is already here or it's just around the corner. You know, speaking about all of those, you know, I hear on a spectrum from AGI is already here. We've got autonomous AI that can do whatever a human can in execution form to actually LLMs are a dead end and they're not ever going to take us to AGI. Like, really, truly opposite ends of the spectrum. And so I'm curious, I mean, you, you've got a much deeper machine learning background than most people I speak with.

• 20:16Speaker 01

  Where do you fall on that? What, what level of autonomy are we at right now with these tools? And what's even realistic to get to in the, you know, foreseeable future?

• 20:28Speaker 00

  What level of autonomy are we on? We're at 80%. Okay, so that's a little tongue in cheek. Okay, it could be 90%, but there's diminishing returns. It's the last few percent that are almost impossible. So, and in fact, as I address your question, let me simultaneously say that, look, hybridizing predictive AI and generative AI together, using predictive AI as reliability layer, I believe that's the next killer app for predictive AI and the saving grace for this hype, right? It's this audacious promise of supreme autonomy that's generally infeasible unless you have like a really constrained like customer service agent that only can answer questions about,

• 21:13Speaker 00

  you know, 20 pages of FAQ level information and it can't conduct transactions. Like if you constrain the scope well enough, then, you know, we're very quickly getting to the point where it is feasible even for end consumer facing uses that even the most vain or cosmetically conscious large corporations has potential to deploy. But once you start allowing it to do transactions or deal with sensitive data and all sorts and sort of open the doors to all things that just high sensitivity to failure and errors, it's the viability goes out the window very, very quickly. And I believe that just from first principles, the way that that's going to be reckoned

• 21:55Speaker 00

  with is to play the odds predictively and say, hey, look, here's a high risk case. The system has predicted that this is relatively five times more likely than average, 10 times more likely than average, to go off the rails to do, to hallucinate or to divulge sensitive data or do a bad transaction, whatever. There's a lot of different things that can go wrong. Let's now escalate to a human. And that decision on the fly to go to that more expensive human in the loop, right? If it were the, if, if, if the system were to do that for the top, let's say 20% of interactions, then potentially the net error rate after doing so, even if only a small

• 22:38Speaker 00

  portion of that 20, relatively small portion, let's say only two out of those 20 points were actually grave error, but the human was on those. Now we're expending, we're, we're not reaching full autonomy. We still have to spend money on humans, but we have achieved 80% of that otherwise extraordinarily audacious promise of autonomy, a lot better than the 0% that we realize if we can't launch the system.

• 23:08Speaker 01

  Well, and your, your point is well taken because if I'm looking at that through like an actual execution and like, I don't know, business process lens, what that's telling me is that at 80%, we can still re-architect a lot of our systems, a lot of our processes in a big way that we couldn't do without these in the system, even if it's not full autonomy. And maybe if it's full autonomy, you know, you re-architect them again and you squeeze even more juice out of it. But yeah, I mean, to your point, when do we get there? So I don't know. I guess what I, what I heard, and I'm curious to play it back to you is, yeah, there, there's a lot of really good stuff here right now.

• 23:49Speaker 01

  So maybe the, maybe the full autonomy doesn't matter as much as the hype is making it out to.

• 23:56Speaker 00

  Uh, well, no, it does matter. If we could achieve artificial general intelligence, that would be amazing. And it makes for a great science fiction movie. And I love science fiction. I enjoy it, but let's get real here, folks. I mean, there's a reason that that, that promise is being thrown around so haphazardly. It's making a lot of people a lot of money. It's, it's bolstering valuations and it's, it makes me groan. It makes me snicker, right? It makes me make lots of kind of pessimistic jokes behind closed doors with my wife and,

• 24:40Speaker 00

  right? But it's really hard to address the hype without sounding negative. And they say people who address the hype, you know, sound smart and people who enable the hype get rich.

• 24:52Speaker 00

  But, but also I don't want to lose tab with the fact that there's a lot to be very positive and excited about. I'm generally a, a tech optimist and I've all, I've thought for a very long time that learning from data to predict and all the ways that that can be used is the next best thing since sliced bread, both in terms of geeking out and how cool the first of two wow's, you know, just how amazing it is, this idea of what's called induction, right? Deriving generalizations from some limited number of examples. You may have many training examples. You may have millions of them, but it's a small number compared to the number of possible situations you could ever possibly encounter, such that you derive generalizations that

• 25:33Speaker 00

  hold, that turn out to have truth about the world because they turn out to pertain to previously unseen novel cases and situations and examples. So that ability to learn from data, geeking out the first wow, that's really awesome. And indeed it is extremely valuable if you actually can get it deployed. That's the second wow. And everything I just said applies for both predictive and generative because the generative is one of the ultimate so far manifestations of that concept, right? I mean, the ability to predict what should be the next word, the next token, right? Is only possible because over the previous two paragraphs or 200 pages or whatever, you

• 26:17Speaker 00

  know, the prompt was for this next step in the basically generating of text, right? It had to derive some real sense of meaning about the language in order to do that as well as it does. And so that as it continues to do that, what it writes is coherent and more often than not actually correct miraculously, right? So it's, it's really, really, really amazing and worthy of geeking out. And yet at the same time, I do believe that the difference between what it's capable of and what humans are will become increasingly clear.

• 26:59Speaker 01

  Do you want to elucidate that a little more? I'd love, I'd love to hear more about that. Like, let's clarify.

• 27:04Speaker 00

  Well, that's, that's, that's, that's the ultimate challenge, right? And I'm, I'm constantly looking for someone that can put their finger on it.

• 27:14Speaker 00

  So it is uncanny how seemingly human-like it is, right? It seems to work with concepts oftentimes very much the same as people. It often sort of reasons about things the way you might expect a person to. So it is actually very, very difficult to clearly define what's the difference between what a large language model can do, right? And what a human could do. Now, first principles would dictate that there is a, I mean, we know there's a difference. We know it hallucinates and it makes errors that humans would never make. Ultimately, the thing that matters is the performance at the particular task at hand. So for any given project, really the name of the game is mundane. You just want to put metrics.

• 27:57Speaker 00

  You want to measure how good it is at the exact text task, how often it makes unacceptable error at that task, right? That's the thing that really matters. But if you want to get a little more philosophical, kind of trying to define the difference between a human and a large language model, large language models have only learned or have mostly learned from text, like all the written text and all the books and the whole web and everything. How much can you reverse engineer about human cognition by learning from all those trillions of examples? And the example is, I've written this much so far, what should I write as the next word? Okay, now I've written this much so far, what should I write the next word? Those make the learning cases, right? Just the same as in predictive AI projects.

• 28:42Speaker 00

  Hey, this customer had done all this and then they canceled their subscription, right? Either way, right? You've got many cases. In the case of predictive, you may have 100,000 cases. In the case of generative AI, you're learning from trillions. But either way, you've got these cases, you're learning from them. How much would that process do over all the human text ever created essentially? How much can that reverse engineer human cognition? Well, apparently quite a bit, but we're going to find that there's a ceiling.

• 29:13Speaker 01

  Yeah, and I guess the question is, is where that ceiling is. And I, I mean, your guess is probably as good as mine for that. But I think, I mean, you mentioned this notion of diminishing returns, right? So I, maybe I'll, maybe I'll ask it that way. As we progress from here on out, do you see us accelerating the progress there? Or is it, you know, going to be sort of logarithmic? And it's going to flatten as we get, as this thing gets better?

• 29:39Speaker 00

  Yeah, I think there's lots of signs that it already is doing that, that there is diminishing returns. We've sort of exhausted those learning examples. There's other feedback that we give it with reinforcement learning, sort of layered on top of that. And there's another force to be reckoned with, though, when we also consider just to what degree the world can adopt this as autonomous or as the sort of manifestation of what's the spirit of the word agent or the spirit of the word artificial intelligence, which is often encompassed by artificial general intelligence. And that other force to be reckoned with is that

• 30:21Speaker 00

  we, we won't accept errors as easily from an artifact. There's not, there's not, there's no getting around the fact that this thing is artificial. It's, it's, it does not have personhood and it does not have accountability. Right? So whether it's for a self-driving car or for any enterprise system, right, there's estimations that range from we could accept, you know, 1%, maybe up to 10% of the error rate of a human. But it's got to be a lot better than a human if it's going to be fully autonomous on a high stakes task.

• 31:01Speaker 01

  Yeah, that makes sense. It does, it does feel like it's an order of magnitude thing. It's not just like incrementally.

• 31:07Speaker 00

  Or maybe two orders, yeah.

• 31:10Speaker 01

  Yeah, yeah, one to, one to two orders of magnitude, let's call it. So I mean, let me ask you maybe a zoomed out question here. But we talked about untapped potential. We talked about the fact that organizations are, you know, frequently confused about predictive AI versus generative AI. The, the investment mix is probably wrong. There's, you know, this whole pilot to production gap that we're seeing. So to come back to the big question, like, what fundamentally is it that businesses are getting wrong here? Like, what are the big rocks that are preventing a lot more value from being created?

• 31:48Speaker 00

  I mean, there's a real amalgam of missteps that are somewhat common. And it depends on the organization. But what, I mean, in general, when we talk about hype and overpromising, we're also sort of alluding to a solutionism. So what's your AI strategy? And there's a famous quote that's like saying, what's your Excel strategy? Right? We should be trying to improve efficiencies and effectiveness and solve specific problems rather than being able, treating a particular technology as a holy grail, right? It's as awesome and amazing and, and as, as it is. And how, and however big the magnitude of that first wow might become, it's still a tool, right? We live in a world of humans run by humans for humans.

• 32:31Speaker 00

  That's not changing. These are still tools. That, that concept is compromised a little by the word agent, I think. So there's, there's, it's, it's, it's a strange thing, though, right? I mean, there's lots of very legit senior techies who know exactly what they're talking about, who love the word agent. But I would say they probably tend to not fully put their feet in the shoes of a less technical executive who then takes the word agent and runs with it, right? Which much to the, much to the pleasure of, of, you know, the large foundation model companies. So

• 33:16Speaker 00

  where, where, where were we?

• 33:18Speaker 01

  We're talking about what businesses get wrong and the major missteps.

• 33:21Speaker 00

  Yeah, okay. So, yeah, so the, so solutionism, right? We got to solve problems, right? We got to improve very concrete, specific operations or introduce a new operation that serves a specific need and that's feasible. And that doesn't just sort of get people euphoric because there's an incredible demo that, that appears to work extremely well on some handful of examples. That's not meaningless, but it's also not necessarily meaningful in terms of value. So, you know, I mean, the, the euphoria with hype that goes with any kind of tech hype, there's a euphoria, right? I mean, and there's a fear that's the flip side of the same thing, right?

• 34:02Speaker 00

  And the, the antidote to euphoria or, or existential fear is sobriety. But sobriety is not boring, man. It's not boring. We have to learn how to geek out. I gave a commencement speech at my alma mater, Brandeis, in one of the grad schools there, like, a dozen years ago. And that was my theme. I'm like, geek out, like, get it, get into it. My dad said that he got through medical school by way of getting really excited and interested in the material because it was grueling how much he had to learn.

• 34:38Speaker 00

  There's an analogy there, right? There's something really, really cool about learning from data in a way that does well on previously unseen cases, right? You've drawn generalizations automatically that hold in general. Like, that's just like a tech accomplishment. That first wow can get you pretty far. And maybe that can be enough to fill that human need we apparently have for getting sort of really excited and euphoric about it without also sort of feeding unrealistic or sort of sci-fi tropes and all this kind of stuff. So I'd say that that broadly speaking is the error that the world is making right now.

• 35:18Speaker 01

  You, you used a model there that I want to talk about briefly because you talked about having like a senior tech architect or, you know, a tech guy or gal, then talking to a much less tech savvy executive or business leader. And so I want to use this sort of, you know, dichotomy of technology leader and business leader. Because when you say geek out, I'm curious if you're saying that to the tech leader, to the business leader, both with the same message, both with different messages? Or, you know, should we actually like just explode that entire dichotomy?

• 35:53Speaker 00

  My favorite pop scientist is Neil deGrasse Tyson because he speaks with such passion about stuff that I'm definitely not an expert in. And I geek out on it. And I, and then, you know, the way I wrote both my books, Predictive Analytics and The AI Playbook, which are both focused on predictive AI, is to make it accessible in sort of that pop science mode, anecdotally driven, right? And my uncle should be able to understand it.

• 36:22Speaker 00

  And everyone should be excited about it. I mean, if people get excited about astronomy and, you know, about certain kinds of pop math and other kinds of science, they should be at least as excited about what I think is the coolest thing in technology or any STEM, which is machine learning at the core of it. There's no reason not to get excited. And in fact, there is a reason to get excited. I think that that sort of speaks to the main reason predictive AI projects so often fail to deploy because there is this semi-technical understanding that the business side, the people in charge of the operation is meant to be improved by the model's predictions. Need to get to that semi-technical understanding. It's not the rocket science. It's accessible. It, it is accessible. It's extremely interesting.

• 37:07Speaker 00

  And you only need semi-technical for any field to geek out on it. So that, you know, maybe geeking out is the ultimate antidote to hype.

• 37:18Speaker 01

  The, that phrase semi-technical is one I've, I've heard you use before and that you're advocating for. And I'm curious, I'd like to hear you defend that a little bit more in terms of the right level. And, and I'll make just briefly the counterargument, which is, is semi-technical like just un-technical enough to be dangerous? You know, and I say that having a background where I've worked with devs, I've worked with designers, and I sometimes worry like, can it create a false confidence that like you know more about something that you, than you do that leads you down a, a wrong path? Or am I just like not defining it in your proper terms?

• 37:55Speaker 00

  No, I'd say that's a misconception that often kind of exacerbates the problem. Hey, don't try to explain this to executives. They're not going to get it anyway. Therefore, in a way, it's, there's, the content is censored. But understanding and getting that, that concrete understanding on a semi-technical level doesn't mean you're assuring the need for a technical expert to also be involved. So think about it as driving, right? I don't know where the spark plugs are in my car. I have opened the hood of my car to show my kids and been like, isn't this amazing? Look at this engine. It's got all these little things, right? But I'm an expert when it comes to driving. I know momentum, friction, rules of the road, mutual expectations of drivers, you know, the way the car operates, obviously.

• 38:41Speaker 00

  So I'm an expert and that's the, in that way, that's the only way I can navigate the streets. To navigate the enterprise machine learning projects successfully through the deployment, everyone involved needs a basic, that kind of, that level of, of semi-technical understanding.

• 38:59Speaker 00

  And in the case of predictive AI, it just comes down to three things. What's predicted, how well, and what's done about it? So what's predicted and what's done about it? That pair defines a predictive AI use case or project, right? So who's going to respond? Market to them. Which transaction is most likely to be fraudulent? Audit or, or block it?

• 39:21Speaker 00

  Where's their oil drill there? Which satellite is going to run out of battery? Let's go check the satellite and replace the battery, right? As you can imagine, there's a million such pairs. Maybe not literally, but there's an extremely long tail of viable enterprise uses of machine learning. But marketing, risk management, fraud detection, those are sort of some of the biggest ones. And most common ones in that most large organizations do all three of those. The other main marketing one is churn modeling is predicting who's going to attrite, cancel, defect, churn. Those are all synonyms in order to target intervention, right? You can't give a discount meant to retain to a hundred percent of your customer base.

• 40:04Speaker 00

  The only recourse is predicting, putting odds on those most likely to leave, right? So it's that very kind of practical stuff. So what's predicted and what's done about it? That defines the project in terms of what, where you're going to try to go with deployment. And then how well does it predict? You know, people often talk in terms of accuracy. That's almost always the wrong metric. The data scientists mostly, almost universally make the same mistake, which is they only evaluate how well it predicts in terms of arcane technical metrics like precision recall area under the curve. These are very common, all the, all the statisticians and, and, and data scientists know those.

• 40:43Speaker 00

  But they need to also translate model performance into business metrics like monetary, like money, right? Like how much profit would it make? Or how much money would we save if we use this fraud detection model to decide which transactions to block? Or at least hold? That kind of thing. So what's predicted, how well, what's done about it? It's not rocket science. It totally matters to anybody on the business side running, who's in charge of these large scale operations that could be improved. And it's cool. It's interesting, right? There's a lot to geek out on there, both in terms of the core technology you're leveraging, learning from data. And what it means to like, hey, what's the, it's, it's, it's just arithmetic, but it's very particular arithmetic.

• 41:27Speaker 00

  Not rocket science as far as like, well, what it mean if I have a million transactions a week? And I'm going to take the top point two percent of them and temporarily block them and have a human audit. And you know, what would the, let's run the numbers, let's crunch the numbers in terms of the monetary advantage of doing that. It's an exercise very rarely done in a, in a sort of cohesive way. And that's why most of those projects, it's theatricality, right? The first wow, the coolness of learning from data to predict and look how well it predicts in terms of these technical metrics. Right? Has everybody consumed, like we fetishize the core technology of machine learning. And it's been like that for a long time.

• 42:09Speaker 00

  Let's take a step back and be like, well, that's cool, but let's not stop geeking out there. Let's be a little more concrete and, and talk in terms of very accessible business terms of what would it mean to deploy it?

• 42:22Speaker 01

  I wanna, I wanna scratch at something because I think I agree with you, but there's some,

• 42:28Speaker 01

  you know, there's some narratives here that I want to debunk or that I wanna potentially argue about. So what, one of them is we've seen that AI, mostly generative AI, I'll say, has actually, there, there's an argument that it's actually going to destroy a large number of technical jobs. That it started coming for developers, for technical folks, and it's going to take more and more of those jobs away. And I think that there is an underlying belief there that with this technology in hand, semi-technical to non-technical is actually technical enough. You don't need someone fully technical because just like you don't need a human spark plug.

• 43:10Speaker 01

  You can be the human driving, the spark plug, the, you know, the spark plug is the tool. The AI is the tool. And that makes me really, really nervous because the, the worst uses of generative AI I've seen in enterprises is when you have someone with limited technical ability saying, oh, I've built my agent and it can do everything. They don't have someone acting as sort of a technical harness. And you know, they don't have someone who understands, you know, to your point predictive AI. And they end up doing something that's like wildly off the mark or in some way completely tech, technically falls flat on its face. And there's just no way for them to know that until they try and scale the damn thing.

• 43:55Speaker 01

  So do, do you buy that argument? Like, do you believe that we're going to see these tech jobs get completely eroded? Or do you think that, as you said earlier, that like semi-technical is still not technical enough to do the full job?

• 44:11Speaker 00

  Yeah, I, no, I don't think that we're going to have a job apocalypse in any sector. I think that that's part and parcel to the hype. But what you just said is, to, to try to address your question, it sort of sounded to me like you're describing, hey, look, there's a problem because people are putting, are being overly reliant on, on a language model and they're like, hey, I can handle this. And then they, they therefore either don't hire somebody or they fire somebody. And in the end, it's a junior, cheaper person who's sort of working with the language model because the expertise is taken care of by the model. And you're sort of, that sounds to me like it's the kind of thing where then people start to see things go wrong and then they backpedal.

• 44:55Speaker 00

  And it, it, it, it's naturally going to avert the job, job apocalypse, right? Whereas the thing that would cause the job apocalypse is if the technology is so darn good that we just don't need the people anymore. It doesn't sound like you think that's going to happen either. Is that right?

• 45:15Speaker 01

  No, I guess, I think we're of the same mind that I don't think the technology can do these things that maybe some senior leaders, boards, you know, creators of the technology are saying that it can do. But we might have to find that out the hard way by trying to do it, by firing swathes of the wrong people, by saying, oh shit, like I'm just, I'm afraid that we're drowning in a tidal wave of stupid right now. We're, we're, we're learning the hard way, the limitations of this technology.

• 45:48Speaker 00

  Yeah, yeah, yeah. But I don't think we're necessarily learning in a terribly hard way. Like there may, in the moment, the, the, the hard learned lessons may burn, burn your fingers, but you're going to pull your hand back pretty quickly. Like it's not going to be, it, it, it's sort of a blip on the radar, especially in comparison to what many are still believing, which is that the thing is going to be so darn good that we're really, you know, there's going to be a too dramatic of an economic shift too quickly. And then that's going to cause massive unemployment and all that kind of stuff. And I don't really, so I don't, I don't tend to caution about, hey, don't automate too much. Don't, you know, or don't hurt anybody's feelings by owning up to the pursuit of automation.

• 46:33Speaker 00

  I mean, that's why we make machines is to automate, right? Any kind of machine that to something otherwise a person or an animal would need to do. And I don't, I don't, I don't walk on eggshells like, oh, we don't want people to feel like we're just trying to take over their jobs. Cause I don't think we can, like, I don't, I'm not worried that things are going to go terribly in that direction. Now, it's not, there are going to be economic shifts. There's always going to be more automation. There's always going to be improved efficiencies, but in general. So my impression is that over the last year, we kind of went through a big press cycle, a big media cycle where it was like job apocalypse, job apocalypse.

• 47:12Speaker 00

  And then pretty recently, all of a sudden, the press is super backpedaled and found the experts who's like, come on, no. Or the antidote, or excuse me, the anecdotes where it's like, well, we maybe went too far. And then we had to backpedal these enterprises. So that's sort of good news as far as I'm concerned, that that type of hype has, has simmered down greatly. I'm sure there's still a lot of people that still kind of buy it. I think that the ability, the vibe coding thing is definitely a killer. The killer app is also a subjective term, right? So there's a way in which we're still waiting for what's, when's the big killer app of large language models?

• 47:53Speaker 00

  You know, if it, it, it's certainly an amazing thought partner and way to sort of get information. A little tricky because you don't know when it's wrong unless you happen to be asking about something you already know. But you, then why are you doing that?

• 48:08Speaker 00

  But it is, you know, but in any kind of use like that, where you're interacting directly with a chatbot, you are the human in the loop. Whereas the, the hype is about automation where there is no human in the loop, right? So that level of killer app, well, when's it going to be? But if, if you're an engineer and a coder, depending on the project, the automatic coding is definitely a killer app. I don't, it doesn't get rid of engineers, it empowers them. That's the general, even I know that you had, who did you have? You had Lawrence Maroni saying stuff like that. A lot of people have been saying stuff like that, you know? There's certain things that it really removes the drudgery, but for the most part, you still need engineering

• 48:52Speaker 00

  and architecture expertise and humans and understanding the, the business problem being solved and all that kind of stuff. So even within that realm where there's something of a killer app, it's not a job apocalypse.

• 49:06Speaker 01

  I think that's fair and I appreciate the clarification and I I'm glad we could revisit some of the talk around autonomy and the difference between, you know, as we said earlier, partial or 80% autonomy versus full autonomy and the implications for for business models and for workforces. I want to come back to like the implementation piece here and getting value and you talked about, you know, something that I've had lots of conversations about, which is starting with some sort of business problem and business goal and then figuring out how you can use AI as a tool versus having, you know, a dedicated AI strategy.

• 49:49Speaker 01

  I want to press on that in terms of like, then what? And I'll tell you where I'm going with this. So saying, okay, we want to, you know, create a new product with AI, we want to cut costs with AI, like that's pretty easy to do in a non-technical way. One of the boots-on-the-ground challenges I'm seeing in a lot of the organizations I'm speaking with is this sort of, you know, this is not glamorous, but almost like a turf war between IT and business leadership when it comes to AI. And a lot of traditional enterprise technology leaders saying, hey, you know, we need to govern this, we need to control this, we need to figure out how to use it.

• 50:31Speaker 01

  And a lot of business leaders, maybe they're semi-technical, maybe they're less than that, saying, you know what, we can kind of do this ourselves, thanks. This is too important for you, or we don't need you anymore because we now have these tools that are automating your part of the job. So assuming you've done that first step, you've said, okay, here are some of the business challenges we want. How do you own this thing? How do you build accountability? How do you actually get to that solution layer from like, I don't know, like sort of an operating model perspective?

• 51:01Speaker 00

  Well, that's all the time we have today. No, I mean, you've asked such, you've asked a really hard question, right? You know, it really depends on the organization and there's that, that's, there's an art to addressing that issue as enterprises evolve. I wish I had one formula, but I'll sort of give a couple of examples. So in the case of predictive AI, it's often the data scientist who's familiar with what a predictive AI project could be and therefore what its business value would be. At the same time, the data scientist is often tragically unaware of enough business constraints

• 51:45Speaker 00

  and business considerations to get the thing deployed, but they're necessary to get it off the ground because that data scientist knows, hey, we could predict maybe this well, you know, which transactions are fraudulent or whatever the thing is that you're trying to fix, you know, which insurance claims are fraudulent, whatever. So we could, we have this kind, we have roughly this amount of data, this amount of training data, historical data from which to learn, and maybe we would end up with them all this well and it could potentially be used that well. So in that way, it's bottom up, right? Or it's kind of, it's a business need, but it's sort of being introduced from the tech side.

• 52:27Speaker 00

  I would say that, you know, ideally, it's the business side that decides what the business needs are, but it's just practically speaking, it's not going to always work that way. The other thing is that, you know, this, what's our AI strategy? What's our AI budget? Which, you know, we need to use AI more, which tends to be solutionism, right? And sort of to a hammer, the whole world's a nail and so it's going to become a hammer party for no, you know, for no good reason. On the other hand, there is value to the idea of a general purpose tool, right? Like a computer, right? And it's worth like making sure there are enough computers in the building, right?

• 53:10Speaker 00

  So the thing is, is that on some level, you have to be aware of a bunch of representative potential use cases, right? Value propositions, problems that could be solved or efficiencies that could be gained, whatever it is. So that's my hand waving, man. I mean, like, it's going to be different for every organization and there's no simple one-size-fits-all solution, I don't think.

• 53:41Speaker 01

  Just, just quick detour because you said something interesting comparing it to computers and a general purpose tool. Does the productivity paradox around AI, like, does that startle you? Does that concern you? Like, do you think that we're just measuring the wrong things or do you think we're really not getting a lot of value out of it?

• 54:00Speaker 00

  I think we're not really getting a lot of value of it yet. And I don't, I think that, look, this is just a gut feeling and be very hard to put real numbers on this, but I think we might get sort of 5% of the value that everyone's making noise, that sort of is being implied or that is very concretely represented by certain valuations. Now, if it turns out to be five, that's not, that's going to be disappointing and it's going to cause some economic pains and I don't know how we, I don't know, economically, I don't know how we get there safely if, if sort of things settle into that. On the other, on the other side, it's kind of optimistic.

• 54:40Speaker 00

  I mean, 5% of, of these mammoth values would be amazing. There's no question that this, that large language models are valuable. I just think that that story of autonomy and artificial general intelligence, right, is out of the science fiction playbook and it's so tantalizing and it sells so well, we are stuck in a hyperbolic chamber.

• 55:08Speaker 01

  Well, and for me, as I think about this practically, there's sort of two points of comparison. They're saying, okay, if you're at 5% right now, you know, can I get to 100% of what I'm hearing? Or is that just like a stupid way to frame it? And like, you should, what you should be thinking about is like, what percent can I reasonably get to? What is my best competitor getting to? What, like, what is best of breed actually look like? Like, what is the gap between where most people are right now and where they could be if they just did this a little bit more thoughtfully?

• 55:46Speaker 00

  Yeah. And the other thing is that it depends on timeframe. So everything I just said, you know, I consider it five to seven years from now. But if we go out more 10, 20 years, there could be entirely new technologies that nobody's conceived of. You know, I always thought machine learning was so exciting, excited because the sky's the limit in a certain way, so long as there's data from which to learn. And supervised machine learning is the kind of machine learning where you have labeled examples. They don't necessarily have to be manually labeled, like this is a picture of a cat, this is a picture of a dog, with, with typical enterprise machine learning, history speaks.

• 56:31Speaker 00

  So you know which customers canceled, you know which transactions turned out to be fraudulent. There's, there's not an additional, there's no bottleneck, there's not an additional effort needed for those labels. And in the case of large language models, the labels are what was the next word written in this document? You know, we, we wrote these 10 and a half pages and then this word was the next one that come right next. So let's try to get the neural network to predict that word, right? None doesn't necessarily will because there's a random element, right? And it's not, it's not perfect. Perfection is not the goal, but if you sort of veer it in that direction, and that's what the training process does. Now let's take all the words up to and including that word, what was the next word after that, right?

• 57:13Speaker 00

  Those are labeled examples, again, extremely large number of them because we didn't have to manually label them or, or that is to say, we didn't have to make any extra effort labeling them. It was organic, we did label, all the humans wrote all these documents, right? And that turns out to be great amount of training data. So at this point, we've tapped that unprecedented resource, that incredible amount. Now, there's other things, there's video, there's other kinds of media and content where it's like, well, here's the frame of the video and then this was the next frame that came after it, or this object moved in this direction in this video, right? So there's, there's still a lot more, but it's not literally limitless.

• 57:57Speaker 00

  And so there is, for things to become, for things to approach the sort of almost deity level promise of supreme autonomy and, you know, a human in a box, right? An artificial human, all that kind of stuff, it's going to need something very different than this sort of, machine learning is the most important general purpose technology of the century. That's what Harvard Business Review said and that always resounded with me, but it depends on the learning examples, the data. And, and it's incredible what it can do with this amazing amount of data that we have, but there's limits and what else could we do?

• 58:41Speaker 00

  What other technology, what other approach? You know, there's replicating each neuron in the brain, but that is like thousands of years in the future or whatever. I mean, that's just not on the table right now. So if, for things to reach the overall level of promising, of supreme autonomy, something unprecedented and unexpected would have to happen.

• 59:08Speaker 01

  I think we both agree with that, that there's like a 20x gap between what people are talking about and what people are doing right now, like order of magnitude. I'm not, you know, trying to be mathematical about it with like 5% to 100%. But if we talk about the here and now, and I'm thinking about this in relationship to like the AI playbook, right, this, you know, sort of handbook you've written for people doing this. The, the difference between I'm sort of bumbling my way through trying to get value out of this and I'm actually doing this properly, like, how much can you unlock there? Like, like, how much better can you get, I guess? And like, how much value is still on the table right now, practically speaking?

• 59:53Speaker 01

  And, you know, I guess, how do you best unlock that?

• 59:57Speaker 00

  Well, medium to large organizations have millions of dollars per, per, per unperformed or unexecuted potential project, right? In general, I mean, there, there's so many ways that it can improve efficiencies. I mean, the way we're treated and served by, or the thing that makes the world go round is how we're treated and served by organizations, right? Who they contact, mail, prove, test, diagnose, warn, investigate, incarcerate, set up on a date or Medicaid, right? And all of these ways that we're treated are gamble, little gambles, right? And you can play those gambles so much better with probability. We don't have magic crystal ball. The next best thing is probability. Sounds boring.

• 1:00:39Speaker 00

  We try not to use the word probability, but look, it's just a number between zero and 100 of what's the likely outcome for this particular case or this individual. And that turns out to be really valuable if only we could execute on that concept and actually get these things deployed systematically, make things work more effectively. It's a win-win in general, right? And you can argue capitalistically that the organization stands to improve more. But in general, if these are efficiencies that really largely also help the consumer for a lot of these projects, you're going to have less fraud. Healthcare could be improved. You're going to have less spam. There's a lot, you know, this efficiency is, is a good thing.

• 1:01:21Speaker 00

  And, uh, right, the, the thing, the, so the AI playbook, uh, my book introduces this paradigm framework playbook called, that I call Biz ML, B I Z M L. And that's the website for the book. It's, it's about to come out in paperback and you can get a free audiobook. So go to bizml.com.

• 1:01:43Speaker 00

  And the six step practice there basically introduces three pre-production steps that and engenders a deep collaboration between the, both the biz and tech, the stakeholders and the data scientists or the data professionals, right? And it introduces sort of three pre-production ones, which have to do with defining in a very concrete, practical way, what's predicted, how well and what's done about it that we were talking about earlier. Then the other three steps are the same technical steps that have always existed since the 60s, when this stuff was first used, which is like, prepare the data, learn from it. That's the, that's the machine learning part of the machine learning project. And notice that, that penultimate step, step five out of six is you don't even get to actual machine learning of a machine learning project till step five, because you've done

• 1:02:28Speaker 00

  everything else properly ahead of time. You're not just doing awesome, cool technology for its own sake. It's okay to enjoy the first wow, um, but unless you're going to plan to get to the second wow, it's just a hobby. And then last step is the actual deployment. That's step six.

• 1:02:44Speaker 01

  Well, it's, well, it's interesting to me, and you kind of touched on it, that the, the model you're using is like so front loaded with, you need to do all this thinking. You need to get all this stuff right before you get into the guts of it to make it successful. Is that fair?

• 1:03:00Speaker 00

  For sure. It's just about positioning it. It's just very practical. You know, it's, it's, it's like, oh, okay. Well, we have this fraud team and, you know, they're looking for a needle in a haystack. If we could make the haystack smaller, what will the numbers tell us? And if so, well, then should we just give a smaller haystack to a smaller team? Well, that's a business decision, right? So that's, that's part of the recognition of actually making this stuff valuable and making it viable to operationalize is that there's just like these very practical questions that are not going to be answered by technology. So if I learn from the data to put odds on each transaction, flagging the ones that are

• 1:03:44Speaker 00

  most likely to be fraudulent, great. But that process and the evaluation of how well it does, what's the predictive performance? That's interesting, but it doesn't direct, it doesn't ordain exactly how you should use it, or whether you should shrink your fraud team, or maybe even grow your fraud team. You can capture even more fraud. Those numbers and the trade offs are informed by the technology and the performance metrics. And they're just a matter of very particular arithmetic, but they ultimately come down to business choices that are going to be informed by the technology. And, and that's, that's where, that's where you need this tight collaboration so that

• 1:04:27Speaker 00

  you can not only make a predictive model, a machine learning model that's viable and potentially valuable, you can make a very nice, transparent decision, informed decision about whether to use it and exactly how to use it.

• 1:04:42Speaker 01

  You, you've got a quote that I found interesting, and I feel like probably comes into play in, in some way here, which is that, um, and I may be paraphrasing, but this is more of a consulting gig and less of a technical install to actually get value out of this stuff. Can you, can you unpack that a bit?

• 1:05:01Speaker 00

  Yeah. And that quote was pertaining, I had to think about it for generative, but that, that quote was meant for predictive projects because what they're, they're about the fundamental execution of your existing largest scale operations, the things that makes your organization profitable, the things that make it provide value.

• 1:05:22Speaker 00

  So it's not like tech, like, hey, let's just put a new database solution or a new server and you just sort of swap it in and everything still operates the same. That's a tech install. No, these projects are all necessarily a, a business consulting gig. So another way to think of it is they're referred to as a machine learning project, but that's kind of a misnomer. They should be reconceived as a business operations improvement project that happens to necessarily use machine learning.

• 1:05:50Speaker 01

  I like that. That, that, that makes more sense to me. And then, um, just, just as we sort of move into, you know, I guess, kind of the, the final phase of our discussion, coming back to the, the AI playbook, you've, you've released a new version of this recently. And, and I want to talk about why, and I guess to, to frame that up a little bit differently, like what's new? What's different now when it comes to having an AI playbook than a few years ago?

• 1:06:17Speaker 00

  Well, the AI playbook is about predictive AI and fundamentally not that much has changed. In fact, the new version is paperback, which means it's cheaper. There is one other difference, uh, which is that we wrote a, uh, I wrote a new preface. So it's the paperback edition with a new preface by the author, and that preface compares generative AI to predictive AI. So the biggest difference from when the book first came out less than three years ago is nomenclature, right? Within the book, I call it machine learning project, machine learning project. So let's go back in time to when I first became an independent consultant in, uh, 2003. And I walked around the streets of San Francisco and I was like, hey, you want to hire me? I'm a machine learning consultant. And everyone was like, you're a what?

• 1:07:02Speaker 00

  And everyone was calling it, uh, data mining. And I was like, well, that's not a great term for it. So I latched on to predictive analytics, which was the time was just of a new term. It was just starting to be up and coming. And that's the name of my first book. The first book's sort of how it works and how it provides value. And the second is how to capitalize on it. But they're both about predictive AI.

• 1:07:25Speaker 00

  So since then, the term machine learning has very much become in vogue and, and AI has gone through all sorts of resurgences, especially recently. AI is nothing if not a subjective term, but, um, machine learning is a well-defined term, and so is predictive analytics. Predictive analytics is the, is the enterprise use of machine learning for those types of projects, essentially. The idea of learning from data to predict, like, that's pretty specific, concrete type of technology. So now with generative AI, uh, excuse me, with generative AI, I'd say that predictive analytics is now perhaps more commonly referred to as predictive AI, but arose by any other name. And to time that I wrote this book, I called it machine learning project.

• 1:08:09Speaker 00

  So now the new preface says, hey, at the very end, there's a little italics note that says, hey, look, for the rest of this book, it says ML project, but every time you see that, just think predictive AI.

• 1:08:21Speaker 00

  So the terms change, but the idea of learning from data to predict and using those probabilities to improve existing large scale operations, that's never going to change. We're never going to have a magic crystal ball. The essence of the discipline of those projects is immutable. It doesn't matter how complex and sophisticated, the, the tech part of the project will change, right? The way in which we leverage large language models in all sorts of varied ways to help improve the predictive project and improve the execution of the project will change. But the concepts, the fact that there's diminishing returns on how well we can predict, but that there's great value in predicting better than guessing. And how do we make decisions about how to actually operationalize that? Those fundamentals, um, I don't think they're going to change.

• 1:09:08Speaker 01

  I think, I think that's well said, and it gives it, I mean, kind of, I think, well-earned timelessness about it as well, that this has been valuable for a long time. It has a lot of runway ahead of it and, you know, untapped value, as I think you said earlier.

• 1:09:24Speaker 00

  Yeah, for sure.

• 1:09:26Speaker 01

  So, so maybe to, to kind of put a bow on this, um, you know, what, what's kind of your parting advice for technology leaders and business leaders as they, you know, kind of think about what they're going to do next to, to create more value, either broadly in the AI space, or maybe specifically with predictive AI?

• 1:09:48Speaker 00

  Oh, just that value, right? Get excited about value, stories about value, not about buzzwords and how many buzzwords you can jump, jam in there, not trends. And I think with that focus, you'll naturally start to say, hey, look, probably about half the time, if not more, that's the lowest hanging fruit is a predictive AI project.

• 1:10:12Speaker 00

  You know, get things to work better, create more value for, for your customers and for your state, your shareholders, right? Make things work better and, uh,

• 1:10:25Speaker 00

  that's, that's going to be the antidote to fetishizing technology, to solutionism and to science fiction narratives.

• 1:10:37Speaker 01

  I think that is extremely well said. Eric, I wanted to say a big thank you for coming on the program today. Lots to think about, lots I'm still digesting, and I really appreciate your insights.

• 1:10:46Speaker 00

  My pleasure. Thank you, Jeff.

• 1:10:50Speaker 01

  Most viewers don't know this, but Digital Disruption is developed by Infotech Research Group, a leading advisor to technology leaders around the world. If that's not you, you don't need to care. So skip ahead and enjoy our content. But if you are a technology leader, Infotech helps IT teams get projects done faster, better, and at a lower cost. Infotech provides unlimited access to practical tools and expert guidance you need to execute at a fraction of the cost of traditional consulting. No matter the project, from AI strategy to cybersecurity to vendor negotiation, Infotech has you covered. Check it out at the link below and don't forget to like and subscribe.