Behavioral economics and AI-driven decision making

Mike Bugembe teaches us how to build a culture of data-driven decision making within a company, leverage behavioral economics, and identify high value use cases for AI. Sponsors: Hired https://hired.com/practicalai – Salary and benefits upfront? Yes please. Our listeners get a double hiring bonus of $600! Or, refer a friend and get a check for $1,337 when they accept a job. On Hired companies send you offers with salary, benefits, and even equity upfront. You are in full control of the process. Learn more at hired.com/practicalai https://hired.com/practicalai. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Rollbar https://rollbar.com/changelog – We catch our errors before our users do because of Rollbar. Resolve errors in minutes, and deploy your code with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Linode https://linode.com/changelog – Our cloud server of choice. Deploy a fast, efficient, native SSD cloud server for only $5/month. Get 4 months free using the code changelog2018. Start your server - head to linode.com/changelog https://linode.com/changelogFeaturing: Mike Bugembe – X https://x.com/mikebugembeChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: JustGiving https://www.justgiving.com/BJ Fogg’s Behavioral Model http://www.growthengineering.co.uk/bj-foggs-behavior-model/Books “Cracking The Data Code” by Mike Bugembe https://www.amazon.com/dp/1781333335Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Beta for change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at change law because of roll bar. Check them out at rollbar.com and we're hosted on Lino servers. Head to linode.com slash change log. This episode of practical AI is brought to you by hired. One thing people hate doing is searching for a new job. It's so painful to search through open positions on every job board under the sun. The process to find a new job is such a mess. If only there was an easier way. Well, I'm here to tell you there is. Our friends at hired have made it so that companies send you offers with salary, benefits, and even equity upfront. All you have to do is answer a few questions to showcase who you are and what type of job you're looking for.

• 0:45Speaker 00

  They work with more than 6,000 companies from startups to large publicly traded companies in 14 major tech hubs in North America and Europe. You get to see all of your interview requests. You can accept, reject, or make changes to their offer even before you talk with anyone. And it's totally free. This isn't going to cost you anything. It's not like you have to go there and spend money to get this opportunity. And if you get a job through hire, they're even going to give you a bonus. Normally it's $300, but because you're a listener of practical AI, it's $600 instead. Even if you're not looking for a job, you can refer a friend and hire will send you a check for $1,337 when they accept the job. As you can see, hired makes it too easy. Get started at hired.com slash practical AI.

• 1:38Speaker 00

  Welcome to practical AI, a weekly podcast about making artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and slack with us around various topics of the show at change.com slash community. Follow us on Twitter, we're at practical AI FM. And now onto the show.

• 2:03Daniel Whitenack?

  Well, this is Daniel, data scientist creating AI for good. And I'm joined here by Chris Benson, my co-host, who's a digital transformation strategist, which is very exciting. How are you doing, Chris? Doing great today. How's it going, Daniel? It's going well. And speaking of strategy, I think you're gonna really like today's guest. We have Mike Bugembe with us today. And when I met Mike, I was really intrigued by his story. First of all, some of the things that he's worked on in the past, but also just the experience in kind of guiding a company all the way through the process of

• 2:43Daniel Whitenack?

  kind of defining and implementing data analytics and AI strategy within the company. And so today, we're going to talk to him about all of those things. So the culture of AI and the operations of AI and, you know, strategy around that and when and how you can find AI use cases. So I think it's, I think it's gonna be really great. Welcome, Mike.

• 3:03Speaker 02

  Hello.

• 3:04Daniel Whitenack?

  Yeah. So why don't, why don't you tell, tell us a little bit about yourself?

• 3:09Speaker 02

  So hello, I'm Mike Bugembe. I'm an all-around data evangelist and a consultant working with organizations, trying to help them find value in this wonderful asset that everybody's so excited about, working with them with identifying AI solutions, working machine learning problems and so forth. And all of them seem to be really interested in building some really interesting algorithm for them and the whole spectrum of AI. And formerly, before I started working as a consultant, I was the chief analytics officer for JustGiving.com, a UK-based company that also happens to be the world's largest online social giving platform.

• 3:51Daniel Whitenack?

  Oh wow, that's amazing. So I know I wasn't super familiar with JustGiving, but it's really large, right?

• 3:58Speaker 02

  I think we're bigger than a lot of people initially think. We are a for-profit organization that operates in the nonprofit space. And so to date, they've raised more than three billion for great causes out there. I suppose I should explain a little bit about how it works for the audience, because it might not be so familiar. So just getting started in around the year 2000, 2001, essentially what they are is a digitized form of the paper form that we used to take around when we were doing an event for charity. So let's say we were running the marathon for charity. We would take a paper form to our colleagues and our friends and say, would you sponsor me for this particular charity that I'm running for? I may be running for Cancer Research UK, for example.

• 4:43Speaker 02

  I've picked that because someone in my family has been affected by cancer and I'm really passionate about them eradicating the disease. So I take this paper form and I go around to different people and they sponsor me. Some people would say they'll give you a pound a mile. Some people just throw in fifty quid, which is fifty pounds, or even more than that. And then companies tend to match that as well. So if you raise five hundred pounds, companies will also add five hundred pounds to that form. And what JustGiving did is, quite ingeniously actually, is they took that paper form and they digitized it so that they could reach effectively more people. As we were becoming a much more connected world, digitally at least, we were able to take that form and spread it around to lots of other people who would then come on to

• 5:23Speaker 02

  that new page, which we called a fundraising page, and they'd be able to donate directly to your page and then you'd raise the money for charity and then do the event. And that was, if you like, the mainstay of the organization. As we progressed into, you know, late 2014, 2013, around that time, we also started building a sort of individual giving platform where people could raise money for whatever cause they wanted. So it didn't have to be a formal charity. It would be things like, let's say, someone needed to travel to South America to see their grandmother before she passed and needed to raise funds for the plane ticket, for example. And they'd create the same page, reach out to their friends. Their friends would come and donate.

• 6:06Speaker 02

  Their friends are people they knew. And then that individual would be able to take those funds and and carry out the for-good mission that they had initially raised the money for.

• 6:17Daniel Whitenack?

  That's awesome. It's great to hear about how JustGiving has really empowered that sort of giving. I know you can only reach so many people, giving them a physical form, right?

• 6:30Speaker 02

  Yeah.

• 6:30Daniel Whitenack?

  But our digital networks are so much larger now, so that's so great.

• 6:36Speaker 02

  It is also one of those ideas that you think, "Well, why didn't I think of that?" It's so simple in the concept, but it works so well. And as I said, they've raised more than $3 billion for really good causes to date. Yeah, very exciting.

• 6:51Daniel Whitenack?

  That's awesome. It's amazing. I've seen you post some stuff related to animal charities and stuff. Chris, you'll have to look into this.

• 7:02Chris Benson?

  Oh, you just piqued my interest when I'm not talking AI and data science and such. I'm all about animal advocacy, so.

• 7:09Speaker 02

  Oh, great, great.

• 7:10Daniel Whitenack?

  That's awesome. Well, now you have JustGiving.

• 7:14Speaker 02

  Exactly, exactly. And I think one of the great things about that is it really begins to democratize the whole idea of giving. You know, moving on from the traditional ways that haven't changed very much, to be honest, where charities would solicit, ask requests from individuals, or you have what we call chuggers on the street who are walking around with those tins asking for money. Here, it's really connecting people to the causes that they care about, and then also connecting people to other people. So it's some really interesting AI use cases, if you like, that come out of that as well.

• 7:50Chris Benson?

  Yeah, looking forward to talking about those.

• 7:52Speaker 02

  Yeah.

• 7:52Daniel Whitenack?

  Yeah, that's awesome. I'd be interested to hear just a few more details on that front as far as, because this will, I think, give some context for the rest of our conversation in terms of when you came on with JustGiving and how it's grown, if it's international at this point. I know you've been, I think it was acquired, right?

• 8:11Speaker 02

  Yes.

• 8:12Daniel Whitenack?

  So kind of how, when you came on and what the state was then versus kind of now and how it's grown in terms of the markets that it's in and size and all of that.

• 8:24Speaker 02

  Uh, yeah, so I think we, we, I joined in two thousand and ten. And at that point, I would say, uh, the organization was very much in its infancy with regards to how they would work with data. They had a lot of the traditional elements in place in terms of having a data warehouse and collecting some of the information. But culturally, there was a long way that they needed to go. It was a team of one when I joined, which was essentially myself with the founders. Main objective saying they've got this valuable data and it is a valuable data set if you think about it. You've got in excess of twenty million individuals from more than, you know, uh, a hundred or so different countries. That have, uh, very clearly stated that they're interested in a particular cause. And more importantly, they've also said why.

• 9:09Speaker 02

  So for example, I'm, I'm doing this for prostate cancer because of what my grandfather went through or something like that. Uh, or I'm from a particular country where there's been an earthquake. So you can see there's a clear relationship. So millions of people, uh, connecting to thousands, hundreds of thousands of causes, telling us why they've connected to those causes. And then you have an additional million people coming to support those causes and leaving, or those individuals leaving comments. That give us an even greater indication of other people that might be interested in those causes or why they're supporting you. It's just a breadth of information that was just sat there in that database.

• 9:47Chris Benson?

  Yeah, that's crazy. It seems like that is like, as a data person, that really excites me. And then also I definitely see how you could leverage it for, for good purposes and also for somewhat nefarious purposes as well.

• 10:01Speaker 02

  Yeah, sadly, that's true with almost any, any, any organization or any data set. It's, uh, that ethical boundary of, uh, what do you actually use it for?

• 10:10Chris Benson?

  Yeah, so, so that begs the question, especially given the fact that you've been there as long as you have. So, you know, when you're coming into the organization and you're trying to, to drive the decision-making process. Based on data at the organization and build that up. Um, how you go about doing that? What, what is the best way to proceed? Can you kind of share some of your experience there?

• 10:31Speaker 02

  Sure, yeah, absolutely. And I, I will start by saying with difficulty. And, uh, that's, that's not just the.

• 10:38Chris Benson?

  Good disclaimer.

• 10:40Speaker 02

  And that's not just for.

• 10:41Daniel Whitenack?

  You mean, there's, there's not a secret sauce?

• 10:43Speaker 02

  Well, actually, so um, I, I've, I've put together what I believe to be a secret formula that helps us, uh, helps organizations understand how they can generate value from.

• 10:54Daniel Whitenack?

  Oh, yeah. Is this, uh, you're, you're writing a book, right?

• 10:57Speaker 02

  Indeed, yes. Um, and.

• 10:59Daniel Whitenack?

  Ah, tell, before you jump into Chris's question, tell us a little bit about your book. I'd love to hear.

• 11:03Speaker 02

  Sure. I actually think the book help, uh, is, is aligned to Chris's question. So I'll sort of put them all together in, in, and bugs them all in my response just now. But the, the book is called Solving the Data Puzzle, primarily because, uh, it is, it's different for every organization, uh, but it has an end state picture, an end state game. But it has many pieces that need to be put down in a specific order sometimes in order for you to eventually get the picture that you're after. And I think the nuances are very, very important, like the shapes of the jigsaw puzzle pieces and so forth. So that's what the title of the book is called. And essentially what it does is it highlights the, simplifies the whole problem that we have in the data space and simplifies it to

• 11:44Speaker 02

  five keys that you need in order to truly begin to see value from data. And this is built from both my experience at JustGiving and the organizations I worked in prior to that, as well as, um, fortunately at JustGiving, just as an aside, I, um, had the opportunity to consult with a lot of other organizations at the same time. I had a really good relationship with the founders of JustGiving that enabled me to do that. And, um, as part of that, got to test this, these five keys to see what was required to be successful and also working with some of the larger organizations out there, such as the, the likes of Facebook and, and, um, uh, Google and so forth, working with individuals.

• 12:23Daniel Whitenack?

  Oh, wow.

• 12:23Speaker 02

  At those, at those organizations to really begin to just see what's, what's working, uh, for them. And, um, and it all comes down to those five ingredients essentially. So I think the question, if you, Chris, your mind reminded me what that question was that you had asked earlier?

• 12:38Chris Benson?

  Sure, just kind of as you are coming into the organization, you're trying to figure out the best way to to build and drive a culture of good decision making that's based on the data that you're collecting. How do you go about that process?

• 12:51Speaker 02

  Great, so I think where you started is is a real indication of some of the problems that we we suffer commercially, let's say. You you immediately said, how do we get into, I'll sort of paraphrase, how do we get into the habit of good decision making within an organization? The first task that I had was to get the organization to understand that decision making was an area that we needed to focus on. So the things that happen with data is people get very excited about what it can do. And we hear a lot of, you know, like we hear about Facebook's edge rank and Google's page rank, get very excited about those sort of things. And you're you begin looking for, okay, what's what's our big algorithm that we're going to talk about? And and just giving, we ended up coining it give rank, which is, you know, not very creative.

• 13:36Speaker 02

  But, you know, you just get a lot of people just excited about what algorithm can be built, forgetting, just completely forgetting that the main use case for data, and it'll be useful to see if you can challenge this. Main use case for data, whether you're playing with data science or AI, is for decision making, right? And decision making beyond just your organization, it's internal decision making and external decision making. Take, for example, the two algorithms that I talked about. Facebook's edge rank, its its purpose is to decide what content to show in order to maximize the chances of somebody coming back or enjoying their time on on on Facebook. Or Google's page rank, that addresses the decision of what content should I serve

• 14:21Speaker 02

  or return based on the query that was sent? So these are decisions that we could do manually, but they've been automated and had some mathematical algorithm placed on top of them. So with data, data's main use case being decisions, you can now begin to see that organizations then need to take some time to try and understand, well, what what are the key decisions we have within our product, within our operations internally, or even with the users that use the product? Am I making any sense there?

• 14:48Chris Benson?

  Total sense.

• 14:48Daniel Whitenack?

  Yeah, definitely.

• 14:49Chris Benson?

  And not only that, but I, when you talk about whether or not that could be challenged, I don't think it can. I think it really comes down to, to decision making can occur in different types of contexts.

• 15:00Speaker 02

  Yeah.

• 15:00Chris Benson?

  Whether, whether it's, you know, supporting humans or whether it's automating a decision or whatever. But I don't think that, I think that your, your, your basis is really firm there.

• 15:10Daniel Whitenack?

  Yeah. And I think that part of my struggle in the past at a, at a couple of places where I've worked is maybe, you know, I can get people to the point of thinking about the decisions that they make on a daily basis, maybe in sales or, or in operations or whatever it is. But I have a harder time motivating them to understand that those decisions can be augmented or even, you know, leveled up in some way by, by data. They, they kind of have this kind of mindset that they, they have the knowledge in their head to make the decisions, but they have a harder time kind of crossing into that realm of understanding

• 15:54Daniel Whitenack?

  how, how data or algorithms actually interface with those decisions. Could, could you kind of comment on that?

• 16:01Speaker 02

  Sure.

• 16:01Daniel Whitenack?

  If that's something you've seen as well, or, or?

• 16:04Speaker 02

  I think you.

• 16:04Daniel Whitenack?

  Am I totally?

• 16:05Speaker 02

  No, no, no. You've hit the nail on the head there. I think that's a, see, the most interesting thing about data is that the, the individuals that tend to be the experts at it are also the individuals that are poorest at interacting with people, right? And, and the whole problem really here is it's a people problem, not a data problem, right? What you've described there is the natural human tendency to overestimate or, or give more credibility to their own opinions, right? Which is fraught with biases. I mean, there are hundreds of books that are written on that right now. You can look at Daniel Kahneman or Donna Riehl, people who talk about the, the frailties in human decision making, right? Where, where we're just surrounded by so many natural biases. I think there's

• 16:49Speaker 02

  over 120 different cognitive biases that influence our decision making. And I play a lot of tricks on some of the audiences when I'm speaking to them, you know, I, I do, like, there's one that I do, for example, I would ask people, uh, does the population of Rwanda, small African country, is that bigger or smaller than um eighty million, right? And you see naturally there, what I'm playing with is is anchoring. And uh uh most people would give an answer that isn't too far away from eighty million either side, right? But the population may actually be less than ten. But because I've said eighty, they're anchored on on that. So we have a lot of those things that humans making correct decisions with.

• 17:29Speaker 02

  So what we did at JustGiving, interestingly, was we spent a lot of time on behavioral economics, spent a lot of time in just bringing the organization up to speed and understanding the frailties in human decision making.

• 17:39Daniel Whitenack?

  So that that's a pretty new term to me. Could you explain kind of what behavioral economics um kind of entails?

• 17:46Speaker 02

  Yeah, it's a it's a it's a field, I I'm not too sure when when it began, but the the the sort of key names when you type in behavioral economics are people like Dan Ariely and Daniel Kahneman. That looks at human beings as irrational beings. So whereas i- if you look at economics and the the decision making from an economic standpoint, it looked at human beings as very rational beings who were always seeking to maximize payoff with each of the decisions that they made. But I think as research continued to to carry on, they they found that we weren't always behaving that way. And we were actually behaving almost irrationally. And it was because of these sort of biases that came into into play, just like with the the example that I gave you, because I said eighty million, most people were anchored at at that number, right?

• 18:31Speaker 02

  Which is a bit irrational, because a rash a a rational econ, if you like, would have thought through a different process. They wouldn't have been nudged by that number, but we are human beings and this is how this just works, right? And so the field of behavioral economics, I believe, came into play to try and understand, research and explain how we are nudged or um how those biases affect our our decision making.

• 18:53Chris Benson?

  Yeah, it's been really popular lately. I've read a bit on it recently and I know it's very much on the rise and trying to kind of fix some of the fallibility in traditional economic thought. But I think it's pretty fascinating that you had the foresight to bring it into the organization and train people in the field enough so that they could get the benefit of it.

• 19:16Speaker 02

  I think one of the things that I would say it did, and this wasn't deliberate, but it seems to have worked, is I find in a lot of organizations, whilst people get very excited about data, there are a lot of people who get very nervous about it as well. Because you'll be looking at, as a data scientist, you look at data in a way that no one else has looked at before. You're able to find things that people haven't seen before and therefore you're challenging things. An example is, imagine going to a CRM manager who's been, whose job it is to send emails to an audience to try and get them to come back to your platform, let's say for example. And the traditional methods for that are using things like recency, frequency, and monetary value as a simple equation. And

• 19:59Speaker 02

  then you come up to them and you say, "Well, I have an algorithm that can, has a 90% improvement on your approach in bringing people back." And they'd be like, "Well, I've been 20 years in the game, so why would I listen to some mathematical formula that..." You know, it becomes a very personal thing for some people. So you have that human challenge of them being threatened by data as well as being excited by data.

• 20:21Daniel Whitenack?

  Yeah, so that actually leads right into what I was just thinking, which was, what kind of strategy can you put into place to express to people their kind of built-in irrational thought while at the same time not negating their, you know, background and their expertise and, you know, giving them, giving them some light at the end of the tunnel that this is gonna, you know, this is actually gonna improve what you're doing. You're gonna be able to make, you know, more sales or whatever it is, versus just kind of, you know, telling them that they're irrational, which maybe they are, but, you know, probably no one likes to hear, right?

• 21:03Speaker 02

  Yeah.

• 21:03Chris Benson?

  That's a high-wire act right there.

• 21:05Speaker 02

  I think the example, so a lot of the examples we were, we were giving them was the interplay between uh gut, uh gut-driven or a gut-informed decision, experience-informed decision, and a data-informed decision. And what we did is, not only did we show the frailties of human gut-making decisions, but we also showed the frailties of data, because data itself is also limited. It's, it's, it's limited by the data that's captured. You can add ten more data sources to an algorithm and suddenly it'll give you a very different answer. You can just add more data of the same uh um labels that you had previously, and you will also get a different answer. So data as well has its problems. So what

• 21:46Speaker 02

  we were trying to do was look for the balance, essentially. The balance that is not fully leaning on either side. So it's not fully data-driven, and it's not fully gut-based-driven, but it's really a combination of the two. It's how we understand and are um aware of our, let's say, the things that don't work very well in human decision-making, and where data can supplement and improve that, but at the same time not neglecting their gut and experience. Because we as human beings, interestingly enough, are one of the best data machines out there. We take data from a range of sources, not just our five senses. I think there, there talk, there's people who talk about a whole range of senses, I think, in excess of, in excess of ten, I think I've read, where, you know, take for example,

• 22:28Speaker 02

  how you can close your eyes and still point at your nose. That's a sense that uh is, is, it's not touch, it's not taste, it's not any of those, but it's some sort of directional sense that we have automatically, right? So we, we, we're really good data, data machines, so we shouldn't ignore our gut at all. And that was a big message that we were saying, that, listen, there's something in that, because we probably captured more data than the machines have um for, for now. Um, so it was getting that balance right.

• 22:53Daniel Whitenack?

  Yeah, so would you say that, like, in, in, in that light, a good guiding principle is to kind of frame things for people in the sense of, you know, augmenting their valuable capabilities, rather than, you know, replacing everything that, that they've done, right?

• 23:13Speaker 02

  Yeah.

• 23:13Daniel Whitenack?

  But um, but kind of u- utilizing-

• 23:15Chris Benson?

  Using their their amazing capabilities, their their skill, their background, but augmenting them to, you know, make them more more effective or bring new things to light or whatever that might be. Is that an okay way to frame it?

• 23:28Speaker 02

  Yeah, I wish we we we had your language to begin with. It might have made things the journey smoother. Yeah, augmenting, I think is the way you should say it because there you're you're giving credit to to their own decision making rather than dismissing it. And all you're saying is you you you bring something to the table, but what we want to do is enhance it.

• 23:49Chris Benson?

  So how do you, when you're considering that and you're kind of educating them and getting them into the right mode of thinking about about this?

• 23:59Speaker 02

  Yeah.

• 24:00Chris Benson?

  And and bringing those practices into the organization's culture? And accounting for the fact that you have behavioral econ? And then you have more of the traditional data strategy? How do you build a strategy out of all of these desperate parts? Into a coherent message that everybody can understand and and follow with?

• 24:22Speaker 02

  Yeah, that's a that's a fantastic question. And you you'll see there that you, one of the things about data is that a good friend of mine describes it as a team sport. The data team alone cannot do that. The strategy needs buy-in from the entire organization. It's one of the reasons why I think I always push for the lead of the the data team to be part of the exec team because they need to span across the the entire organization. But in order to develop the strategy, there's several things you need. Firstly, you need to make sure you understand the organizational strategy. And by that, I mean, you need to know the the objectives of the organization, the the boundaries of scope and the approach. And usually those sort of organizational strategies are hidden in reams and reams of documentation.

• 25:05Speaker 02

  And you've got to just try and simplify it because data also needs an objective to work work towards. So when building the strategy, the first thing that we did was make sure that we could disseminate the organizational strategy into a sentence that everybody could understand. And also that, you know, could be easy, we could work very easily. Easily with data, so it was a little more discreet in its numbers. It had very clear numeric objectives, a timeline for it, which it was working with. A bounded scope, so it wasn't just any free idea. Uh, and uh, and also a clear advantage that we were using. So for example, just giving. The advantage that we had is that we had millions of causes on the site, you know, so.

• 25:46Speaker 02

  And nobody else had that, so we had to work with that piece of content, you know, rather than just coming up with something arbitrary at the time. So the getting the business strategy right was one of the most critical things we needed to do in order to get the data strategy. It was then from there that we went on to start looking at the possible use cases. And those use cases were really disseminating those decisions, as I said, to try and understand what are the decisions we're making operationally. For example, sending an email about a new e- campaign, let's say there was an earthquake and. It was like, for example, we had the uh, earthquake in Ha- Haiti a while ago. You know, so who exactly are we going to send that email to? Because every time you send an email, as with any decisions, there are trade offs. So. Something happens, you send the email, there are people who will unsubscribe.

• 26:30Speaker 02

  That means there are less people available for us to email for the next cause. So we needed to be personal. So we needed, that was a decision that we had to make, who do we send that email to? And that's where we could apply AI. So that helps with the use cases. And then also looking at the decisions that are being made externally by by our audience. So when someone comes onto the site. Are they deciding how much they want to give? Are they deciding who they want to give to? Are they making a decision on whether they want to come just read or absorb content? So trying to understand those and support those decisions. Uh, and the last thing, actually, so I said, the strategy, uh, the use cases. So the last thing in the data strategy was, um. Understanding where you are as an organization. So.

• 27:10Speaker 02

  Uh, looking at it on almost two spectrums. The first spectrum is what capabilities do we have to develop any of these data solutions? Are we at the stage where we can only say what's happened and why it's happened? Or can we build build algorithms that can predict what's happening and even prescribe? And the second spectrum was we were looking at was, um, how well do we know our decisions? And once we get an indication of where we are, you almost have a game plan, a roadmap of how you're going to get to the desired destination. There was a lot there, so. Let me know if I need to go through.

• 27:39Daniel Whitenack?

  Yeah. No, appreciate that. That gives a ton of great, great context. Okay, Mike, so I definitely have learned a ton about implementing, you know, data-driven strategies and a strategy towards decision-making in a, in a company culture. But after all, this is practical AI. So if we get to that point of kind of scoping out certain decisions that we want to tackle within a, within a company in terms of data-driven decisions, how then do we make the leap to considering AI and machine learning efforts? Was that something that you considered, you know, right off the bat as you were implementing

• 28:22Daniel Whitenack?

  the strategy or did that come somewhere down the line?

• 28:25Speaker 02

  Yeah, you know what? It's something we had in our, in our back pocket the whole time. We knew we did have to take the organization on a journey, but there were some of these that were, I've mentioned this a few times, that, that email example was one that was so obvious to us very early on. We, we needed a, a machine-driven way to very quickly identify who was interested in what cause because we, no matter how, how intelligent we thought we were as humans, we just couldn't figure that out. Whereas there were a lot of clues hidden within the data. But we needed to go through that, that journey of getting the strategy understood and the use cases, which this one was one of those. The other thing that we needed to do quite practically, and this is, this, it depends on where the organization is, but you needed to get an indication of what the payback was, right?

• 29:09Speaker 02

  So as a result of doing this, what's the cost versus the corresponding payback? If it works. And so there's some calculations that we had to do there, which makes the, the case for investment much easier. And, and once you've got to that stage, this is where data becomes, it continues to be a bit of a puzzle. You're not done yet. So you can have your game plan. You can have the areas for which you, you know exactly where you're going to apply some algorithms. You can get a team deployed on starting to think about, you know, you know, are we going to use a collaborative filter or a genetic algorithm? And get really excited about that. But there is nothing worse than spending all that time building, you know, your, your really sophisticated algorithm and it sits on the shelf and it never gets used.

• 29:52Speaker 02

  Ultimately, this has to be deployed either to some level, on some level of production, or given to a team for them to actually use it. So your biggest challenge after that is getting the culture of the organization, getting them into to value and to have new behaviors for where these algorithms actually get used.

• 30:12Daniel Whitenack?

  You're speaking, you're speaking our language here at Practical AI. So, uh, so you're, you're helping us make it, make it practical. You know, in term, in terms of that, uh, like, is that something that you just kind of like, you know, in, in building up this strategy, is that something you instilled in people from the start of a project that like the real value comes, you know, once things are operationalized and deployed, not, you know, when things are conceptualized or, or when a model is trained in a Jupiter notebook or something. Is that something that you instilled in early in, in that mindset? Or is that something you learned? Uh, I mean, I think where I learned that is by lots of painful scenarios where it wasn't the case.

• 30:55Speaker 02

  We, we had a lot of, I don't know how to get away from the painful scenarios, uh, because culture change, you know, behavior change is very difficult to, to sort of instill. Now imagine being a, a data type individual for which we are where, you know, you, our interests aren't in how do we change people's behavior, you know, personally, I mean, they're, they're psychologists that have that as a profession. So, you know, for us to try and jump into that, it's always going to be difficult.

• 31:22Daniel Whitenack?

  I'm definitely not a psychologist. Maybe Chris is. No, my wife can verify I'm not.

• 31:30Speaker 02

  So I think, uh, but, uh, this is why there's, there's lots of other pieces that need to come into play. So for example, if you have the buy-in of the exec team, that helps, uh, because then working with each of the individual teams, you can begin to make sure that they, they get into the, uh, change the reward of, of value base for using some of these. What I found, I found a really simple equation to help sort of summarize behavior change from the cultural side of things. It was by a chap called BJ Fogg. He does a lot of product behavioral design stuff and, uh, He has a formula called, um, which is B equals M A T. Um, and that's B is for behavior is equal to motivation, ability and triggers.

• 32:15Speaker 02

  And it's the, the, the multiplication, the product of each of those three things. Which means if any of those don't exist, you don't get a change in behavior. So by motivation, you've got to uh get this, the organization in, in a sort of a, a behavioral approach where they are motivated to use the algorithm. So there's something, they've got skin in the game, there's something interesting there for them as well. And you can't do that without working collaboratively with the teams. Uh, and that's a key thing that the, the leadership team has a huge amount of involvement in. You cannot do that alone. As a, as a, as a data team. Uh, the motivational structures for doing that. If they're, if they don't exist, and I've seen this and I have experience of this where, you know, you can build something really amazing.

• 32:58Speaker 02

  Um, will, will show an uplift, all the maths adds up, you know, it's definitely something worth doing. Uh, and you've built it, but it just doesn't get used because there is, there is no motivation to do so, you know, and, and if individuals are rewarded by doing something else, they would really struggle to sort of go off piece. So motivation's a big one. Ability, that's massive. One of the things we forget as um data individuals is we don't speak English to the rest of the organization. So we almost limit their ability or create this, this environment where we just look like really brainy eggheads. And it were not so accessible. So a, a good example is how you can, you can build that algorithm, but it's not designed in a way that your, your, your team can use.

• 33:42Speaker 02

  Your CRM team or your customer service team have to be able to have the ability to, to use it and understand it. So we as, as data individuals have a lot of work to make what we do more accessible and the output of what we do to be way more accessible. And that's there in the ability. If it's difficult and there's so many brain cycles required to even process, you know, the output. Uh, ability goes down and then that breaks that equation straight away. And the last one is triggers. The other thing that I've seen and also been privy to is we can build some amazing things and never shout about it. And I remember being a frustrated analyst many, many years ago, sitting there thinking, how come all the marketing teams get all the accolades? And we've got all this amazing stuff here.

• 34:27Speaker 02

  And then you find out, uh, you know, your CEO would come and say, well, you never told me about it. Right? So that's what the triggers is just, we need to work on our communication. And in fact, just giving, we took the whole data team and taught them how to communicate, how to share some of the stuff that they're doing in English, in a language that the rest of the organization can understand. And we began to slowly see some real changes in behavior when we took this equation and intentionally addressed every single one of those aspects.

• 34:56Daniel Whitenack?

  You mean the, the rest of the world doesn't communicate in Python and data frames?

• 35:02Daniel Whitenack?

  You mean there's, there's another way?

• 35:04Speaker 02

  There is.

• 35:07Daniel Whitenack?

  I, I need, I, I apparently need to learn a few more things.

• 35:10Speaker 02

  I mean, the fact that you guys are running a podcast is pretty amazing. I've got to tell you, I've got, I've got a great story about an analyst here. When, when we took the team offsite and one of the things we did first was to do some sort of, one of those psychological evaluation tests to see, you know, whether you're a blue type individual, red type individual, or some of those, you know, um, Myers-Briggs type things. And the whole team came out exactly the same. We were all very blue, introverted type individuals. And you could see immediately where the problem was because we had no one who was like yellow or green to communicate with the rest of the organization. And a classic example was, um, we had one analyst who used to start a sentence when he was talking to the business by, with the following words. Let me explain to you how stupid you are.

• 35:55Speaker 02

  And then carry, he'd carry on his sentence.

• 35:57Daniel Whitenack?

  Oh, that's not effective?

• 36:02Daniel Whitenack?

  No, Daniel, that's not effective.

• 36:06Chris Benson?

  I am learning so much.

• 36:11Chris Benson?

  Okay, so I have been learning so much as well, especially as we've dived into behavioral economics, because I don't think we've ever touched on that in any of these episodes that we've had to date. And I'm pretty fascinated by how you've taken strategy and added that in and kind of all the various facets that you've been talking about. What I'm wondering is, could you tell us what you think makes a good AI use case to dive into? And if you have a specific example that you're able to share and kind of talk about how you made it real at a nuts and bolts level, I would love to hear that kind of like pedal to the metal kind of thought process.

• 36:53Speaker 02

  Yeah, I think I'm now reaching in my head. I actually had an approach that helped you. Okay, so I think what we were thinking about was there's lots of decisions that take place within an organization, right? You have some operational, you have some strategic. A good AI use case is a decision that is typically quite is repeatable, right? So it happens more than once. And whenever it happens, it's relatively the same questions asked. So I keep referring to that email example, there's the one that's on the top of my head. Well, let's take Amazon, for example. Whenever somebody's put something in their shopping basket, there is always the decision

• 37:32Speaker 02

  of what else could we serve them in order to increase the basket size, right? So you get that recommendation at the bottom. That decision is a repeatable decision. It's one we're making every and it's a repeatable decision in a specific point in time. And that point in time is very clear. So every time someone puts something in their basket, you know, you have the opportunity to upsell them so that you can increase their basket size. That decision happens all the time. And so the questions you'd ask is, can the organization identify when that decision will be made? Yes. Can they decide or have an indication of what information is considered every time that decision takes place? And it's the same information.

• 38:14Speaker 02

  And are the possible actions that they could take consistent? So it's relatively the same ones. And then next you say, can the outcome be measured? One of the most valuable things about working in AI and machines is how measurable those the outcomes of the algorithm are. Because ultimately, it's an investment. Data scientists are not cheap. Um. Cloud computing costs, although most say they are, they should be cheaper. They, I, I always feel to some extent, they can be more expensive than on-prem. But then also your costs can go through the roof with the amount of compute and the amount of data that's available. So looking at some repeatable decisions that are relatively complex, but they are measurable. That's the sort of lens that uh we would look for a good AI use case.

• 38:58Daniel Whitenack?

  Yeah. And so when you, you already mentioned kind of uh, you know, data scientists are expensive, you know, one big piece of this that's really tough is actually building up your team for, you know, AI and the operations around that and the uh, you know, the building of the models, but the operationally, operationalizing of them and the monitoring of them. In terms of your experience at JustGiving or maybe other places, have you taken a strategy of kind of building up software engineers that are existing within a company into kind of, you know, AI engineers or machine learning engineers? Or kind of just brought in fresh AI and data science people? Or has it been a mix of both? And do you see,

• 39:44Daniel Whitenack?

  do you see advantages or disadvantages to one or the other?

• 39:47Speaker 02

  Yeah. All right. So um, I think it's, it's a mix of both. I'll answer this by going through the roles that I think were, were really critical for our team and we're building the team. And they were essentially four key roles that we were looking for. So the first was your traditional business analyst. So this is the individual that is perhaps more communicative. They've come from a consulting background. They're almost your, the front-facing ones. They're the ones you take out of the basement, right? And allow them to speak to users. These are quite important because, excuse me, um, data always needs a face and this is a representation, representation of that. But we want them to be able to speak some of the languages of the rest of the tea, the rest of the business. Um, go native if they need to, but really understand the decisions and the key needs.

• 40:32Speaker 02

  Then we've got the engineers. So one of the things we stumbled across, uh, which was very interesting is when wrangling with large amounts of data um and making it available for data scientists. What we found was, you know, that rule, that Pareto rule, that eighty percent of your time is spent on getting the data ready and only twenty percent on building the algorithm? We thought we would try and turn that on its head and reduce the time that a data scientist spent on data preparation. So we built a team of data engineers whose sole purpose was to make data go really smoothly from the source all the way to being available for whatever algorithm was going to be built, so that the data scientist's time was not spent on getting the data correct.

• 41:17Speaker 02

  And majority of the time was spent on just making sure it was it was in the usable format for whatever approach they were going to use, whether, let's say, they needed to do um, uh, you know, it's a logistic regression, for example. Uh, the data engineers would make sure that that data was set up so that you literally just had to run the logic command in in whether it was R or whatever it is that you were using. And spend more time on the results and perhaps consider a different approach to, you know, whether you wanted to move it through to a decision tree, for example, or a random forest. But to try and take away the expense that was being paid on uh working on the data side of things. So we had data engineers who could do that.

• 41:58Daniel Whitenack?

  And I think that sounds like a good way to uh both make things efficient and to keep data scientists happy, probably.

• 42:06Speaker 02

  Yes, exactly, yeah. Um, and of course, there's the mix and match, because uh the data scientist ultimately is the is the hero role that everybody wants to be. So we had to just make sure they could all, they had their strengths. So the data engineers typically came from strong ETL backgrounds, but they were coders to some extent as well, because a lot of the data that needs to be moved around, sometimes it's easier just to write um some some, you know, uh, some code to rather than build the traditional ETL funnels that we used to have. So spark jobs, for example. So those are data engineers had the capability of doing those. And then finally. Finally, the we found that the data scientists couldn't build production ready algorithms. So, you know, you could run it an hour, get the results.

• 42:49Speaker 02

  You still needed to build the model to work in real time for when a user came, if it was one of those. Or at least to have the calculations available and the results available so that the automatic decision could take place. Um, a a quick example would be when a user comes to the site. Uh, which charities should we show them and in which order is a calculation that needed to take place overnight? But then we found that it was changing every hour as soon as they were different interactions that were taking place. So the data scientists couldn't build a solution for a robust solution, at least to work in in real time. We needed engineers to do that. So we needed engineers who could understand the language that the data scientists were using. But then who were experts at building production ready systems.

• 43:33Speaker 02

  So you can see all of these roles, they they need to have the same fundamental skills, but then they have their areas of expertise. So the engineers, we were training into machine learning so that they could understand what was taking place. But really what they were good at was engineering. The data engineers, really what they were good at was working with large sets of data. But they still needed to have machine learning training and so forth. The data scientists, we reduced their work on on the data prep and made them focus more on on on uh machine learning. That make sense?

• 44:04Daniel Whitenack?

  Yeah, that makes that makes a lot of sense. I know, um, you know, that that can be a hard road to navigate and. But I think you you've expressed it well in the sense that you wanna you wanna build up people's skills. But also build up people's skills into what they're they're interested in and what they they're uh. They do well in, right? And uh, as as already established on another uh episode, I I actually really like data munging and cleaning. So maybe I would actually fit more into the uh into the uh data engineering uh part part. There's a sickness that Daniel has in this. Yeah.

• 44:41Chris Benson?

  Yeah, I uh, it is what it is. I won't deny it.

• 44:47Speaker 02

  That's a good place you've come to though, where you that, you know, you've, you've, you've looked inward and you, you can see what you enjoy most about the whole data science process, if you like, because some people needed some convincing, sadly, because you know, the, the most famous of those three or four roles is the data scientist, which a lot of the team are aspiring to. So for some, uh, we just called them the data scientists, but we knew they were data engineers, right?

• 45:16Chris Benson?

  Gotcha. So I guess having gone through all this process over eight years, I guess. If, looking back, um, at, at challenges that you've had, uh, are there any standout lessons learned? Or, or things that if you could do a do-over that you might do differently? Is, I thought that might be a good one of winding the conversation up.

• 45:39Speaker 02

  Yeah, absolutely. Um, I think I really wish I educated the organization as a whole on what data was and what it can do. This, and I wish I'd started at that point because I found I was explaining that, you know, to different individuals at different points in time, whereas if we had done a, a, a bigger exercise on, you know, almost like a transformational exercise and just saying, look, the whole organization needs to be educated on this. I think we would have had less problems, not, not saying we would have had, we won't have had any, but we would have had less problems. This became even more clear to me when I remember, um, uh, speaking to someone at Facebook and they were telling me that the whole organization goes to a data academy for two weeks.

• 46:22Speaker 02

  And a small part of that is understanding how to work with SQL, but a larger part of that is understanding how to ask the right question. And and this really just hit home to me because if we had done more in educating the organization, you know, we we would remove a lot of redundant work as well, where people were asking things because they were just interested, or they weren't really asking the right question. And sometimes data teams can be quite literal. So if they get a request from the business, they would do exactly what's been requested. But if you dug down, the real question was something else, you know, and that's the fault on both sides. So I loved hearing that. There are other businesses that are creating what they call

• 47:05Speaker 02

  data universities, where they put the whole organization through just to get them up to speed, primarily on what is this thing called data? Let's demystify some of the terms that have come around and confused a lot of people. And why is it, you know, something that we can make more democratic? And you don't have to be technical to understand what data is and how it works, because we use it every day. We use information to make decisions all the time. And just demystifying that whole process for the organization. I wish we had done more of that. I think we would have got a lot further. I think we would we developed some amazing stuff, but I think the road would have been much smoother.

• 47:44Daniel Whitenack?

  Awesome. Well, I know that I have learned a ton today. I've really enjoyed our conversation. Thank you so much for joining us, Mike. And we'll put in the show notes a link to Mike's information online, but also he's got this book coming out. You've heard a lot of great principles today, but I think that's going to be the, you know, a really great resource for people that are trying to create this sort of culture in a company. And so we'll put the link in our show notes as soon as that's available. I know it's still in the works, but we'll put it there once it is available. And just thank you. Thank you so much, Mike. I really appreciate you walking us through this process.

• 48:22Speaker 02

  No, thank you very much.

• 48:23Daniel Whitenack?

  Thank you very much.

• 48:24Speaker 02

  Thank you as well very much. Thank you so much for the invite. It's great to be able to talk about this stuff. And me personally, I am really motivated by organizations getting this to work. There's a lot of hype around it, but equally, you know, there are a lot of organizations that are just struggling to get the Value that they can the game changing value that data promises so we can all sort of evangelize this a bit more than I think we can we can really see some some some big things happen you know not just for commercial gain but even for us as as people in an intensely information world that we can you know data help us get smarter live healthier lives and so forth so I'm I'm very excited about what can happen.

• 49:06Daniel Whitenack?

  Awesome. Yeah, that's super, super inspiring. Making uh making AI practical since 2018 here on uh here on Practical AI. Yeah, uh thanks Mike and really looking forward to the book. Hope to hope to talk again soon.

• 49:22Speaker 02

  Great. Thank you. Thank you very much.

• 49:23Daniel Whitenack?

  Thanks. Bye bye.

• 49:24Speaker 02

  Bye.

• 49:27Speaker 00

  All right. Thank you for tuning into this episode of Practical AI. If you enjoyed the show, do us a favor. Go on iTunes and give us a rating. Go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend, whatever you gotta do. Share the show with a friend if you enjoyed it. And bandwidth for change log is provided by Fastly. Learn more at fastly.com. And we catch our airs before our users do here at Change Law because of Rollbar. Check them out at rollbar.com slash change log. And we're hosted on Lino cloud servers. Head to linode.com slash change log. Check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. Editing is done by Tim Smith. The music is by Breakmaster Cylinder. And you can find more shows just like this at changelog.com. When you go there, pop in your email address.

• 50:11Speaker 00

  Get a weekly email keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.