Finding success with AI in the enterprise

Susan Etlinger, an Industry Analyst at Altimeter, a Prophet company, joins us to discuss The AI Maturity Playbook: Five Pillars of Enterprise Success. This playbook covers trends affecting AI, and offers a maturity model that practitioners can use within their own organizations - addressing everything from strategy and product development, to culture and ethics. Sponsors: Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Rollbar https://rollbar.com/changelog – We catch our errors before our users do because of Rollbar. Resolve errors in minutes, and deploy your code with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Linode https://linode.com/changelog – Our cloud server of choice. Deploy a fast, efficient, native SSD cloud server for only $5/month. Get 4 months free using the code changelog2018. Start your server - head to linode.com/changelog https://linode.com/changelogAlgolia https://www.algolia.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our search partner. Algolia’s full suite search APIs enable teams to develop unique search and discovery experiences across all platforms and devices. We’re using Algolia to power our site search here at Changelog.com. Get started for free and learn more at algolia.com https://www.algolia.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Featuring: Susan Etlinger – X https://x.com/setlingerChris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Machine Learning for fair decisions https://www.microsoft.com/en-us/research/blog/machine-learning-for-fair-decisionsBias examples - programmer, black maleIncreasing Trust in AI Services through Supplier’s Declarations of Conformity https://arxiv.org/abs/1808.07261Introducing AI Fairness 360 https://www.ibm.com/blogs/research/2018/09/ai-fairness-360The AI Maturity Playbook: Five Pillars of Enterprise Success https://marketing.prophet.com/acton/fs/blocks/showLandingPage/a/33865/p/p-011d/t/page/fm/0AI Now - Algorithmic Impact Assessments: A Practical Framework for Public Agency Accountability https://ainowinstitute.org/aiareport2018.pdfAmazon’s Facial Recognition Wrongly Identifies 28 Lawmakers, A. C. L. U. Says https://www.nytimes.com/2018/07/26/technology/amazon-aclu-facial-recognition-congress.htmlGeneral Data Protection Regulation https://eugdpr.orgUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Bait with for change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at change law because of roll bar. Check them out at rollbar.com and we're hosted on Lino servers. Head to linode.com slash change log.

• 0:20Speaker 00

  Welcome to Practical AI, a weekly podcast about making artificial intelligence, practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show at change.com slash community. Follow us on Twitter. We're at Practical AI FM. And now into the show.

• 0:46Chris Benson?

  Welcome to another episode of Practical AI. This is the podcast where we try to make AI practical, productive and accessible to everyone. I am Chris Benson, one of your co-hosts. I am the chief AI strategist at Lockheed Martin RMS API innovations. And with me is my co-host, Daniel Whitenack, a data scientist with SIL International. How's it going today, Daniel?

• 1:09Daniel Whitenack?

  It's going great. How about with you, Chris?

• 1:11Chris Benson?

  Doing good. What you been up to lately?

• 1:13Daniel Whitenack?

  Well, I finished out my course that I was teaching at Purdue University. So I'm enjoying one grading and then throwing some eggnog in there when I can, when I compare the two. That's, that's working out well.

• 1:29Chris Benson?

  Sounds great. As I mentioned, as we opened up, I actually started this new job at Lockheed Martin. Very excited about it. Been been ramping up on that. And I've never worked for a defense contractor before. So I'm learning all sorts of new things, you know, about how to apply AI. And it has been absolutely fascinating the last couple of weeks doing that.

• 1:48Daniel Whitenack?

  Yeah, it's exciting. Don't share too much, or you'll, you'll have to, have to kill us, I'm sure.

• 1:53Chris Benson?

  Yeah, I'll have to kill myself. So there we go.

• 1:58Chris Benson?

  So I wanted to introduce our guest today. Our guest has become a good friend of mine in recent months. Susan Etlinger is an industry analyst with Altimeter, which is a profit company. And Susan and I met at the Adobe AI Think Tank earlier this year in New York City, where she moderated a 90-minute broadcast on Facebook. And I was privileged enough to be one of the people on the panel. How's it going today, Susan?

• 2:25Speaker 03

  I'm great, thank you. It made it sound like we spent the entire 90 minutes talking about Facebook, but we actually talked about AI.

• 2:31Chris Benson?

  Very true. I'm glad you said that. Very, very true. Yeah, we had a great panel and talked about AI with a lot of really smart people that were able to contribute to that conversation. And so it was a great, great time to meet. And I've enjoyed talking to you ever since. And it became obvious really, really early on that I had to try to twist your arm to get to see if you would come on to our podcast, because there is so much about the world of AI that you know. And fortunately for us, you have just, I know that you have been working on a report that is fascinating called "The Maturity Model for AI in Enterprise", where you're talking about AI in enterprise in the industry. And I was wondering if we could start off with you just telling us a bit about that?

• 3:15Speaker 03

  Yeah, absolutely. Actually, it's just gone live as we're recording this. So by the time this podcast airs, everybody's going to be able to see it. So what I've been trying to do over the course of the past, you know, depends on how you count it, several months to several years, is understand a little bit about the way that artificial intelligence is evolving, not just as a technology or as a kind of societal or social impact, but also just in terms of the impact on business. You know, because the impact on business is so different in so many ways, the kind of enterprise impact versus the consumer impact, that I wanted to try to get a handle on it. So this report is about two major things. One is kind of what are the four trends that are really affecting the way that enterprises implement AI? And those four trends have to do with how we interact,

• 4:00Speaker 03

  you know, moving from rules, screens to senses, so moving from URLs to kind of speech and images and that sort of thing. The next is around how we decide. And that is, you know, moving from the old way of programming, if-then statements, so, you know, from business rules to probabilities, you know, because AI is, of course, inherently probabilistic. The third is around how we innovate. And in the past, or actually in the current, in the future, we're going to go to more of a kind of a data engineering world, where we're actually incorporating data into the engineering process in a much more fluid way than we can do today. And that's something, Chris, that your insights really helped me shape. You know, today, we're kind of, in many, in many places, anyway,

• 4:43Speaker 03

  we're in a sort of reporting on the past kind of world. And we need to be able to use data in a much more forward, forward-thinking way. And then the last is around how we lead, you know, because we live in a world that's very hierarchical, that's very expertise-driven. And of course, data and the ability to get clean data is going to help us make decisions based on data.

• 5:06Chris Benson?

  I've had the ability to go ahead and read this. I know you had quoted me as one of, a number of people in the article, and let me see a preview. And the big thing that I really was thinking through this process was how much I wish I had had this over the last couple of years as I worked for previous employers and trying to put together the business case and the operational aspects of AI teams. So we're starting to see organizations like Google and Amazon and stuff offering up some of their internals. But this report, the AI maturity playbook that you've put out, is a huge, huge tool to get people started in this. And I wish I'd had it all along. And Daniel, were you going to say something?

• 5:47Daniel Whitenack?

  I was just going to say, it's interesting that the thing that stood out to me when you were talking through that is, is kind of the emphasis on engineering that you were talking about and integration within a company's infrastructure. And I don't know if you've seen this, maybe you can comment on this, but it seems like we've seen a trend, at least when I'm looking at like job postings and people's titles and such. There was kind of a time when we were talking about, oh, everybody needs to be a data scientist and we're all going to use data for stuff. And then like, it kind of moved into everybody needs to be doing like AI and be an AI person or a machine learning person, scientist. And then like, now it's kind of drifted into, I see a lot of job titles looking for like, machine learning engineers or AI engineers

• 6:30Daniel Whitenack?

  or data science engineers, whatever that is. But I think it's like, people are gradually coming to the realization that they actually have to do some type of integration of this stuff in their, in their infrastructure. Yeah, I don't know. Are people feeling that pain? What's pushing that side of things forward?

• 6:49Speaker 03

  Yeah, well, it's funny because this topic, this issue of sort of data analytics to data science to data engineering really popped up in my interview with Chris, you know, not to do too much log rolling here, but I mean, it's really where I started to think about it. And so in the subsequent interviews, when I spoke to other people, I asked them, you know, and even just people that I know in the industry who weren't necessarily formally interviewed for the report. And I'd say, so what, you know, how is this working for you? Like, if it's a startup, what are you seeing with your clients? Or if it's a big enterprise, you know, kind of enterprise platform, what are you seeing? And then in enterprise companies, I was asking them, like, kind of, where are you on this spectrum? And they're like, oh, yeah, yeah, because we've brought in all these data scientists who have a very

• 7:29Speaker 03

  particular way of of working. And the challenge is getting to scale, you know, and getting to be able to build, you know, not just models, but products that we can scale across the organization. And that's a whole, you know, not only a technical challenge, but a cultural one as well. And also a recruiting challenge in terms of trying to figure out what are the qualities we should be hiring for in order to be able to build scalable infrastructure. So that's been, you know, that's been kind of a, or scalable products. And that's been kind of a big theme, really, that I wasn't, I didn't really know to expect.

• 8:02Chris Benson?

  When we had that conversation, Susan, and we're discussing that, it was, I found it in my own experience, as I went into a previous employer and was creating a full AI operation within that organization, that a big surprise for me had been that I was hiring on some new people and I was pulling people from other parts of the organization. And I had a mixture of skills there. And some of our team members were just straight data scientists, in a lot of cases, fresh out of school. And that had been their exclusive focus. And being this new field of, you know, neural network, model creation and such, I think myself and others on the team really expected that to be the strongest skillset.

• 8:45Chris Benson?

  And what we were surprised to find was some of the other members of the team had already been in industry and had created products and services for other companies, or previously for the same company. They had been programmers in various other roles and they had moved Moved in and maybe gone back to school in some cases for data science and to learn this. And I was surprised that those people were able to apply the after model creation, that they were able to apply that better after the fact. And so in some ways, potentially, the people who had focused exclusively on this had had a leg up. But as soon as some of the others caught up with them, the fact that they knew how to deploy and how to meet a business need from in terms of products and services was a huge advantage for that crowd.

• 9:29Chris Benson?

  And that was something that surprised us all.

• 9:31Speaker 03

  Yeah, and you know, I think what's interesting is that this seems like just part of the evolution. You know, if we think back on other technologies and how they became, you know, kind of enterprise ready. You know, you see similar trajectories where you're hiring for a skill and that skill may or may not come with another set of with another particular set of skills. Right? That's a challenge with every technology. But I think particularly with AI, because there is so much hiring that comes, you know, directly out of the academic setting. And that's such a different, it's such a different set of expectations.

• 10:02Daniel Whitenack?

  So I'm curious on your, your opinion on, on the following. In light, in light of that and in light of the other things that you mentioned that are changing around how we will be interacting with systems, for example, and how systems will be more dynamic and reactive. Do you think, you know, for the software engineers out there that are listening to this podcast that are maybe interested in, in AI? I know that there's like some concern amongst software engineers that they're kind of being like their job will need to drastically change and that sort of thing as AI is more integrated into, into the products that we're building. Do you, do you see that like software engineering as a whole is going to see a very dramatic shift?

• 10:45Daniel Whitenack?

  Or will it more be like AI is just going to be something they interact with? But you know, it'll be another layer in the stack or something like that.

• 10:53Speaker 03

  Yeah, well, you know, it's funny, Daniel. I mean, I can answer as a non software engineer, just in terms of what I've observed. And what I've observed is I don't think I've ever seen a software engineer who hasn't had to change. You know, who hasn't had to evolve their skills, who hasn't had to figure out something that they weren't expecting. You know, if you think back to the beginnings of the internet, like. That was a massive, massive change in the mid 90s and the early 2000s. You know, and the development of even like social technologies and so and mobile technologies and all of that, you know, every single time there's a massive shift, there's a massive set of changes that reverberate through the industry. And I just don't ever see that changing, you know.

• 11:36Speaker 03

  And then in terms of kind of the long view, I do think that learn, you know, intelligent systems, the ability to learn from data, autonomous systems, that's going to be table stakes. I don't know how many years, you know, I can't, I don't have a crystal ball, but it's going to be what we're, what we're thinking of as sort of exotic now is going to be table stakes. And that's really a lot of the thrust of the report, too.

• 11:56Chris Benson?

  So I know that having had the advantage of seeing it ahead of time, you, you started off the report kind of talking about some of the macro trends that would affect AI and stuff. And, and you were really thoughtful in how you were approaching how kind of the real world would affect this. You talked to, I remember about kind of the interactions that we're having with computing. I remember one of the, the sections was talking about as we move from screens to different senses that we may not have used historically. And then I believe you went on to kind of how we decide, how we innovate, how we lead. And I was just kind of wondering, um, you know, what some of those insights were that we could share with our listeners.

• 12:31Speaker 03

  So, you know, the screen thing, you know, how we interact is really interesting because we're just so used to, you know, if you're older than like, you know, 30, you know, you're used to interacting with a laptop computer or even a desktop computer and a phone, you know, if you're younger than 30, more of your life has been spent, you know, talking to your phone and talking to that weird little cylinder on your dining room table or your thermostat or whatever it is, you know, that you're talking to. We're certainly becoming much more, uh, much more. Accepting of things like facial recognition and image recognition, although, you know, obviously that comes with issues. And there are even people who are working on sensory-based interactions based on smell and taste, you know?

• 13:16Speaker 03

  So like, none of our senses is actually, you know, none of our senses is going to be left behind. And of course, touch, you know, using haptics and pinch and zoom are all very normal to us now, you know? And you go back 10 or 15 years and like, that was just, that was Minority Report. That was something that lived in science fiction. The biggest shift to me, though, of all of these shifts is around how we make decisions because we are so used to living in a world that is based on if-then statements. If my balance drops below $500, send me an alert. If I make a transaction more than $300, send me an alert. If I do a transaction, you know, if I try to buy something in a, in an airport in Berlin,

• 14:01Speaker 03

  decline my credit card. And now what we're seeing is that the world is a lot more probabilistic. And sometimes that's fantastic, right? And it's really easy to understand and it's intuitive. And sometimes it actually creates a lot of stress for organizations because, you know, you could say something with an 85 or 87% confidence level is fine for one industry and completely off the table for something else.

• 14:26Daniel Whitenack?

  I mean, I imagine that that creates, I don't know, in my mind, I'm thinking for a lot of people, maybe including myself, in, in certain scenarios that creates a lot of trust issues, right? It might be harder for me to understand naturally the, the probabilistic way of dealing with all of these complicated scenarios, but I kind of have to put my trust in the modeling at that point, right? And not just kind of in an easily understandable if-then statement.

• 14:53Speaker 03

  Yeah, absolutely. The thing is too, that it's not just about putting your trust in the model, right? It's the engineering and user interface and other kinds of communicative decisions that are made to let you know whether you should trust the data. So I'll give you an example, and this sort of, almost kind of is a nice segue into the conversation about ethics. In Turkish, and as in many other languages, there are no gendered pronouns. So the word for he and the word for she are the same. It's actually the word O with the letter O. And if you take the sentence, "She is a doctor," on Google Translate, you can do this yourself, and you translate it into Turkish, it will come back with "O bir doktor." Sorry about the pronunciation, Turkish speakers.

• 15:36Speaker 03

  And then if you take "O bir doktor" and you translate that back into English, Google will assume and write, "He is a doctor." Now, this is probabilistic because if you look at the Word2Vec dataset, we know already that the word "doctor," as many other professions, is biased toward male humans because there are more instances in that data of men being doctors than women being doctors. And even if it's 50.5%, you know, it's going to be a man. And so here's the thing. The language Turk, you know, the Turkish language has been around a bit longer than

• 16:16Speaker 03

  Google, and yet, and is not likely to change, you know, for Google's sake, and yet, there's no indication, when you do Google Translate, that what you're looking at when it says "doktor," you've probably got a 97, 98% probability that it's correct, but when you're looking at the "O" that signifies the gender of the human being discussed, that, you know, it's way, way, way lower. And so what I'm saying is that sometimes we actually need to incorporate into engineering and into user interface design some indication for people that what they're looking at may or may not require further analysis.

• 16:54Daniel Whitenack?

  Yeah, and I do think that this leads right into a great discussion on ethics, which I'm eager to get into. But before we kind of jump into those details, I'm wondering if, kind of based on what you were just saying, those are kind of real problems, real biases, real kind of dangers, if you want to put it that way, that exist right now in machine learning and AI. I'm wondering, like, so much of the conversation around, like, the danger of AI and other things, people kind of naturally go to the scenario of like the Terminator scenario or consciousness or something, right? Do you think that that, you know, distracts from these real kind of dangers and biases that we're experiencing now?

• 17:38Daniel Whitenack?

  And should we, should we even be having that conversation? Or should we as practitioners, kind of, how can we help bring a more balanced view into what, what we should really be talking about in terms of ethics? I guess is my question.

• 17:52Speaker 03

  Any of us who work in this field, you know, somebody like me who's an analyst, you know, really with a humanities background versus, you know, you guys who have much, you know, deeper technology, technological chops than I could ever hope to have. Like, you know, you, maybe you hang out with your family at the holidays and they're asking you what you're working on. You say AI and they're like, one of the robots coming to get us. And that's the conversation really that much of the world is having, right? That, the trolley problem, you know, if the car is driving down the road and it has the, you know, it's going to kill one person or five people, or it's going to kill you or a woman with a stroller, like all that kind of stuff is where people, people's minds naturally go to. And I'm not saying that those are trivial issues. Obviously they're not. And when you get into things like autonomous weaponry, I mean, that's a whole other topic.

• 18:37Speaker 03

  But AI isn't a monolith. And so when we think about the, both the benefit, you know, the innovation benefit and the risks of AI, we have to think about it in a, in a particular context. And that context could be something like a financial services context in which you're trying to manage risk, or it could be a diagnostic context in the healthcare industry. And so what I really think is important is for us to understand some of these nearer term issues, some of these very pragmatic, practical issues around what happens when we use algorithms to kind of abstract humanity. Just, you know, not that, not that, you know, not that that's bad per se, it's just

• 19:20Speaker 03

  that it has implications that we then have to deal with on the other end. And so this is part of responsibly learning to use the technology, just as we would responsibly. Learn to use any other technology that is extremely powerful.

• 19:34Chris Benson?

  So Susan, as we are kind of talking about how about what ethics are in AI and how to apply them, which is very personal for me, as I've come into a new job in a new company, new industry, the defense industry, where we're looking at AI use cases. I think this is the first time in my life where I'm almost leading with ethics. And I think, you know, there are many other people that will be in similar situations because AI has such tremendous capabilities. What types of advice do you have for people who are moving into jobs or are now having to face, how does AI affect our products and services at our company? What kinds of things would you advise them to do in terms of their thinking that maybe they haven't had to consider in the past?

• 20:18Speaker 03

  Yeah, I mean, I think there are a few very straightforward things. The first is to understand that algorithms are as good as the data. You know, this is like the classic garbage in, garbage out, right? The algorithms are only as good as the data and the way the data is modeled. And, you know, the data that we have in many cases is simply, it's just absorbed from society, right? You know, in the case of the Google Word2Vec or in the case of the Word2Vec dataset, that includes all that language stuff that I mentioned earlier, you know, it just absorbs the reality that we live in. And sometimes you want to perpetuate and amplify that reality, and sometimes you maybe don't. So for example, if you're creating a segment, if you're a marketer and you want to do audience segmentation

• 21:02Speaker 03

  to doctors, you don't want anything assuming that all doctors are male, right? You're going to alienate all those female doctors out there and potentially even stifle the potential of younger female, you know, students who maybe want to get into the medical profession. So, so we just need to know these things and we need to actually have processes in place to ensure that when we can and can fix and catch bias that we do. We can't change society, you know, by changing technology, obviously, but we can be mindful about it. Um. The second thing is around, uh, explainability. So there's a woman, Rachel Bellamy from IBM. I heard her speak in London not too long ago. And she said, explainability is the new user interface for AI.

• 21:46Speaker 03

  And I thought that was a really interesting point because one of the things we're not used to in probabilistic systems is the idea that you put data in and then there's the sort of black box and then there's the output. And so in many cases, we do need to understand what some of these, you know, what some of these decision criteria were. In some cases, it's fairly straightforward. You know, maybe there are a few kind of keywords that were determining the outcome or suggesting the outcome. And in some cases, for example, maybe with disease diagnosis or with pharmacological types of use cases, it might be very, very complex, you know, or weather, you know, these very complex systems. So this idea of trying to understand the, you know, what happened between the input

• 22:30Speaker 03

  and the output is very important so that people do have a sense of trust. You don't simply say, well, Chris, I'm not giving you a mortgage loan, even though you have, you know, financially pretty much the same profile as Daniel. And then three months later, you give Daniel the mortgage loan, even though he pretty much matched where you match. You have to be able to go back and, and understand what happened. You have to understand a little bit about what caused that action to be taken. So explainability is interesting and it's also become kind of a huge issue in the industry. And I think there's a lot of controversy around it. And then the third piece is, you know, there need to be, there needs to be an understanding that, um, that ethics in AI are simply just norms of behavior.

• 23:14Speaker 03

  And we don't really have norms of behavior in the digital world, the way that we do in the physical world. You know not to push in front of somebody getting on a bus. You may do it anyway, but you know not to do that. We don't have those same norms in the digital world. And so having internal controls, making explicit the decision criteria, all those things are really important.

• 23:33Chris Benson?

  I'm glad that you addressed that because that was actually going to be my next question is kind of what do you need in place around it in terms of what you're calling internal controls so that the burden isn't entirely on the individual that is trying to figure their way through this and apply ethics as they do that. From an internal control, do you need systems in kind of AI implementation that you might not have needed in other environments? And if so, what might they need to be thinking about? What might those systems need to be addressing?

• 24:06Speaker 03

  Yeah, we do need systems. In some cases, it's like grandfathering existing processes and controls, grandfathering AI into that. In other cases, it's entirely new, entirely new types of controls. So for example, some industry examples out there, AI Now, which is a really phenomenal organization focused on ethics and AI, they've issued what they're calling an algorithmic impact assessment, very similar to like an environmental impact assessment that when you're going to build something or excavate something that you need to understand the environmental impact. So this is built on that same premise that if you're going to introduce algorithms and algorithmic decision making into, in this case, it's meant for governments and for cities, if you're going to introduce that into kind

• 24:51Speaker 03

  of a civic environment, that you need to think through some of those potential impacts to vulnerable people, to systems and processes and all those things. And so that document lays out kind of a template for assessing the impact of your algorithmic system. I think something like that can and should be customized for industry. That's one example. IBM has built a couple of things that are quite interesting. One is called a supplier declaration of conformity. So imagine as a defense contractor, or as a retail bank, or as a healthcare provider, you're not only using your data, but you're using data and systems from other organizations, other companies. You want to make sure that you've gone through

• 25:34Speaker 03

  the process of understanding and holding your systems up to the highest scrutiny. But you also want to make sure that your suppliers and vendors and partners Done the same thing. So that's another example. They've also built, you know, and this is again, something that's a bit, I wouldn't say controversial, but it's open to scrutiny. This idea of a dashboard that shows kind of a bias quotient, right? So and a confidence quotient. So as a simple example, if you're trying to settle car insurance claims, you should know that the data that you have for 19 year olds is very, very scant. Whereas the data that you have for like 42 year olds is very, very rich. And so if you're settling a car insurance claim on a 19 year old,

• 26:17Speaker 03

  you need to dig down into some other things and really probably use much more human intervention to understand what the situation was, simply because those recommendations are based on just, you know, less rich data. These are just some examples of things that people are doing. Microsoft is rolling bias check into Word and PowerPoint. So if you use a word that you're maybe not aware has some kind of connotation that is hurtful or unpleasant, it will let you know the same way to let you know if you misspelled a word.

• 26:46Daniel Whitenack?

  Yeah, that's really interesting. And piggybacking off of that, for my own selfish reasons, I want to ask the next question because I've taught a few corporate workshops recently and we kind of, of course, talk about, you know, oh, you want to make your, you know, maybe you want to make your training set as representative of reality as you can. And then you try to optimize for accuracy or whatever it is. And then, you know, I bring up bias and these issues that we're talking about. And we, in the midst of those discussions, I think every time I've done this, someone, somewhere in the audience asks the question about like, well, if we include, you know, gender or zip code or income or whatever it is in our model and it makes it more accurate,

• 27:30Daniel Whitenack?

  why wouldn't we want to do that? Isn't that just the accurate representation of reality, even though it produces a biased model? And I know kind of how I've tried to answer that question, you know, but I was curious your thoughts on how you would help that sort of person understand why they should care maybe about bias in their predictions? And why they, they might want to consider that a little bit more seriously and not just talk about accuracy?

• 28:00Speaker 03

  Yeah. Well, Daniel, you've hit on, I think, one of the most crucial issues around algorithmic bias we're going to see in 2019. And that is, there's a little bit of a storm brewing. Uh, between some data science scientists and engineers and sort of people, I'll just be brave here, say, people like me who run around talking about AI ethics. And here's why. It's really complicated. It's not, you know, and there is a tendency, and I've had this conversation with some data scientists, you know, who work at very well-known companies off the record. There's a tendency, I think, for some folks to kind of do a little so- social justice virtue signaling around, you know, these darn data scientists, they don't understand people and they don't understand humanity and they're going to ruin the world by allowing bias to creep in.

• 28:45Speaker 03

  And, you know, then on the other-

• 28:46Daniel Whitenack?

  No biggie.

• 28:47Speaker 03

  Pardon me?

• 28:48Daniel Whitenack?

  No biggie.

• 28:49Speaker 03

  Yeah, no big, right? And then on the other side, we have data scientists saying, well, okay, so who, you know, elected you the arbiter of all that is good and just in the world? And these are both completely valid points of view. So here's where I stand on it. We do have to have this conversation with precisely the group of people that you're, that you're talking about in a productive way. These industry conversations need to happen because as somebody who I'm not allowed to quote, said to me not too long ago, who gets to choose who's the person who puts their finger on the scale? And that is really critically important because what we may ameliorate in terms of bias for one group, we may actually impact for other people

• 29:33Speaker 03

  or have unintended consequences that we're not even able to forecast. And I'll give you one simple example, okay? So if you think about what happened with Amazon's recognition system, where it incorrectly identified John Lewis and six members of the Congressional Black Caucus as criminals, as matching faces in their criminal facial recognition database. You know, okay, that's like argue, it's not even arguably, that's unarguably bad. Like, bad, bad, bad, right? You know, we've got John Lewis, who's one of the greatest, you know, civil rights activists ever known to man, who is now a ben- basically, along with six members of the Congressional Black, I have been matched to a criminal. If this happens to John Lewis, you know, you can only imagine what's happening to other people.

• 30:17Daniel Whitenack?

  Yeah, and similar with like the recidivism models and other things that I've seen.

• 30:21Speaker 03

  Okay, so here's the other side. And this is, why is this? Because image recognition and facial recognition is really much less accurate at recognizing and understanding people of color than it is in recognizing and understanding Caucasians. Okay, so how do you fix that? Do you make facial recognition better so that it better identifies people of color? How are you going to get that data? Do you, do you, you know, start encouraging people? No, no, no, really, it's, it'll be great for you. Just, just give us your face data, you know, let us, let us analyze your face data, you know, and put you in our system. We promise that will just help in terms of accuracy. It won't have any, you know, bad, bad impact on you. Like, this is a really, you know, who's going to say yes to that? Right? And

• 31:06Speaker 03

  so this is, you know, some people will say, you know, we're, we're perfectly happy, you know, that the false positive rate is so high. Like, just let it stay high because we don't want to be included in those systems. And, you know, there are absolutely valid reasons for that. So, you know, this stuff is not easy. And one thing I would say is, I don't stand on a soapbox, you know, trying to say I'm more ethical than anyone else. I am cowed every single day by how complicated this stuff is. I just feel like we have to have these conversations.

• 31:38Daniel Whitenack?

  Yeah, I appreciate your perspective there. I agree that the discussions are complicated because oftentimes, immediately after I have that conversation, and people are like, oh, well, we'll remove the gender column in our data set or whatever. But if there's, you know, 1,200 other features, who's to say that the, the model can't infer gender from, you know, from those other features? So the, it's not just a, like, take all the sensitive data out, sort of thing.

• 32:06Speaker 03

  Yeah, and zip code, my God. I mean, there's no better predictor of your race than your zip code. And there's no better predictor of your health outcome than your zip code, not even your genome. So there is a way in which people could say very disingenuously, oh, well, we just, you know, we didn't, you know, we didn't include race. Race isn't a fact. We can't do that. It's a protected class. And, you know, and then, but we just chose zip code. You know, like, so this is why we all need to be educated about these things, right? The business people need to be educated about proxy data. And data scientists need to kind of game out and scenario plan some of this stuff, or at least be part of that conversation. And we have to get past virtue signaling and actually into some real methodologies

• 32:50Speaker 03

  that people can get behind.

• 32:52Daniel Whitenack?

  Yeah, at least monitoring for bias at the least.

• 32:56Speaker 03

  Yeah, and then that's hard, too, actually, because who wants to be liable for that?

• 33:00Daniel Whitenack?

  Yeah.

• 33:01Chris Benson?

  So if this isn't complicated enough, trying to take all this into consideration, we now have the reality of regulation and stuff coming into it. Obviously, in Europe, you have the General Data Protection Regulation, which we call GDPR for short. And when you throw that in the mix with all the other complications of trying to be ethical in your use of AI, how does regulation impact that? It seems like there's quite a balancing act that a practitioner is trying to manage through this process.

• 33:31Speaker 03

  Yeah, GDPR is really interesting. Interesting is probably a diplomatic word. I will say, I am a huge fan of GDPR as a philosophy. And yesterday, as a matter of fact, was the 70th anniversary of the UN Declaration of Human Rights that came out of World War II. Eleanor Roosevelt was involved in crafting that. And the whole point really was to protect the civil rights of individuals, protect their rights from unreasonable search and seizure, and from discrimination and disenfranchisement and actually physical harm, all these things, coming out of the Second World War. And GDPR is really built on the UN Declaration of Human Rights from a digital standpoint,

• 34:15Speaker 03

  so that we should be in control of our own data. We should know when algorithms make decisions about us, why those decisions were made, and be able to contest them. And so from a philosophical and historical viewpoint, it's critically important. However, most of us experienced GDPR in the weeks and months leading up to May 25 of this year as an onslaught of horrific opt-in emails, and then not being able to get to a couple of websites that we usually frequent, and not a whole lot more than that. So there's theory and practice. There's the fact that GDPR and its enforceability is a bit of a gray area for global company. If you're global, of course, you have to comply, just in case people do wander into the EU.

• 35:01Speaker 03

  But fundamentally with with regulation around technology, it is always so far behind the reality of the technology. You know, we're still literally in the in the wake of the 2016 election. We're still literally grappling with, you know, is Facebook a magazine or a magazine stand? I mean, that's that's the that's the law this is based on. And and so when you think about it in those terms, I mean, yes, there does need to be protection. What protection? I am not an expert on that.

• 35:33Chris Benson?

  So I guess as we start to come to the end, I want to pose it. And I'm trying to not scope this final question too big. I know in this paper that you've just put out, you kind of finish up by kind of taking practitioners through how to build up their playbook. With that in mind, maybe if you could just kind of give us some pointers or some starting tips on how you might start that process, recognizing that that our listeners should definitely go download the playbook that you're offering on how to build their own playbook. But what are some good finishing points where you can leave them with to get started on that process?

• 36:08Speaker 03

  Yeah, I mean, what I've published is really a meta playbook, right? It's a playbook for a playbook, as you just said.

• 36:13Chris Benson?

  Fair enough.

• 36:15Speaker 03

  Part of that is that as a, you know, as an analyst firm, we publish our research for free as a service to the industry. So this is really intended to help people think through the issues that they need to think through in order to do what they need to do. And of course, you know, I'd probably be beaten around the head and shoulders if I didn't say that I'm more than happy to help with that if people need that. But there are five areas that I think are really critically important. The first is looking at your business strategy, you know, moving from kind of optimizing existing processes to actually, you know, business model innovation, customer experience, and using intelligent systems to enable those things. You know, in data science, you know, we're moving from kind of a specialty or an exotic, you know, an exotic specialty within organizations to the ability to scale.

• 37:00Speaker 03

  With product and service development, we're moving from kind of reactive, you know, taking in all the signals that, you know, about what's happened in the past to anticipatory, trying to anticipate what's happening. You know, we're finally getting to what we were promising for the last 20 years around agile enterprise. From an organization and culture perspective, we don't talk about this enough, but we're moving from a hierarchical to much more dynamic organizational culture. And when you have agile development in an organization and agile mindset, it really changes the way people work together. And some people don't like that very much, and some people are highly empowered. And that makes a lot of difference in terms of how successful AI can be.

• 37:42Speaker 03

  So one major piece of that is you have to have the willingness to fail and fail fast. And that doesn't mean move fast and break things because, you know, that's probably a relic of the last 10 years. But it does mean actually the ability to move in tandem very quickly, learn from mistakes and keep moving because that's just the essence of these systems. And then finally, it's around ethics and governments. We're not in the anything goes era anymore. We've seen in the last year tremendous stories about what happens when we don't pay attention to these issues. We do have to start thinking about the ethics and the customer experience of AI in a much more rigorous way.

• 38:25Speaker 03

  And, you know, as we talked about earlier, that's not the easiest thing to do, but at least there's some early thinking in here about how to start to frame those conversations internally.

• 38:35Chris Benson?

  I really appreciate it. I love what you've done with this. For our listeners, we will have a link to the AI maturity playbook, pillars of enterprise, five pillars of enterprise success in the show notes. And Susan, if people read through that and they want to engage you so that you can come in and help their organization, how would they do that? How would you like people to reach out to you?

• 38:57Speaker 03

  Yeah, I'd love to hear from people. You can, so most directly, you can email me at [email protected]. You can connect with me on LinkedIn. I'm S@linger on LinkedIn and, or I'm sorry, I'm S@linger on Twitter and obviously Susan@linger on LinkedIn. I'm easy to find.

• 39:13Chris Benson?

  Fine, great. Thank you very much for coming on the show. This was a great conversation. I so wish I had heard a conversation like this before I was getting started in industry. So I think you're really helping some people that are still trying to get in and get their organizations involved in this and thinking about it the right way. So thank you so much for coming on the show.

• 39:32Speaker 03

  It's my pleasure. Thank you both so much for having me.

• 39:35Chris Benson?

  All right, well, thank you very much. And Daniel, I will see you in the next show. Bye. Bye bye.

• 39:42Speaker 00

  All right. Thank you for tuning into this episode of Practical AI. If you enjoyed the show, do us a favor, go on iTunes, give us a rating, go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend, whatever you gotta do, share the show with a friend if you enjoyed it. And bandwidth for change log is provided by Fastly, learn more at fastly.com. And we catch our airs before users do here at change all because of roll bar, check them out at rollbar.com slash change log. And we're hosted on Lino cloud servers, head to linode.com slash change log, check them out, support this show. This episode is hosted by Daniel Whitenack and Chris Benson. Editing is done by Tim Smith. The music is by Breakmaster Cylinder. And you can find more shows just like this at changelog.com. When you go there, pop in your email address, get our weekly

• 40:27Speaker 00

  email, keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.