Practical AI Ethics

The multidisciplinary field of AI Ethics is brand new, and is currently being pioneered by a relatively small number of leading AI organizations and academic institutions around the world. AI Ethics focuses on ensuring that unexpected outcomes from AI technology implementations occur as rarely as possible. Daniel and Chris discuss strategies for how to arrive at AI ethical principles suitable for your own organization, and what is involved in implementing those strategies in the real world. Tune in for a practical AI primer on AI Ethics! Sponsors: Linode https://linode.com/changelog – Our cloud of choice and the home of Changelog.com. Deploy a fast, efficient, native SSD cloud server for only $5/month. Get 4 months free using the code changelog2019 OR changelog2020. To learn more and get started head to linode.com/changelog https://linode.com/changelog. Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship. Rollbar https://rollbar.com/changelog – We move fast and fix things because of Rollbar. Resolve errors in minutes. Deploy with confidence. Learn more at rollbar.com/changelog https://rollbar.com/changelog. Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Practical AI episode 85: Achieving provably beneficial, human-compatible AI https://changelog.com/practicalai/85Google - Responsible AI Practices https://ai.google/responsibilities/responsible-ai-practicesMicrosoft - Responsible AI https://www.microsoft.com/en-us/ai/responsible-aiU. S. Department of Defense - Ethical Principles for Artificial Intelligence https://www.ai.mil/docs/Ethical_Principles_for_Artificial_Intelligence.pdfOECD Principles on AI https://www.oecd.org/going-digital/ai/principlesCambridge - The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions https://dl.acm.org/doi/pdf/10.1145/3306618.3314289Nature - Principles alone cannot guarantee ethical AI https://www.nature.com/articles/s42256-019-0114-4Books “Artificial Intelligence” by Stuart Russell and Peter Norvig https://www.amazon.com/dp/0134610997“Human Compatible” by Stuart Russell https://www.amazon.com/dp/0525558632Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Chris Benson?

  It's quite a long process, actually. I would say it starts with principles and that's what most people think of now. But that, once you've done your principles, you've basically just gotten to the starting line.

• 0:13Speaker 01

  Being with your change log is provided by Fastly. Learn more at fastly.com. We move fast and fix things here at change law because of roll bar. Check them out at rollbar.com and we're hosted on Lino cloud servers. Head to linode.com slash change log.

• 0:31Speaker 02

  Lino is our cloud server of choice. Grab the Nano plan for just $5 a month, just five bucks. That gets you a gig of RAM, a blazing fast 25 gig SSD and one terabyte of transfer. Let's be honest, you can go a long ways on that five bucks. When you do need to scale up, their prices are predictable. So you can put your calculator down, you won't need it. We've been running change.com on Lino for years and we've always impressed by their award winning support team. Check them out at linode.com slash change log. Once again, that's linode.com slash change log.

• 1:12Speaker 02

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show at change.com slash community and follow us on Twitter. We're at Practical AI FM.

• 1:37Chris Benson?

  Welcome to another fully connected episode where Daniel and I keep you fully connected with everything that's happening in the AI community. We're going to take some time to discuss the latest AI news and we're going to dig into learning resources to help you level up on your machine learning game. My name's Chris Benson. I am a principal AI strategist at Lockheed Martin and with me as always is Daniel Whitenack, data scientist with S I L International. How's it going today, Daniel?

• 2:01Daniel Whitenack?

  It's going pretty good. It's been an interesting week. Went to visit a couple of family members and of course, traveling during COVID time is somewhat interesting. So navigating those waters and trying to stay safe. So it's been an interesting week. What about you?

• 2:20Chris Benson?

  Same here. I've been trying to kind of reassess everything. I'm in Georgia, as longtime listeners will know, and definitely our numbers are in terms of transmission are going through the roof on COVID.

• 2:32Daniel Whitenack?

  Yeah, it's kind of all across the South at this point, I think.

• 2:35Chris Benson?

  It really is. Big questions about like how we're doing schooling going forward and different local governments and such, trying to figure out what they're going to do. And so it's interesting times. This whole year has been just quite a crazy year in a lot of different ways.

• 2:50Daniel Whitenack?

  Yeah. And of course, it's still a lot of things going on in our country related to politics and also, you know, justice for marginalized groups and

• 3:00Chris Benson?

  Absolutely.

• 3:01Daniel Whitenack?

  Various injustices that are trying to be righted. And I mean, there's sort of a mix of encouraging things, really sad things, confusing things and stressful things all at once.

• 3:15Chris Benson?

  Yeah.

• 3:15Daniel Whitenack?

  Which I guess does interface with what we had thought about talking about today. I don't know if you want to get into that a little bit about what your thoughts were in terms of what to dive into related to AI today.

• 3:29Chris Benson?

  I will.

• 3:30Daniel Whitenack?

  Yeah.

• 3:31Chris Benson?

  Yeah, I will. And I know I've alluded to it kind of, you know, in passing on a number of episodes, but one of the things I do in my own job is I lead AI ethics for the company that I work at, which is Lockheed Martin. And this show is not about specifically Lockheed Martin. I may mention it a little bit in terms of things that have been approved for public release by our communications department, but that's going to be very little. I really wanted to talk about the fact that I spend a lot of time focusing on the details of implementing AI ethics, both internally and externally in our own operations. And it is a topic that I get asked about more and more often these days in terms of people

• 4:14Chris Benson?

  For guidance on how to do that in their own organizations. And in my own organization, we are fairly far along that process relative to most other organizations out there, I think, and the the feedback that I've gotten is such. And so wanted to really kind of start with what most people that have delved into this topic are probably most familiar with, which is kind of principles that are being adopted by various organizations, kind of explore what some of those look like, but then talk beyond just principles of AI ethics and talk about how that affects your organization and your operations when you are trying to take the principles that you have chosen and move them through

• 4:56Chris Benson?

  your organization from end to end so that they become part of what you are. So I thought that'd be a fun thing to talk about. You up for it?

• 5:03Daniel Whitenack?

  Yeah, I'm definitely up for it. It's probably not hard to convince people to think about AI ethics in terms of some of the things that we've seen in our world of recent times, especially usage of things like facial recognition and other things and, you know, by governments and other institutions that are trying to discriminate against certain groups or something like that. So that's sort of a very AI ethics comes up very easily in that conversation. I think that there may be though, some people out there that are practitioners and maybe, let's say they're using AI to optimize their infrastructure or they're using AI to plan their marketing campaign or something like that. It might

• 5:43Daniel Whitenack?

  be a little bit less clear why AI ethics and AI ethics statement about your principles around AI ethics and how you approach AI is relevant to those scenarios. I don't know if you have any thoughts on that before we just jump in, like if you're doing AI, why it may be important that you start to consider some of the ethics behind how you approach your AI development regardless of, you know, the specific use case or are there some motivations that everyone should be thinking about?

• 6:17Chris Benson?

  Yeah, I think so. What we are calling AI today, which is in a practical sense, it is probably the, at least the area, you have a lot of dispute, obviously, on what AI is out there. And every organization and really every person that's in this field kind of has their own definition. So that muddies the water a little bit. I think if there is probably one area that most people would agree is AI, you know, as we record this in 2020, it'd probably be the field of deep learning and kind of the other adjacent fields that are inclusive or tied to that. And so I think one of the notable aspects of that is that

• 6:58Chris Benson?

  they are non-deterministic systems. And that means that you're not guaranteed that a given input is always going to yield exactly the same output. You can have variability in there. And as a matter of fact, you know, obviously, when we train a neural network, we introduce randomness on purpose, you know, for purpose of getting to a better model. It doesn't run into some of the, the technical issues with training, which is kind of outside the scope of this conversation. So I think, given that you have these capabilities, but it's in the early days, and we've already talked in previous episodes about how AI will evolve probably fairly dramatic in the years to come. But as we're looking at it right now,

• 7:42Chris Benson?

  today's AI, as, as is mostly deployed, it's not aware of right and wrong. It's not aware of values that you'd want to have govern your actions and your thinking. You don't have a framework that ties ethical considerations, moral considerations, value considerations into the output of an inference. And so you could think of AI ethics as a new field, which probably will stop in the future, stop being kind of this thing unto itself and become part of artificial intelligence development, just like in other fields, they have ethical considerations that are part of the core field itself. But right now, we're kind of

• 8:26Chris Benson?

  building that from scratch. And so we're kind of talking about it as a thing unto itself. But that's really why we're having to do that, is to ensure that we get the outcomes that we are shooting for, that we are anticipating versus the unexpected outcomes that can cause real problems. And, and we've certainly seen quite a few of those over the last few years.

• 8:46Daniel Whitenack?

  Yeah, I think that there's definitely things that are unexpected in terms of how AI behaves. That a problem set might not in and of itself appear like there's ethical issues with a certain problem you're trying to solve with AI, but they can crop up in sort of an unexpected way. And, you know, I'm thinking of like, I gave the example of optimizing infrastructure, right? And while you may not be like discriminating based on race or gender or something when you're optimizing infrastructure, there are certain things about the way that we handle

• 9:25Daniel Whitenack?

  infrastructure as a company that are maybe like energy related and that sort of thing that that might have some implications on the world at large. Whereas, you know, there's other scenarios, especially if you're doing like advertising campaigns online for hiring at your company or something like that. That, of course, is something that is very fraught with ethical and fairness issues in terms of how you're presenting job postings to people and all of those things. So I think there's certain problems, I guess, that the ethical issues are more obvious and certain problems where they're less obvious.

• 10:07Daniel Whitenack?

  But I guess the having principles set down for how you approach AI problems at your company is the purpose of like having that set of principles from your perspective. Is it to help tease out what ethical issues might be at play in a certain problem set? Or is it, what's the real purpose behind having a set of principles that you adhere to as a group of AI developers? What's the main sort of advantage of that and how that might play out?

• 10:40Chris Benson?

  Yeah, I think, you know, principles have been where the majority of the conversation has been so far. You know, there have been a lot of, you know, press releases between with major organizations adopting and publishing their principles. Those really are intended to explicitly establish

• 10:58Chris Benson?

  What your values as an organization are and the kinds of things that your organization is thinking about and cares about to ensure that if you're operating the way you think you should be, then you should be in full compliance with what those principles are. And if you operate in such a way, either on a regular basis or by exception, where you feel those principles, one or more of those principles that you've adopted are violated, then it's a sign that you're moving outside that zone of normalcy for you. And so really, the principles are trying to capture that organizational ethic, but it's not enough. You're going to have to take those principles and push them down through

• 11:42Chris Benson?

  your operations in a meaningful way so that you can execute. Otherwise, you end up having, you know, a set of principles that sound very wonderful, but don't have a meaningful connection with what you're doing day to day. And so it's very important that you start with that, but then you find your way down below that. But nonetheless, we should start with principles today because there's been a lot of thinking in that area, certainly.

• 12:04Daniel Whitenack?

  Sure. So if you're an organization that's using AI, what's a good way to start developing a set of principles or guidelines that kind of guide your AI development team or your AI development department or whatever it is? What's maybe a good way to put a flag in the ground and get started?

• 12:27Chris Benson?

  Well, I think what a lot of organizations have done is to collect a representative group of people from their organization that represent all of the different functions that that organization tries to address in its operations. And so, you know, there could be product teams and service teams. It could be representation from your legal team or Or your HR team, all of these have a unique voice. And so you will bring these people together typically, and you will have a set of conversations on looking at operations, you'll tend to look at use cases and say, what does this mean for us? How do we use technology? How do we think we're using AI? What do we expect to be doing in the future with AI as far as what we understand it to be today?

• 13:12Chris Benson?

  And you'll collect a set of cases to go through and to dive into that. And from that process and bringing all those different perspectives and diversity is so important in this process that you start kind of working together to agree on a set of things to address. Use cases is a good word for it. You know, scenarios, if you will, it can be at a functional level. It might be an internal thing, like how does AI apply to an HR system? Or it might be, how does it apply to the services and the products that we offer our customers and how does it affect our customers? So in doing that, you really say, this is what we do. And you start identifying what would you feel right about doing with your customers and with your employees and such.

• 13:57Chris Benson?

  And you're really focusing on a set of values that will describe what a good process, a good operation looks like there, the one that you want, the outcome that you would like to achieve. And you have to narrow those down to a set of specific words that are the principles with some kind of definition around those that describe what it means to you and your organization. Some version of that is what most organizations that we've seen have addressed.

• 14:26Daniel Whitenack?

  Yeah, I think maybe I naturally struggle with this process a little bit because it seems a little bit to me like, you know, when, when people develop like vision or mission statements for their organization. It seems to me like a similar sort of process and I have trouble with that. Maybe not surprisingly because we named this podcast Practical AI. And I try to be a very practical person. And so a lot of times those seem so far from practicalities that I struggle understanding how they play out in everyday scenarios. I don't know if that makes sense.

• 15:03Chris Benson?

  No, it does. And I think that a lot of people feel that way. And in particular, given the fact that AI practitioners tend to be very down-to-earth people, they're engineers, they're data scientists, they're people who work with the tangible side of technology to achieve something. AI principles can feel very wishy-washy. And so that's one of the challenges. But ultimately, to get those people satisfied, you're going to have to start with those principles. You're going to have to figure out what does that mean for your own policies? The policies that govern how you operate? How are you going to govern that once you've reached that point? How do you know that you're in compliance with what you've decided that you want to do and govern?

• 15:44Chris Benson?

  And then what kind of tooling and workflow integration are you going to implement to ensure that all of that happens and that it can be done by people who are not necessarily experts at ethical AI?

• 15:57Daniel Whitenack?

  So Chris, you were getting into some details about how companies can start getting their AI principles together and also how to connect that in a practical sense to day-to-day workflows. I was wondering if we could maybe take an example. So let's say that fairness in terms of, you know, we're a marketing company or something like that, or a hiring company and fairness in terms of gender and and race is something that is important to us and we have some principle around that. Where might you go from there in terms of? Making sure that an ethical principle actually filters down to the people that are

• 16:39Daniel Whitenack?

  doing the development and building products. It seems like maybe that's a place where a lot of people get hung up.

• 16:46Chris Benson?

  So like if you're talking about like fairness being one, and by way of example, Google has five principles and one of those principles is fairness that they've chosen is is one of the descriptive words for how they're operating. And so you really, in the beginning, it's deceptively simple. You think that's fairness. You know, I know what fairness is. Well, not everyone does and not everyone actually will agree on that. And that's pretty true all the way across. There's a lot of ways to define it. So one of the first ways that you might do that is if your organization, and this tends to happen probably with larger organizations more often, they may have a formal description about what fairness is. If it's a large company in particular, they may have an ethics department

• 17:30Chris Benson?

  at the company, or if not, maybe a smaller company, the HR department will take this kind of thing on. And being able to tie the principles that you're adopting for AI back to what you already believe, and particularly what you've already documented, is really important. Because otherwise, you end up having these AI principles that are kind of this beast unto themselves that are separate from everything else. But it's pretty crucial that you end up with principles that apply directly to your organization in a very organic way and fit in with what you already have.

• 18:03Daniel Whitenack?

  Yeah, I agree with that. I think maybe this is a tension that I often feel when I get into these vision and mission statement conversations. And maybe these conversations are similar around AI ethics and principles is that there's also a tension between who you are now and who you want to be as a company, and kind of casting that vision out. And all of the things you're doing now, maybe they aren't consistent with the company that you envision. You're wanting to put these in place partly to push people to do quote unquote, good development work. But then also, they have to be integrated into the

• 18:43Daniel Whitenack?

  sort of natural sort of philosophy and values that already exist within your organization. Otherwise, it's going to be hard for a company to implement them, I think. So I think that there is that tension. You can't sort of from a top down view, just impose a new set of AI principles that just kind of shock and are totally coming out of left field to everyone in the company, but at the same time, you want to put some things in place to maybe push people to do certain things in a different way. I don't know if you've seen that sort of tension play out, but.

• 19:19Chris Benson?

  Yeah, I have. I mean, that's it's kind of that aspirational tension of where you're at today versus what you have been thinking you want to go to. And when you're first designing your AI principles, and I say design on purpose because you are designing your way into it, you don't just pick them. It really has to feel authentic, not only to your own teams, but also it needs to feel authentic to the people that your organization will interact with. So without that sense of authenticity, like the principles are in alignment with existing policies. You know, we talked about fairness as that example that Google has, and it's notable that that is one that you would typically find fairness was also selected, for instance, by Microsoft as one of theirs. And as you're doing that, you need to ensure

• 20:03Chris Benson?

  that whatever HR policies, for instance, that you have around fairness of what your employees might do internally, that that is reflected because, you know, with AI being an enabling mechanism, an enabling tool that you're typically going to apply to existing things that you're trying to do to enhance them, the output from your AI needs to comply with what your idea of fairness is. And fairness is mostly the same, you know, there's a general idea, but it might mean a little bit of something different to Google from Microsoft, at least in terms of their official policies. So it feels very fluffy, I know, at this level. But if you don't go through this, then when you get to the more pushing it

• 20:44Chris Benson?

  down through the practical levels, you're going to struggle. This really has to represent your organization's values within a diverse context. It needs to be as real to your HR person as it is to your salesperson, as it is to your engineer. So all of those people need to be able to touch these principles and say, I know what that means to me in the context of my organization.

• 21:08Daniel Whitenack?

  So in starting out with getting a set of AI principles in place for your organization, since I'm mainly involved in practical development work, I tend to take the viewpoint that, I think it was Kelsey Hightower in a talk that I saw, I always remember him saying that good developers copy and great developers paste. So looking at some examples from, that people have already put a lot of work into out there, I think is a good thing. I actually, while we were talking, I was looking up some papers on the subject. There's a paper from a group at the Chinese Academy of Sciences, and they took sets of AI principles from academic organizations and governments and industry organizations,

• 21:53Daniel Whitenack?

  including Google and Microsoft and IBM and others, and they did a bit of analysis on them. So they did some word-to-vec analysis on them, it looks like, kind of categorized the statements into a set of topics that occurs very frequently in these subjects, or in these statements of principles. So that's actually very helpful to me, because I think, you know, looking at what people are valuing in their principles might be a good point to kind of get you thinking. So these topics include humanity, and that has to do with keywords around being beneficial for humanity and human-centered and human-friendly. A second topic is collaboration, having to do with things like partnership and cooperation and dialogue.

• 22:38Daniel Whitenack?

  There's a share topic, which has to do with share and equality, inequity, inequality sort of topics. There's a fairness topic, which has to do with bias, discrimination and prejudice. Transparency topic, having to do with things like explainability and audit and tracing. A privacy topic, a security topic, a safety topic, having to do with, you know, controlling risk and human control. Accountability topic. And then a topic around artificial general intelligence or superintelligence and that sort of thing. So I don't know, is that consistent with what you've seen in various statements of AI principles?

• 23:21Chris Benson?

  I think so. I mean, we see so much commonality across different groups, different organizations' principles, that I think it is fair to stand on the shoulders of giants, if you will, and benefit from their own work. I think there has to be a point where if you just adopt and move it in, it doesn't have a lot of meaning for your organization. So there's a point where you're applying it to your own internal policies, your own way of doing things. You have to customize it there to your own organization so that it is meaningful because otherwise it's just a set of words written down. And, you know, once you've implemented these, you should be able to have a common meaning through your organization that understands it. And from even the same word, like we mentioned fairness, from one

• 24:04Chris Benson?

  organization to another, it may mean slightly different things to them based on what their worldview is as an organization and the employees that are working there, trying to affect that, what their operations are. So you can start by borrowing from others and stealing from others, if you will. I think that's perfectly reasonable, but at some point you do have to make them your own.

• 24:24Daniel Whitenack?

  A good starting point would be to take some of those bigger topics. Like I'd say privacy and security kind of cross a lot of these statements, I think. But I guess what you're saying is that in privacy, like, you know, in my organization, SAIL International, we'd have to decide what does, how does privacy impact the things that we do with AI? And how could the things that we're developing that are AI-driven influence, you know, privacy in positive or negative ways for our specific users or customers or that sort of thing? Is that kind of what you're saying?

• 25:02Chris Benson?

  I think so. It's interesting. One of the things that I found is that choice of words matters. If you look across a number of different principles as outlined by different organizations, they'll have different numbers of principles. They'll have different words that they use to describe. And it may be that one organization uses the word privacy and another organization does not. That doesn't mean that privacy is absent from their principles, but it may mean that they've taken the principles that they believe to be privacy and divided them into maybe two other of their principles because they see that as two subsets. It's really dependent upon how you see the world and how you believe you're interacting with it

• 25:46Chris Benson?

  and the language that you use to do that, beyond just AI ethics, the language of your own operations.

• 25:52Daniel Whitenack?

  Yeah, and I guess depending on the industry that you work in, some of these might be more important than others. Like if you're an actual company that works in the HR hiring space, then some of these are going to become extremely important. Whereas if you're a software as a service company or developer tools company or something like that, then other of these might be kind of immediately important. I mean, a lot of these intersect with people, no matter what company they're working in. A lot of these, when I read them, I think could be interpreted, like depending on who's reading them, they could be interpreted in different ways. How do you make sure that, you know, what you're going after with your AI principles

• 26:37Daniel Whitenack?

  actually trickles down to the work that's going on in your AI development teams? Is there tooling around that or is it mostly an education sort of thing? Or what's your thought there?

• 26:48Chris Benson?

  It's going to be both, but what you have to start doing is you have to connect on the assumption that your operations, whether internal or external, are governed by some essentially policies internally. You really have to connect the principles to the policies that you have because those policies are representing the reality, or at least an ideal reality that your organization is trying to achieve. Trying to achieve. And so until you can map principles to the policies that you already have in place and figure out if you have gaps and need to create new policies or change existing policies, that's the first step is saying, I understand how the principles that

• 27:30Chris Benson?

  we've said identify our values map out to the policies that govern our operations and our functions. And so that's really the first step. And then you'll follow that up with, how do we know that we are in compliance then? What's the mechanism and how do we govern that? If something goes awry with the way that we're thinking about this, how does our organization handle that and process that? So, you know, like you would have in anything having not to do with AI ethics. If you have something in your operations that is not compliant with your expectation, how do you handle that? You need that for this as well. And then you need to connect that to the people that are doing the actual work. It needs to fit into the workflow and it needs to fit into the tooling. And if

• 28:15Chris Benson?

  it doesn't, or if that's not an easy thing to do, then you need to figure out what that delta is and figure out, do I need new tools? Do I need to adjust existing tools, which has a lot of variability based on, are they vendor tools that we're paying for and we're limited to the features that the vendor offers us? Is it stuff that we've written for ourselves? How do we know that? And how do we get these to filter in at the workflow level in terms of being productive? So it's quite a long process, actually. I would say it starts with principles and that's what most people think of now. But that, once you've done your principles, you've basically just gotten to the starting line.

• 29:04Speaker 02

  I'm Jared Santo, GoTime's producer and a loyal listener of the show. This is the podcast for diverse discussions from around the Go community. GoTime's panel hosts special guests like Kelsey Hightower.

• 29:17Speaker 04

  And sometimes you can leverage your cloud provider and make margins on top. That's just good business. But when we're at the helm making the decision, we're like, yo, forget good business. I'm about to deploy Kafka to process 25 messages a year.

• 29:33Speaker 05

  It's nerd pride, right?

• 29:35Speaker 02

  Picks the brains of the Go team at Google.

• 29:38Speaker 06

  You don't get a good design by just grabbing features from other languages and gluing them together. Instead, we tried to build a coherent model for the language where all the pieces worked in concert.

• 29:48Speaker 02

  Shares their expertise from years in the industry.

• 29:52Speaker 07

  Don't expect to get it right from the start. You'll almost definitely get it wrong. You almost definitely have to go back and change some things. So yeah, I think it goes back to what Peter said at the start, which is just make your code, write your code in a way that is easy to change. And then just don't be afraid to change it.

• 30:06Speaker 02

  And has an absolute riot along the way.

• 30:09Speaker 08

  Yeah, you know that little small voice in your head that tells you not to say things? What, what is that? How do you get one?

• 30:18Speaker 08

  You want one of those? Is it like an in-app purchase?

• 30:21Speaker 02

  It is Go time. Please select a recent episode, give it a listen and subscribe today. We'd love to have you with us.

• 30:43Daniel Whitenack?

  So Chris, as a practitioner, when I hear you say the word governance and that sort of thing, immediately I, um, I have these feelings inside of me that are just sort of naturally negative.

• 31:00Chris Benson?

  I'm not surprised. A lot of us do.

• 31:02Daniel Whitenack?

  Yeah, I think it's sort of natural for a developer to have that feeling because it's like, oh, governance. Or a natural first thought might be that governance in some way, whether that be over the certain data and access to data thing, or whether it be in ensuring compliance to certain things, that naturally means slow, I think. To most developers, it's like, oh, this is going to complicate my workflow. It's not going to allow me to get anything done and it's just going to be a slow process. So I do think that there is. You know, some validity to that in that, you know, in order to check some of the things that should be checked in terms of an AI application.

• 31:47Daniel Whitenack?

  Um, you're gonna have to spend some time on those things, though. But at the same time, I, I forget which guest it was. I'll actually have to look through our transcripts to figure out who it was, but I. I remember, and maybe it was just a conversation I had at a conference, but um, the idea was that. Doing good data science or doing good AI development. In good meaning in an ethical sense. So being an ethical data scientist or being an ethical AI developer actually works to your benefit. Development wise in the long run. So one example of that is thinking about, you know, when I read those list of topics in that paper that I was mentioning.

• 32:28Daniel Whitenack?

  One of them had to do with like auditing and tracing and that sort of thing. So basically understanding like. This data was used to train this model, which output this prediction at this time. Based on this data from my user. And so all of that's sort of auditable and, and traceable. There's that data lineage. So if you put the work in and, and it's extra time, right? But if you put the work into putting tooling, whether that's just. Some good logging or whether that's some great, you know, there's several products now that allow you to. Trace experiments and um. And that sort of thing. But if you put the work in to implement those solutions.

• 33:10Daniel Whitenack?

  Then actually it does help you in the long run because as you're doing your own training. You know, being able to trace what hyper parameters you selected and trained on before and what. Data you used before to get these certain numbers. It actually helps kind of elevate your future work because you have a better understanding of what you did in the past. And you can track things much better over time. So I think the argument is that. You know, yes, it's faster to just spin up a notebook and do the thing and export the model and just, you know, copy paste it over to somewhere and run it somewhere in a totally non-traceable manner.

• 33:50Daniel Whitenack?

  But even outside of the ethical side of things, you're kind of shooting yourself in the foot at some point if you plan on doing that over and over again. And so there is some of this argument that, you know, it's slower to implement some of these things, but once they're implemented, I think you can have confidence that actually doing good or ethical work will actually work to your benefit development-wise as well in many, in many cases. I also think of like the scenario when I teach, I often, and I think I've mentioned this on the show maybe a time or two, like if you're developing a self-driving car in Sweden and you do, make a great product and then you want to export it internationally

• 34:34Daniel Whitenack?

  and you, you know, ship your car over to Australia and the first thing it sees is a kangaroo and it runs off the road and kills its, you know, passengers. That could have been solved by thinking about the biases in your data and the target markets that you're going after and what's represented in your data set and all of that. So these sorts of things can work to your benefit development and product-wise over time.

• 35:00Chris Benson?

  I totally agree with that. And I actually want to point a distinction is I know at the very beginning of when you were talking, you said governance. I would actually argue, having listened to you just now, that much of what you were talking about was really more on the compliance side. The two go together, but how do you know that as you're going through something that you are in compliance with what those values are without slowing you down? So I think I would start with the fact that this is a great place where you can use, in a lot of cases, technology to help you get there. And then on the process side, the governance side of kind of being able to handle when things don't come out as you expect. And, you know, you can go back and do that analysis. You know, when you were talking about tooling helping you and being able to, to know what

• 35:44Chris Benson?

  data was used when, as an example, what Came to mind as you were saying that was we've had pachyderm on the show. And if I recall, that sounds like some of the capability in the file system on that solution. So when you're implementing AI ethics, that might or might not, there may be others as well. But that might be one of your ways of trying to automate some of the abilities to be in compliant. So that you are not necessarily just slowing yourself down with a lot of manual intervention with this. And you have a governance process so that when unexpected outcomes do arise, going back to your kangaroo example, you can kind of go back and figure out, you know, what went wrong? And, you know, how did it deviate specifically from what you were trying to do?

• 36:29Chris Benson?

  I know for me, I look at, we've talked about a couple of industry principles. I tend to work a lot with the US Department of Defense principles, which are responsible, equitable, traceable, reliable and governable. And there's verbiage with each of those that our listeners can go look at. But from the standpoint of your kangaroo example, you can say, well, right off the bat, that would at least be a violation of responsible and reliable. And there's specific language that tells us what those two terms mean in this context. And so as you're looking at that problem that you just described, then you go back and you start tracing that back. And traceable happens to be another one of the five in terms of what goes back to allow you to assess what went wrong.

• 37:14Chris Benson?

  Why did that model give you an outcome in that situation that was not as expected? And that's what governance is right there. That is going back and saying, how do we fix the problems that we're going to have? Cause we certainly will have those. This is why you go through this whole process. It's to ensure that as you go forward, things do get better and better. And better means outcomes that you expect. And so I don't know if I answered everything that you posed to me at that one.

• 37:40Daniel Whitenack?

  Yeah, it's good points. A lot of what I'm thinking of with this is the sort of trickle down. And I think part of that was motivated by some of the teaching I've done on the subject. And oftentimes a question, when I'm entering into this sort of discussion, a question comes up about, well, if including a sensitive feature in my model, like let's say gender or whatever it is, improves the model. Performance in terms of the, you know, accuracy on my test set or whatever it is. Why would I not include that feature in the data? Why would I be careful about how I treat that feature in my data? Isn't the sort of accuracy what I'm most concerned about?

• 38:25Chris Benson?

  It might be. The answer is maybe on that. Go ahead and finish because I didn't mean to cut you off, but I'm going to come back to that.

• 38:30Daniel Whitenack?

  I agree with you maybe, but I think also that. So there's a paper that I was reading before a conversation today on the role and limits of principles in AI ethics towards a focus on tensions from a group at Cambridge. And they were talking about some of these natural tensions or trade offs. When we're thinking about ethics and one of those was this, you know, using data to improve the quality and efficiency of services versus respecting privacy and autonomy of individuals. And there's other of these tensions, but I think what it comes down to for me is that as AI practitioners, the things that drive that sort of performance and accuracy for us is data that has been generated in the real world, right?

• 39:15Daniel Whitenack?

  We parameterize our models based on observations that have happened and been recorded in data from the real world. And the bottom line is that our real world is broken in many ways, right? And there are very bad things that are happen in our real world. There's over representation and under representation that happens in our world. And depending even on just where you get the data or how you gather it, you know, you're never going to have the ideal data set. And so really what it comes down to is for me is, are you trying to get the best performance you can on your sample of test data, which is not representative of the whole world anyway?

• 39:59Daniel Whitenack?

  Or are you trying to ethically create a model that performs very well, but also at the same time does not discriminate or treat unfairly or use data in an unethical way? I think that those things have to be paired together. You can't, you know, from my perspective, it's not right, you know, to just take the performance at the expense of considering those other things.

• 40:25Chris Benson?

  I think not only I would agree with that. It's interesting. We've used privacy as our example a bunch of times. And in some contexts, you're looking and saying, well, if I had a performance capability that I can add in, but I do that at the expense of privacy, do I want to do that? That is exactly why you want to have those ethical principles in place. And while on first blush, that may seem self-evident to most of our listeners and to us as we sit here, but I'll give you an example of where it may not be as evident, you know, or at least you have to think about it just a little bit. There are cases where privacy is not necessarily one of the things that you're trying to achieve. An example of that, and this comes from obviously the larger world in which I work,

• 41:08Chris Benson?

  is if you have an intelligent satellite that does intelligent surveillance and reconnaissance, and its purpose is to go over for a nation and get as much intelligence as you can. That is very distinctly not something that is putting privacy as one of your values. You're basically rating that. So in that case, in that particular example that you gave us, the performance versus privacy, performance wins every time because of the nature of what you're trying to do and the governing principles that you're using. But if you were to say, I work for one of the large tech firms, Google or Microsoft or Amazon, something like that, and said, for performance, am I willing to give up privacy? Well, even if you personally were willing to give up privacy,

• 41:53Chris Benson?

  your customers probably aren't going to agree with you. And to do something like that would probably be devastating to your business, you know, particularly if you get caught doing something that your customers are not okay with. And we've seen examples of that many, many times. Facebook is another one that comes to mind as I talk through that. And so you're literally looking at the same set of value judgments, but coming out in very, very different ways, depending on how you're choosing to arrange those values in your principles. So the point I'm making is that having those principles really matters and the context of what your organization does with those principles really matters. And you can have very, very different outcomes based on those.

• 42:36Daniel Whitenack?

  Yep. I guess one thing I should say is we'll put links to some of the main industry AI principles that Chris and I have reviewed in preparation for the show. We'll, we'll put a link to those in our show notes along with the various papers that we're discussing. One other paper that drew my attention was talking about principles alone cannot guarantee ethical AI. And now that we're kind of getting down into how this plays out, I think that that was a very interesting paper that maybe people should take a look at. It appeared in Nature and it's talking about where we might fall short in terms of the AI principles that are coming out.

• 43:20Daniel Whitenack?

  One of those might be this sort of trickle down effect and creating a, you know, sustainable pathways to impact and accountability in terms of ethical AI, which I think comes down to more of an implementation thing. But they also kind of bring up this kind of interesting idea, which I've also heard other people talking about in terms of AI. And I don't know if I fully developed a thought process around, but they talk about licensing AI practitioners similar to like a doctor would have a license or something because their profession is very risky in terms of what they do. Or maybe a pilot license in terms of, you know, when you're flying people around, there's an inherent risk in that.

• 44:02Daniel Whitenack?

  And there are risks in terms of the AI that we developed, especially if we're talking about, you know, cars are autonomous or maybe systems that are implemented and. Airplanes, or you know, whether you get insurance or not, and that's driven by certain algorithms. I don't know if you have any thought about that.

• 44:22Chris Benson?

  I do.

• 44:22Daniel Whitenack?

  It kind of scares me a little bit because I don't know if I want to get a license, but I get where they're coming from. There's a lot of people that have put a lot more thought into this than myself.

• 44:33Chris Benson?

  Yeah, so I think I would note that we are still in the very, very early days of artificial intelligence. And the world in which we're operating right now is largely devoid of legal frameworks and regulatory frameworks. Now, before people start objecting and naming their favorite one, there are some, and I'm acknowledging that. But compared to other areas that may be regulated, we are still trying to figure out what that means. So national and local laws are, and international law is trailing far behind where the technical capability is. So I think that is one of the conversations that we need to have in the years ahead, is figuring out how to have the appropriate regulatory, whether it's

• 45:18Chris Benson?

  licensing, or whether it is other constructs that you may do. You know, our legal and regulatory frameworks are supposed to reflect our values in the world and the various things that we have to do to keep people safe and protect certain rights. And we have largely not done that, not only with AI, but with lots of different technologies in recent years. So there's a conversation and an adjustment that really needs to be had in the days ahead. And I think that's going to be pretty crucial. And it's funny, I know on this show, we are, you know, either AI practitioners, or we're at least minimally, by tuning in, you're an AI enthusiast in most cases. You care about this topic and

• 46:01Chris Benson?

  want to do that. And so it makes sense for us, our collective community here, to want to engage in this. But what I would actually urge you to do is because AI impacts so many people, and ultimately, most of the people in the world can be impacted in various ways, you really need to bring the people into the conversation that aren't necessarily As familiar with it as all of us are because it does impact them. They have a right to have a voice in it. And I think all of us, it's incumbent upon us to explain these issues objectively as scientists, you may say ourselves, to make sure that the general public understands what the implications of these things are. So we haven't really, in a meaningful way, started

• 46:45Chris Benson?

  that conversation yet. I'm hoping to see that in the next few years.

• 46:49Daniel Whitenack?

  Yeah, definitely. I'm hoping for similar things. As we close out here, this episode, of course, in these fully connected episodes with just Chris and myself, we always like to share some learning resources and maybe Chris has one that he wants to share. I'm not sure, but

• 47:05Chris Benson?

  I do, actually.

• 47:06Daniel Whitenack?

  Yeah, one that I wanted to share, which I've mentioned a couple of times on the show, which is just a very practical piece of tooling, which gets to maybe some of these, but not all of the ideas that we talked about, but at least gets started is the project from Driven Data called Deon or Deon, which is an ethics checklist for data scientists. And what I like about this is it's just not a static checklist. It's an actual Python project where you can embed a checklist in your Jupiter notebooks or in your Python code repository and actually have that there right with your code where you're developing your project. And it's fully customizable. So they

• 47:49Daniel Whitenack?

  have a sort of default one that you can use, but you can customize it for your own company's ethics checklist. So that actually, as you develop these things, you can implement them in an ethics checklist and embed that right in your project. So that every project, even if certain things in the checklist aren't relevant to a certain project, you can at least check them off that you've thought about them or considered them in completing the project.

• 48:14Chris Benson?

  That sounds like a good one. I know you've already noted that we're going to put links to a lot of these principles that various organizations have published out there. From a learning resource, and I'm going to suggest folks do what I have done in quite a bit of detail, is go to half a dozen different published principles that are out there. Go to the Googles, Microsofts, and others like that, and read what they've done and compare it, analyze between the two. Have them up on the screen at the same time with their explanations, and try to discern why they might have chosen the verbiage and the particular ways of describing it that each has done. And you'll find a lot of commonality, and you'll find some very distinct differences.

• 48:57Chris Benson?

  To do that, you might throw in the Department of Defense one, because it has a different thing that it's trying to achieve from a commercial entity. So it allows you to see a diversity of thought that has a lot of common points, but allows you to kind of put some critical thinking around it. And then as a second step, is after you've done that, choose what you think would be good principles for your organization, and try, do a little mind exercise where you think about how does this work in your own organization? How do you get it from the principles into your own organization's approach to work and the policies that govern that, through governance and the compliance that you're trying to implement into the tools and the workflow integrations that your organization is engaged in?

• 49:40Chris Benson?

  How do you get from that fluffy high level all the way into something that directly affects your customers or whoever your organization is working for? So that's, just going through that process is really educational. And then there's another one that I wanted to suggest. Not long ago, on episode 85, we had Stuart Russell as a guest on the show. He's a Berkeley professor. He's a legend in the AI community, has been doing this for decades in its various forms, and he's written several editions of Artificial Intelligence: A Modern Approach, which is kind of one of the key textbooks in this industry as it's evolved over time. And he has also, more recently, written a book that's intended for a general audience called Human Compatible. And

• 50:24Chris Benson?

  Read that book because it talks about some of these same issues and his own thoughts on how he would address it and what he thinks is important on that. And I know in that episode, I thought that was a fascinating conversation. I thoroughly enjoyed the conversation with Professor Russell. And so he really addresses AI that avoids harmful unintended consequences and offers a path forward towards a future in which humans can safely rely on provably beneficial AI. And at the end of the day, that's what you're trying to do with AI ethics. You know, that is the outcome you're trying to get to. So I recommend the book and hope that will get people started on this path.

• 51:00Daniel Whitenack?

  Excellent. Yeah, we'll put that link in our show notes. Please let us know in our community online, your thoughts around AI ethics and things that have been useful for you. You can find us on changelog.com/community and in Slack and LinkedIn and Twitter. And looking forward to hearing from you. Thanks for bringing your thoughts around this today, Chris. Really enjoyed the conversation and looking forward to talking to you next week.

• 51:25Chris Benson?

  It was good. Thanks a lot, Daniel.

• 51:26Daniel Whitenack?

  Uh huh. Bye bye.

• 51:31Speaker 02

  Have you joined the free changelog community yet? I'm not sure what you're waiting for. You get changelog news, email notifications of new podcast episodes, access to our community Slack and Practical AI channel where fun and interesting AI discussions take place all the time, all for the price of a free hot dog. Check us out at changelog.com/community. We'd love to have you. Practical AI is hosted by Daniel Whitenack and Chris Benson. It's produced by me, Jared Santo, and our music is provided by the mysterious Breakmaster Cylinder. We're brought to you by some amazing companies who get it. Thanks to Fastly, Linode, and Rollbar. That's all for now. We'll talk to you again next week.