Balancing human intelligence with AI

Polarity Mapping is a framework to “help problems be solved in a realistic and multidimensional manner” (see here https://universityinnovation.org/wiki/Resource:Polarity_Mapping for more info). In this week’s fully connected episode, Chris and Daniel use this framework to help them discuss how an organization can strike a good balance between human intelligence and AI. AI can’t solve everything and humans need to be in-the-loop with many AI solutions. Sponsors: SignalWire https://signalwire.com – Build what’s next in communications with video, voice, and messaging APIs powered by elastic cloud infrastructure. Try it today at signalwire.com https://signalwire.com and use code AI for $25 in developer credit. Changelog++ https://changelog.com/++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this https://changelog.com/++! Fastly https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com https://www.fastly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorshipLaunchDarkly https://launchdarkly.com/?utm_source=changelog&utm_medium=podcast&utm_campaign=changelog-sponsorship – Ship fast. Rest easy. Deploy code at any time, even if a feature isn’t ready to be released to your users. Wrap code in feature flags to get the safety to test new features and infrastructure in prod without impacting the wrong end users. Featuring: Chris Benson – Website https://chrisbenson.com, GitHub https://github.com/chrisbenson, LinkedIn https://www.linkedin.com/in/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaShow Notes: Polarity Mapping https://universityinnovation.org/wiki/Resource:Polarity_MappingBarry Johnson’s paper “Reflections A Perspective on Paradox and Its Application to Modern Management” https://www.semanticscholar.org/paper/Reflections-A-Perspective-on-Paradox-and-Its-to-Johnson/758d3dfe8842e4e39fd9c06f88ab7254133d8485Upcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars!

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Chris Benson?

  With the podcast that we have here, we have the privilege of meeting and talking with lots and lots of amazing people. I find myself very interested when people reach out in conversation where they have a really meaningful need and they're solving it with AI because it's the right solution. And conversely, the conversations that I don't find myself really gravitating to are when people are trying to sell their stuff based on AI. And rather than talk about the problem they're solving and why that matters to people out there, it's all about the AI. Hey, we have AI now. And I think that goes back to the fact that no matter what your job is, no matter what your role in the world is, we're all productively trying to solve things that need solving. I think AI is fantastic, but

• 0:43Chris Benson?

  it also needs to be the right thing to solve your particular problem. Whereas if it's just doing it because the marketing people say you need to do it, that's a good warning sign that you may be off track.

• 0:58Speaker 01

  Big thanks to our partners, Leno, Fastly, and Launch Darkly. We love Leno. They keep it fast and simple. Check them out at leno.com/Changelog. Our bandwidth is provided by Fastly. Learn more at fastly.com and get your feature flags powered by Launch Darkly. Get a demo at launchdarkly.com.

• 1:14Speaker 02

  Signal Wire is real-time video tech to help you create interactive video experiences previously not possible. It gives you access to broadcast quality ultra-low latency video that's proven and trusted by Amazon, Ring Doorbell, Zoom, and others. See why the future of video communication is being built on Signal Wire. They have easy-to-deploy APIs, SDKs for the most popular programming languages, and expert support from the OGs of software-defined telecom tech. Try it today at signalwire.com and use code AI for $25 in developer credit. Just visit signalwire.com. That's signalwire.com and use code AI to receive that $25. Once

• 1:59Speaker 02

  again, that's signalwire.com code AI.

• 2:12Speaker 02

  Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive, and accessible to everyone. This is where conversations around AI, machine learning, and data science happen. Join the community and Slack with us around various topics of the show at changelog.com/community and follow us on Twitter. We are at Practical AI FM.

• 2:38Daniel Whitenack?

  Well, welcome to another fully connected episode of Practical AI. This is where we will keep you fully connected with everything that's happening in the AI community. We'll take some time to discuss some of the latest AI news and topics, and we'll dig into some learning resources to help you level up your machine learning game. I'm Daniel Whitenack. I'm a data scientist with S I L International, and I'm joined, as always, by my co-host, Chris Benson, who is a strategist at Lockheed Martin. How you doing, Chris?

• 3:10Chris Benson?

  Doing very well, Daniel. It's a nice, uh, fall weather has arrived and-

• 3:14Daniel Whitenack?

  Yeah, it's rainy and gloomy here.

• 3:16Chris Benson?

  Oh, I'm sorry, man. We've had a, we've had a week of that.

• 3:19Daniel Whitenack?

  But it is cool. It's nice. It's not like an oven outside, which is good.

• 3:22Chris Benson?

  True. So I'm enjoying the weather at this point. We've had a week of rain and now it's so we get to jump into some AI and sunshine and, you know, we'll see how it goes.

• 3:32Daniel Whitenack?

  Yeah, that's good. You know, every week I introduce you as a strategist and sometimes I'm thinking, what does a strategist do day to day? Maybe my question is more like, how is strategy developed at a large organization? And this is related to what we'll talk about today, but I'm just sort of curious, like, how a large organization thinks about developing their strategy, maybe related to something like AI or a new advanced technology? What does the development of strategy look like at an organization?

• 4:09Chris Benson?

  So I would say the short version of an answer is that organizations have needs, and those needs- Needs have to be solved and most as simple as that sounds, it is very common for people not to start and say, what is the need I have to solve? They say, I have this cool new technology, man. It's incredible. It's called AI. And it's, you know, all these awesome things and we just need this in our product or service. It's amazing. And we call that an approach in the strategy world. It's, in other words, it is a solution in search of a problem.

• 4:46Daniel Whitenack?

  Yeah, it's something that could enable something, but it might just.

• 4:50Chris Benson?

  Yeah. And here's the funny thing. If you think back through your career, I'll bet that 99% of your technology conversations where people are excited about the way forward, start with them saying, I have this amazing thing. I'm so thrilled with it. And everyone will need one. If anyone tells you everyone will need this, that's a sure sign that you're starting in the wrong place. Everyone will need this. So you start with a need and you solve it with an approach. And that's how you get to a good answer instead of the other way around. So that's the short version.

• 5:20Daniel Whitenack?

  So are there tools or frameworks that you use? Maybe certain of these can't be revealed from your proprietary work, but are there like general tools or frameworks? That you use in developing strategy? I know, for example, you gave a talk last week at an event that I was helping organize, which thank you for doing that. There was a wonderful talk. I think you mentioned some sort of frameworks that you think about. I'm always intrigued by the different ways people frame their strategy. What are those, some of those tools that you use?

• 5:54Chris Benson?

  It's actually going to follow through on what I was just talking about is that a lot of approaches to strategy are incredibly complex. And because of that, no one sticks with it. There's a lot of, you know, mundane documentation and stuff. There's a really simple framework that I use as baseline and we'll build around that as we need for specific business cases. It's called NABC, which is Need Approach Benefit Competition, where you're, start with a need and you're looking at what are all the possible approaches? In this case, they may be AI approaches. And then you figure out how those benefit a solution. And there may be a lot of options there and what the competition is between them. And you try to get to an honest answer. And that very, very simple framework is how I start my thinking around it.

• 6:38Chris Benson?

  And then we do have various proprietary, you know, business specific frameworks, but I always come back to that because it's so simple that it's hard to go wrong if you keep it in your brain.

• 6:47Daniel Whitenack?

  Yeah, and this gets, I mean, on these fully connected episodes, we're always talking about maybe the latest AI news or a new capability that comes out, or a new type of method, or new data. And definitely those things separate from like a strategic mindset. I mean, in the very least, they could not bring value, right? In the very most, maybe they bring harm if they're sort of misapplied.

• 7:14Chris Benson?

  Yeah, suboptimal outcomes are incredibly common. And so even if it's a simple framework, it takes a lot of discipline. You'll put something like that out there and people will kind of nod and go, yeah, yeah. Nice, simple four-letter acronym. I got it. And they move on, but then they go do the wrong thing right after that.

• 7:30Daniel Whitenack?

  Yeah.

• 7:30Chris Benson?

  And you're like, no, no, we're going to come back and solve the need that we actually have identified and is validated.

• 7:36Daniel Whitenack?

  Yeah.

• 7:36Chris Benson?

  And that's hard to do. It's deceptively simple in speaking it.

• 7:40Daniel Whitenack?

  Yeah, and I think that maybe a lot of the conversation and blog posts or like conversations I've had have been centered around can we justify the use of AI here? Or is AI the strategic choice for this application for XYZ reason? But I think that, so something happened actually earlier today in my own work where someone asked me to do a bit of analysis of the polarity or the balancing of two things, human intelligence and artificial intelligence. And I thought this was an interesting question that maybe I hadn't fully thought through yet,

• 8:20Daniel Whitenack?

  which is not so much like, is AI going to bring value for a particular solution? But how do you weave together human intelligence and human expertise with artificial intelligence and balance those two in a good way for your organization or for your particular problem that you're looking at? So that's what I was maybe gonna bring up today. Would you be interested in kind of going through that exercise with me, helping me in my homework assignment?

• 8:50Chris Benson?

  Absolutely, but I feel since you have taken me down this path, I feel obligated to point out that we're kind of starting with an approach there. So maybe we pick a use case and we kind of talk our way through that use case because then we have a validated need to have context for it.

• 9:06Daniel Whitenack?

  Sure. So yeah, I mean, I think in general, we could use the use case, which is definitely the case in, although we're talking in a little bit more specifics on the project that I work on, we're generally talking about the application of AI in local language contexts. So let's say there's languages where something like chat dialogue technology, sort of like named entity recognition, sentiment analysis, that sort of stuff isn't being applied yet. Maybe it's machine translation, which doesn't support these local languages yet. Maybe it's speech recognition. The premise is that the local language community would benefit from these types of technologies

• 9:51Daniel Whitenack?

  being extended to support their languages, but also that the AI community at large would benefit from local language communities actually being part of that natural language processing conversation in a wider way. So that's the sort of context that we're working in is this sort of AI technology, should it be applied to do things? We can kind of scope it down. Should AI technology be applied to support machine translation for all these local languages? Or speech recognition for all these local languages where it's not currently supported? How does that work? Putting your strategist hat on.

• 10:30Chris Benson?

  So I think it really depends on the stakeholder upfront in that if you're an organization, either a commercial or nonprofit, not-for-profit organization, either way, then there is a mission, there's an objective that your organization has, and you have to figure out whether that use case is going to fit in. If you are a large internet-based company, you know, and without getting into specific names, and those, and a significant proportion of your users are in those local languages, it would make sense for you, but it would not make sense for you necessarily, but it might make sense for another organization that is trying to help that. So what I'm saying is there's a cost to be borne, and you have to figure out where the cost should be borne so that you can serve all users well.

• 11:12Chris Benson?

  And so I think there's room for everyone in getting in on that. You just have to figure out where that goes.

• 11:18Daniel Whitenack?

  Yeah, and I mean, you could think of a scenario like the most glaring one that we had recently with COVID and this sort of massive need for around the world.

• 11:29Daniel Whitenack?

  Rapid translation of COVID-related information into as many languages as we could get it into.

• 11:37Chris Benson?

  Yep.

• 11:37Daniel Whitenack?

  Because there was an immediate need. Now, the question comes up, well, what role does AI play in that? And how is it balanced with human intelligence in the solution to that problem? What are the advantages of one and the disadvantages of the other? And how should they be balanced together? The specific thing that came across my desk today was this idea of a polarity mapping, which I kind of like because my background's in physics. So anything with polarity is probably okay. I just chatted you a link to that and we'll link it in our show notes. But there's this sort of what I would consider a goofy looking picture.

• 12:19Daniel Whitenack?

  But there's this framework, a framework for tracking this sort of problem. I guess according to this Wiki article, it was created by Barry Johnson, who I don't know who that is.

• 12:29Chris Benson?

  I don't either.

• 12:30Daniel Whitenack?

  Barry Johnson, there's probably a million Barry Johnsons. But Barry Johnson, and it was created to help problems be solved in a realistic and multi-dimensional manner. Now, normally, because I'm a practical person and I like to spend most of my time in FEM and, you know, in a collab notebook or something. I really sort of cringe when I see these kind of like innovation frameworks or innovation map type things. They sort of like, I don't know, they give me a bad feeling right away.

• 12:58Chris Benson?

  Hey, I'm a strategist and they give me a bad feeling too. So I'm right there with you.

• 13:02Daniel Whitenack?

  They're also like, I mean, it's kind of a goofy picture. It almost looks like maybe what Prince would have on some of his like stage art at a show or something. I don't know.

• 13:13Chris Benson?

  With lasers, you have to have lasers.

• 13:15Daniel Whitenack?

  Yeah, maybe. So the idea of the picture, I think, is there's like these four quadrants. And you can follow up in the show notes. But the idea is that sort of on one side, you've got human intelligence. It's really a framework for comparing two things. But on one side, you've got maybe human intelligence. And then on the other side, you've got artificial intelligence. And Then what you would want to do is think about the positive results and the negative results from each, or like the pros and cons of each.

• 13:48Chris Benson?

  Yeah.

• 13:48Daniel Whitenack?

  And then in looking at the pros and cons of each of those, think about the action steps and warning signs when they aren't in balance, or how you can keep them in balance. Because we're assuming that both will play a role. So in our context, like in the machine translation of COVID information into local languages, how does human intelligence play a role? How does artificial intelligence play a role? And when and how would we know if one is out of balance in comparison to the other one? So does the premise make sense?

• 14:21Chris Benson?

  The premise makes sense. And I think it's, I think it's an interesting topic that I think is very relevant to our future.

• 14:26Daniel Whitenack?

  Yeah, definitely, definitely. So maybe we'll talk about, let's think about the, the values of each of these first. So on the top of these quadrants, what they have, the user of this framework do is think about the values, which they've labeled as the positive results from focusing on. And then you fill in the blank with whichever one you're thinking about, human intelligence or artificial intelligence. So what are some positive results from focusing on human intelligence when you're trying to solve this problem of machine translation into a local language? What do you think? Any ideas?

• 15:02Chris Benson?

  The target, if you will, of what you're trying to accomplish is a human. And so having a human involved automatically allows you to connect. There's a whole bunch of problems that just don't exist. And you get the benefit of us complicated humans in terms of not only do you get the benefit of communication and understanding, but you have things like empathy. And I, there's huge value in that.

• 15:26Daniel Whitenack?

  Yeah, so a human is the target of this technology. So that's definitely good to keep in mind. I think, like you were talking about earlier, we can get enamored by the tech. Right?

• 15:40Chris Benson?

  Yes.

• 15:40Daniel Whitenack?

  And think about like, this is a cool solution to a thing, but ultimately a human's gonna interact with it, right?

• 15:48Chris Benson?

  Absolutely.

• 15:48Daniel Whitenack?

  So I don't know, do you think about that like term of like empathy? When you're thinking about strategy? I've heard that term thrown around.

• 15:57Chris Benson?

  I do. So not everyone does think that way, but it is value. And the older I get, the more I care about that. Most problems, there's the basic problem and there's a whole bunch of surrounding concerns around that. And you have to figure out what you're going to address. So for me, I see benefit to there.

• 16:29Speaker 02

  Change log news is the best way to keep up with the fast moving software world. We track, blog and contextualize the coolest projects, the best practices and the biggest stories each and every week. Make change log.com your daily destination or hit the snooze button and subscribe to our weekly newsletter that hits inboxes on Sunday mornings. Join more than 15,000 enthusiastic readers. It'll cost you exactly $0 and you can subscribe right now at change.com/weekly.

• 17:13Daniel Whitenack?

  So Chris, I think you brought up a couple of interesting things related to the positive results from focusing on human intelligence to solve the hypothetical problem we're considering. So humans being the target, it sort of breeds trust, maybe? So I'm going to bring up the word trust. It maybe brings trust when a human does the thing. So like if I'm translating material about COVID into a local language, if I have a human that does that translation, maybe there's more trust there? I don't know. Would you agree?

• 17:47Chris Benson?

  I think it would be a fair statement to say. Most people are able to connect with other humans long before they connect with a particular technology. There's an adjustment period to new technologies being accepted. We've seen that in many, many cases of technology release and development over the years. And so that's a big concern. You can look back and say kind of the classical, my grandmother doesn't know how to use a cellphone kind of context. So yes, you are circumventing a whole set of issues and by taking advantage of the fact that you inherently will probably have trust with the human to human contact.

• 18:23Daniel Whitenack?

  And the other thing you mentioned, I think is maybe related to like flexibility or adaptation. So like a human, we're used to adapting to all sorts of situations, right? But generally, like an AI model for machine translation, there are good general purpose models, but oftentimes there's certain domains, like if we think about this domain of translation of COVID material, there's really not that much in public corpora out there, public data that is representative of COVID type of information. And so machine translation model might just simply not know that domain of translation, whereas it's maybe fairly

• 19:06Daniel Whitenack?

  easy for a human to adapt to translating COVID information in a very quick period of time. So we're adaptable, I guess.

• 19:15Chris Benson?

  Absolutely. I agree.

• 19:16Daniel Whitenack?

  Now, depending on, and this is, of course, there's a whole range of philosophical and religious positions on this, but at least I would argue that humans as opposed to machines, like there's a difference between humans and machines. Now, like I say, depending on your worldview, you might think of that in different ways, but I think we could probably all agree on the fact that at least today's machines have a very task focused way of solving problems and humans have a natural creativity, productivity, adaptation. There's a different element about how they solve.

• 20:00Daniel Whitenack?

  A problem as opposed to a machine. So I don't know if that's creativity in addition to adaptation. I don't know if you have thoughts there.

• 20:07Chris Benson?

  I think that's right on is it's the complexity inherent in the human mind that allows for the adaptation. And in many cases for those, it makes those interactions interesting. I mean, a good example of that is if you look at humor, this machine generated as of today in 2021, I've yet to find a source that I find funny, and yet you and I can have a conversation. Neither one of us is a professional comedian, despite evidence possibly contrary at times, but we have a good time.

• 20:34Daniel Whitenack?

  I'm definitely not.

• 20:35Chris Benson?

  We can laugh at each other's jokes and that's something that's uniquely human at the moment at least. And yet, on the other side, as you certainly know, and maybe some of our listeners, I recently got my private pilot's license. And I would argue that it's an incredibly manual procedural thing to learn. And most pilots would probably disagree with me, but I actually think that technology can fly a lot better than we can. I know that I am prone to make small mistakes. Thankfully, they are small, but there's ways that by doing it very procedural, very task oriented, that you have superior capabilities from technology.

• 21:11Daniel Whitenack?

  So I guess getting to that and thinking about the positive results from focusing on an AI approach to solving this machine translation problem, what comes to mind first?

• 21:23Chris Benson?

  You'll have to give me an example to go for that.

• 21:25Daniel Whitenack?

  So like if we didn't have a human in the loop at all, and we had a great AI that could, or we thought could solve this problem of machine translation of COVID information. What are the benefits of that AI solution in isolation with no sort of human expertise or intelligence infused?

• 21:45Chris Benson?

  Once you've absorbed the cost of that, assuming for a moment, constant, you know, static model based on where you, what your development was, then you can deploy it at scale without incurring additional cost. And not only that, but you can, because of that, you can deploy it much more widespread. There's a very limited number of human translators out there. And so there are costs associated with that. And so, so long as you're willing to take the cost of the development into account and the cost of deployment into account, then you can scale a certain capability across a very wide group of users. And that's a huge benefit. I mean, that we see that in all sorts of business use cases.

• 22:26Daniel Whitenack?

  Yeah. And when I teach classes or workshops normally, the question comes up like, when is an AI solution appropriate or under what conditions? And generally, I do think about that in terms of two things. One is scale. Even if a human, which I think a human would in most cases, if they knew both languages, create a better translation. But if we have to translate four million sentences, the fact is that it's going to take a human an incredible amount of time. And so there's a scale factor and then there's also a complexity of the problem factor. So like, some problems, just by their very nature, even if they're not at scale, are hard for a

• 23:06Daniel Whitenack?

  human to visualize. And maybe it's combining a bunch of sensor data from IoT devices to determine when anomalies are happening in a network or in a manufacturing system or something. Those are like really complex problems. It's hard for a human to actually parse all of that information and make a decision. Maybe they could retroactively, but in the moment, it's difficult. So there's the scale of complexity, but then also the scale of scale.

• 23:33Chris Benson?

  I get what you're saying. I'll offer an alternative point on that one is in translation, if you're looking at that you have a string of words that you're trying to translate, there may be context around those words, though, that is not captured by the technology. If you're, you may be in a, we've all lived this pandemic the last couple of years and, you know, there are all sorts of challenging situations that humans have to go through. And sometimes the human can take into account things outside the direct task at hand, which shape the world, the entire interaction is happening within. You know, there's a trade off either way. In this case, you just identified an excellent reason to apply technology. And then we also can turn around and say, there's a great reason to have a human in the loop. So it kind

• 24:17Chris Benson?

  of depends on how you want to value those different attributes.

• 24:19Daniel Whitenack?

  Yeah. I also think that if we think about the combination, and maybe this is the, I don't know if it's a pure AI value, but there, one feature of the AI side of things is that there are many processes where the Combination of the AI model plus a human. So the human plus computer is actually both faster and higher quality in its task than either one in isolation. And this has been true in like healthcare applications where

• 24:50Chris Benson?

  Absolutely.

• 24:50Daniel Whitenack?

  Doctors are recognizing tumors or whatever it is. So there is this element of the two together can produce actually higher quality results than either one by themselves.

• 25:01Chris Benson?

  Absolutely. In my industry, we have, I think, a unique term for that, and that is man unmanned teaming is what we call it. We call it MOM-T. So there's a new acronym for you.

• 25:12Daniel Whitenack?

  Okay, yeah, I'll make sure and fit that in.

• 25:16Daniel Whitenack?

  At a few points.

• 25:16Chris Benson?

  There are great outcomes that can be had by taking advantage of the strengths of both sides.

• 25:20Daniel Whitenack?

  Yeah, so actually, I think we've already sort of got into some of the downsides. But I think it is interesting to think about on both sides of things. Cause a lot of times people think about the downsides of AI. Like whether that be bias or whatever.

• 25:37Chris Benson?

  Yeah.

• 25:37Daniel Whitenack?

  But what are the negative results from over focusing on human intelligence rather than bringing in any AI?

• 25:46Chris Benson?

  Right off the bat, the same thing you just mentioned applies to both sides, and that's bias. And we in data science tend to focus on bias in data and data processing, you know, when we're building models. But oh, boy, all you have to do is look at the last election cycle with humans. And you see that we have the same faults in that way. So that's one right off the bat. And another one that you mentioned earlier was the fact that humans don't necessarily take all of the data in and give it all processing time before an output. If you're looking at all these sensors that you mentioned earlier.

• 26:21Daniel Whitenack?

  Yeah.

• 26:22Chris Benson?

  Some of the things that make us so strong in some areas make us terribly weak in others.

• 26:27Daniel Whitenack?

  Yeah.

• 26:27Chris Benson?

  A human inference is very different from a computer inference, and they both have strengths and weaknesses.

• 26:32Daniel Whitenack?

  One of the interesting thing about bias on the human side, which someone, my coworker, brought up to me today is he said, on the human side biases are covered by shame. And I was like. Okay, well, like unpack that a little bit for me. What do you mean by that? And he was basically saying people hide their mistakes, right? If you have a test set on the AI side, like you can just count up how many you got wrong. Now, there could be bias in that and you can actually measure it, right? But a human oftentimes, either they subconsciously have a bias and it's hidden because they just don't even know that they have it.

• 27:15Daniel Whitenack?

  Or they do know that they have a bias and they intentionally try to hide it, right? Which is maybe a more interesting situation. So when a human makes a mistake, they generally like people to know that they didn't make the mistake. So it's really hard to measure sometimes when people are making mistakes and what their actual perception is and how a situation went down. And so there is this, in addition to there being bias on the AI side, there's this interesting kind of bias on the human side that's really hard to measure and deal with.

• 27:49Chris Benson?

  Yeah, you know, a few minutes ago, we were talking about these external concerns for whatever your primary task or motivation is. And when we talk about on the model side, we say, well, a model is creating an inference to solve a particular task. And, you know, unfortunately, it didn't take into account the external environment that it was operating in. And so it didn't have empathy, it didn't have other, other attributes that we might, we might have value for. But at the same time, that's also the strength of it, as you just pointed out, in that you are getting the benefit of that inference as it should be. Whereas we humans, we want to look good, we want to sound good, we don't want to make mistakes, we want our peers to like us. And that creates a whole set of concerns that change the way that we're both in the interactions

• 28:34Chris Benson?

  and the way that we're communicating this.

• 28:35Daniel Whitenack?

  So we have, I think, on many episodes, highlighted some of the downsides of an over-focus on AI solutions. So I don't know that we need to go into incredible detail on that. I mean, some of those are this sort of black box side of things where you have a lack of interpretability, which Those problems with trust and debugging and all of those sorts of things. You've got sort of disillusionment of people where they hype up AI and you know, it's actually not as great as they think. Of course, AI systems depend on data generated by humans and humans have bias. And so that data has bias and humans kind of infuse that sometimes into models.

• 29:20Daniel Whitenack?

  And you know, in general, it's just hard to, because AI is hard to explain. It's hard to trust and it's hard, maybe also in this context, we're talking about translation into local languages where this sort of power to create these AI systems might be centralized in large tech companies, in big GPU clusters and not accessible to local language communities. Although I think that is rapidly changing. There's evidence throughout like efforts like Masakane in Africa, where people are doing amazing AI research. People from local language communities are doing amazing AI research with things like Google Colab. So I think there's a balance, of course, to all these things.

• 30:05Daniel Whitenack?

  But anything else you'd like to highlight on that sort of over focus on AI side of things?

• 30:10Chris Benson?

  I think for me at least, having spent several years thinking about this stuff with you and others, I am now in the habit of trusting AI that solves procedural and very task oriented things. And I've seen so many cases where it's doing it better than humans, even if it's a series of tasks that together create a complex task like flying an airplane. But you know, I think I'm probably in the, very much in the minority, but I would actually be very comfortable in an aircraft that was mostly modeled, maybe entirely model driven without a human at the wheel, so to speak. And I don't think most people are there yet. I don't trust AI to handle things that are complex and nonlinear

• 30:54Chris Benson?

  and where there are many external concerns that can influence a situation. And that's kind of where I've arrived after several years of thinking about it. That's kind of how I'm looking at the trust issues.

• 31:05Daniel Whitenack?

  So what do you think would be a symptom or an early warning sign, maybe, that you are over focusing on human intelligence or that you're over focusing on artificial intelligence? What would be a symptom of either one of those conditions? That's another part of this kind of framework is, hey, how would I actually know if I'm over focusing in one of these areas?

• 31:31Chris Benson?

  So the way I would arrive at that is I would not start with those things. I would not start with the human intelligence and what I think of it versus machine intelligence. I would start with what I'm trying to solve. And I would think, what are the kinds of things in the abstract that will solve this? And then we've called out some fairly substantial differences in those two sides, the human side versus the machine learning side. And I think that there are characteristics that kind of automatically lend themselves to one way or another. And I think I would arrive that way. I think the biggest warning sign, going back to the very beginning of our conversation, is starting with an agenda, which we call an approach. Starting with an approach and trying to hammer that into whatever it is that you're trying to solve,

• 32:16Chris Benson?

  you go to the other end. And so if you feel like you're hammering something in, it doesn't quite fit, it probably means that's exactly what you're doing. And you should reassess and go back to what you're trying to solve and figure the characteristics out. And then which way do you go with that? It leads to a natural solution, quite honestly.

• 32:32Daniel Whitenack?

  Yeah, maybe that's part of, on the overemphasis on the AI side, maybe that's part of it. That is where you have this inclination to love your AI tech solution. And you are the one that has created it apart from these end users. You know, these people you're doing the translation for, and you're just sort of giving the output to them. And they're not using it, they're not consuming it, it's not being adopted. Well, that's probably a sign that, hey, you haven't involved them, maybe?

• 33:07Chris Benson?

  Yeah.

• 33:08Daniel Whitenack?

  Maybe what you're doing isn't as great as you think it is. Like you were saying, you're trying to sort of force a solution. On another target audience that maybe you don't haven't really involved from the beginning. Of course, if there isn't a disadvantage to always involving everybody from the start, right? It makes things slower, right? It makes things slower, more costly. And so it is a balance there. I realize that, but maybe that is one of the early warning signs.

• 33:38Chris Benson?

  I'll give you an ironic answer for that. It's with the podcast that we have here. We have the privilege of meeting and talking with lots and lots of amazing people. But there are also people out there with agendas that'll reach out. And in my day job, I also have the same thing about, you know, people trying to reach out. I find myself very interested when people reach out in conversation where they have a really meaningful need and they're solving it with AI because it's the right solution. And conversely, the conversations that I don't find myself really gravitating to are when people are trying to sell their stuff based on AI. And rather than talk about the problem they're solving and why that matters to people out there. They just, it's all about the AI. Hey, we have AI now.

• 34:22Chris Benson?

  And frankly, ironically, on an AI podcast, I find myself getting very bored with that conversation very quickly. So I think that goes back to the fact that no matter what your job is, no matter what your role in the world is, we're, we're all productively trying to solve things that need solving. And I think AI is fantastic, but it also needs to be the right thing to solve your particular problem. And that makes it a fascinating conversation. Whereas if it's just doing it because the marketing people say you need to do it, that's a good warning sign that you may be off track.

• 34:54Daniel Whitenack?

  And the other question here in this framework, in addition to those early warning signs, they're asking, how will we gain or maintain the positive results from AI without sort of over focusing on AI and maintaining some balance with human intelligence? Any thoughts there?

• 35:16Chris Benson?

  Kind of what we already said. I mean, there, humans are amazing. And as much as we talk about how amazing current deep learning technologies are in the AI space. Humans are amazing too. And if you can optimize both sides and you can find that teaming between the manned and the unmanned side and find a way for them to fit together to serve whoever it is they're trying to serve, that is a really, really good approach. And if you, if you over-focus on one or the other for whatever your alternative reasons, then that tends to send you off.

• 35:47Daniel Whitenack?

  Yeah, and maybe the way, or a way to shift our thinking in, in this respect is to, when we think about one of these solutions like machine translation for COVID information, we should be thinking from the start maybe about who is the human that needs to be in the loop in this process? And what is their interaction with the AI model? Rather than simply saying, how can I create the best AI model? Which is normally where we start. And to be honest, that's normally where I start. Just out of habit and, but I definitely know that I need to be more focused on that human in the loop element.

• 36:29Daniel Whitenack?

  And it definitely is important and I think can help maintain that balance.

• 36:33Chris Benson?

  I agree with you. And I think COVID is a great, since we're talking about that as our use case, it is such a strong reason to have language models in many, many languages. And what that does is you can let those models do that work because they can, because it can be very procedural to do that. But there's also a role for humans there and that all of that communication is happening inside a context. It's happening inside an environment that are people's lives and their emotions and their, and their relationships. And I think that's a really good example of where you can scale language across many, many user groups effectively. And yet there's still room for people to be there to add that human element that is so necessary, especially in, in times like this.

• 37:17Daniel Whitenack?

  Yeah, for sure. Well, we will link this polarity map into our show notes in case you want to go through this exercise with your own team or thinking through various other balances that need to be had in technology. I would recommend taking a look. We do normally share some learning resources at the end of our episodes. I've got one I wanted to share, which I don't know how popular it would be, but I hope it's popular. So normally we share like machine learning courses or something like that, but I think I have someone early on in my, oh, I actually know, shout out to Manish, who is the CEO over at D Graph.

• 38:02Daniel Whitenack?

  In one of our early conversations, this was quite a while ago, he told me that one of the biggest improvements in his own development, professional development as a software engineer, was someone helping him understand that putting time and focus on to his code editor, his IDE, and really understanding that very deeply was an extremely important element of development and can really give you a huge boost in terms of your work. And I don't know if you've found that to be true, but I definitely have found that to be true over time. So my code editor is Vim. I know it's not very cool. I don't use like VS Code or anything,

• 38:43Daniel Whitenack?

  but there's a website called Vimcast.org. I learned about this from a recent couple of episodes on the Change Log Podcast. And this is just like a great wealth of info, but also that Vimcast, the guy who runs that podcast, he also has a course about Vim and the core Vim course. And I've been going through that and I've really been enjoying it. So I wanted to mention that on the podcast. Thank you so much, Drew at Vimcast for putting together this great course that is benefiting me a lot, but also another podcast that I love listening to and learning from. So yeah, just wanted to give a shout out there. I don't know if you found that to be true as well in your own work, Chris, with your code editor?

• 39:28Chris Benson?

  Yep, I do. I am not tied to one strictly, so I kind of bounce between several. I use Vim some, but I also, I think these days I'm probably using Visual Studio Code. Most.

• 39:39Daniel Whitenack?

  Seems to be the popular solution.

• 39:41Chris Benson?

  Yeah, I've kind of moved over to that one as well. If I'm on a Linux server, you know, doing something, I'm always in Vim.

• 39:47Daniel Whitenack?

  I mean, that's what a Vim person would tell you. Like, if you go to a Linux server, Vim will be there and then you're not crippled anymore. But, you know, that's also sort of like VS Code is pretty amazing.

• 39:58Chris Benson?

  It is. I agree with you. I'll offer up one as well. I'll go back to what I was talking about at the very beginning of the conversation. The NABC Value Proposition Framework, which is beautiful in its simplicity and allowing you to stay on target and keep disciplined about it. It was actually created by a guy named Kirk Carlson, and he has a website at practiceofinnovation.com/nabc-value-proposition, and we will link to it in the show notes. But that is a good place to go and learn a little bit about it. And I think that's why it keeps it straight and simple. And that means that when you're in the middle of a conversation, it allows you to stay on target. So that's what I'll offer up for a strategy.

• 40:41Daniel Whitenack?

  Awesome. Thanks, Chris. I appreciate you letting a novice like me enter into the strategy world

• 40:49Daniel Whitenack?

  and operate where I'm not qualified to operate.

• 40:52Chris Benson?

  Oh, boy. Oh, gosh.

• 40:53Daniel Whitenack?

  So thanks so much, Chris. Hope you have a good week and we'll talk to you soon.

• 40:58Chris Benson?

  Sounds good. Thanks, Daniel.

• 41:02Speaker 02

  Thank you for listening to Practical AI. We have a bundle of awesome podcasts for you at changelog.com, including our brand new show, Ship It with Gerhard Lazu, a podcast about getting your best ideas into the world and seeing what happens. It's about the code, the ops, the infra, and the people that make it happen. Yes, we focus on the people because everything else is in implementation detail. Subscribe now at changelog.com/shipit or simply search for Ship It in your favorite podcast app, you'll find it. Of course, the galaxy brain move is to subscribe to our master feed. It's all changelog podcasts, including Practical AI and Ship It in one place. Search changelog master feed or head to changelog.com/master and subscribe today.

• 41:45Speaker 02

  Practical AI is hosted by Daniel Whitenack and Chris Benson, with music by Breakmaster Cylinder. We're brought to you by Fastly, Von Starfleet, and Linode. That's all for now. We'll talk to you again next week.